Migrate to JSON schema v2

Linked to https://www.ebi.ac.uk/panda/jira/browse/EVA-2324. Reported by Kostas Tsirigos on 2021-02-03 via email:

Dear EVA team,
We are very happy to share with you our new JSON schema which should be used from the upcoming release (21.04 – April 2021) onwards. There are major changes in the schema. We encourage you to start the migration process as early as possible and raise any questions or problems you might encounter. You will find more details specific to you and your resource below. The  last version of the JSON schema can be found here, while the validator to use alongside can be accessed here.
The necessity for a new, rewritten, JSON schema is part of our rewrite project, which will also come into effect with the 21.04 platform release (April 2021). The rationale behind this change is to streamline the evidence data-model across data-providers and improve the performance of our pipelines. The data you provide to us will now have a flatter format and hopefully it will require a simpler code-base on your side. We encourage you to read the schema documentation to understand what each field is about.
Some key points regarding the new JSON schema that we would like to draw your attention to are the following:
· You might notice that the new flatter schema has a lot of fields, but are not yet available to your resource. If you think you have data for any of these fields, please contact us and we can make them available to you.
· JSON schema validator: we strongly encourage you to please indicate whether you have validated the evidence strings and with which version of the schema. Please try to use the  latest version of the schema at all times. Also, let us know if you have any issues running the validator and we can assist you in the process.
· The unique_association_fields field does not exist in the JSON schema anymore. We will account for the uniqueness of the evidence based on some common policies. We hope this decision also simplifies the process on your side.
· We no longer support the target_id field. This term is reserved and will be added on our side. Instead, we expect from you to provide targetFromSourceId which corresponds to the internal identifier used on your resource to refer to gene/transcripts/proteins, whether is an Ensembl gene identifier, a Uniprot accession or an approved gene symbol by HGNC. The values you submit to us will be subsequently mapped to Ensembl gene identifiers using a common pipeline across datasources.
· In an analogous manner, the disease_id field is not available anymore. We encourage you to populate the field diseaseFromSourceMappedId with the result of your mapping/curation to EFO. Additionally, some other fields related to disease/phenotype/traits characterisation are available to you. diseaseFromSourceId can be populated with any internal ID your resource uses to refer to diseases/phenotypes/traits (e.g. Mesh, ICD, etc). Importantly, this field should not be the result of an additional mapping process, just the raw representation of the trait. Similarly, if a raw description of the disease/phenotype/trait is available, this can be made available in the field diseaseFromSource. Ultimately, if there are a set of phenotypes that characterise the set of individuals the evidence is based on, this can be available in cohortPhenotypes.
· To illustrate the migration process, we provide below (also attached to the email) an example of how a JSON object from EVA looked before and after migrating it to the new schema.
Some new data we would like you to include in the JSON following the schema:

diseaseFromSourceId - this field shall include, if present, the disease ID captured at source by the EVA team during the curation process. We ignore whether there is any standardised disease ID that you use internally. If such an ID is not available, please ignore this comment.

cohortPhenotypes - as discussed in issue https://github.com/opentargets/platform/issues/1013, this field shall include an array with all the different phenotypes describing the RCV.

diseaseFromSource - as discussed in issue https://github.com/opentargets/platform/issues/1013, this field shall include the first phenotype from the cohortPhenotypes array in alphabetical order. While this decision might look arbitrary, we believe it is enough to have a string representation of the disease.

variantId -  this field shall include an identifier of the variant using CHROM_POS_REF_ALT notation.

variantFunctionalConsequenceId -  this field is currently absent in eva_somatic evidence. We would like to include the field similar to the eva datasource including the functional consequence of the variant.

We have attempted to cover the issues opened in EVA and OT platform github trackers. However, there might still be pending issues to consider that might not be totally resolved by this work. For complex issues, we would encourage you to migrate the evidence first and we can iterate on further improvements at a later stage.
We are at your disposal to further discuss the schema, should you have any questions that we can resolve and/or suggest fields (data) that we can be included on top of the ones we have already.
Best regards,
Kostas Tsirigos on behalf of the Open Targets team

Before
EVA germline
{
  "type": "genetic_association",
  "access_level": "public",
  "sourceID": "eva",
  "variant": {
    "type": "snp single",
    "id": "http://identifiers.org/dbsnp/rs1021025464"
  },
  "validated_against_schema_version": "1.7.3",
  "disease": {
    "id": "http://www.orpha.net/ORDO/Orphanet_905",
    "source_name": "wilson disease",
    "name": "Wilson disease"
  },
  "target": {
    "target_type": "http://identifiers.org/cttv.target/gene_variant",
    "id": "http://identifiers.org/ensembl/ENSG00000123191"
  },
  "unique_association_fields": {
    "gene": "ENSG00000123191",
    "clinvarAccession": "RCV000626321",
    "phenotype": "http://www.orpha.net/ORDO/Orphanet_905",
    "alleleOrigin": "germline",
    "variant_id": "rs1021025464"
  },
  "evidence": {
    "variant2disease": {
      "is_associated": true,
      "evidence_codes": [
        "http://purl.obolibrary.org/obo/ECO_0000205"
      ],
      "urls": [
        {
          "nice_name": "Further details in ClinVar database",
          "url": "http://www.ncbi.nlm.nih.gov/clinvar/RCV000626321"
        }
      ],
      "provenance_type": {
        "database": {
          "dbxref": {
            "url": "http://identifiers.org/clinvar.record/RCV000626321",
            "id": "http://identifiers.org/clinvar",
            "version": "2017-08"
          },
          "id": "EVA",
          "version": "1.0"
        },
        "expert": {
          "statement": "Primary submitter of data",
          "status": true
        },
        "literature": {
          "references": [
            {
              "lit_id": "http://europepmc.org/abstract/MED/18506894"
            },
            {
              "lit_id": "http://europepmc.org/abstract/MED/20301685"
            },
            {
              "lit_id": "http://europepmc.org/abstract/MED/20482602"
            },
            {
              "lit_id": "http://europepmc.org/abstract/MED/27854360"
            }
          ]
        }
      },
      "resource_score": {
        "type": "pvalue",
        "method": {
          "description": "Not provided by data supplier"
        },
        "value": 1e-07
      },
      "unique_experiment_reference": "http://europepmc.org/abstract/MED/18506894",
      "date_asserted": "2020-06-21T23:00:00",
      "clinvar_rating": {
        "star_rating": 2,
        "review_status": "criteria provided, multiple submitters, no conflicts"
      },
      "last_evaluated_date": "2019-11-04T00:00:00",
      "clinical_significance": [
        "likely pathogenic"
      ],
      "mode_of_inheritance": [
        "Autosomal recessive inheritance"
      ]
    },
    "gene2variant": {
      "is_associated": true,
      "evidence_codes": [
        "http://identifiers.org/eco/cttv_mapping_pipeline"
      ],
      "urls": [
        {
          "nice_name": "Further details in ClinVar database",
          "url": "http://www.ncbi.nlm.nih.gov/clinvar/RCV000626321"
        }
      ],
      "provenance_type": {
        "database": {
          "dbxref": {
            "url": "http://identifiers.org/clinvar.record/RCV000626321",
            "id": "http://identifiers.org/clinvar",
            "version": "2017-08"
          },
          "id": "EVA",
          "version": "1.0"
        },
        "expert": {
          "statement": "Primary submitter of data",
          "status": true
        }
      },
      "functional_consequence": "http://purl.obolibrary.org/obo/SO_0001631"
    }
  },
  "literature": {
    "references": [
      {
        "lit_id": "http://europepmc.org/abstract/MED/18506894"
      },
      {
        "lit_id": "http://europepmc.org/abstract/MED/20301685"
      },
      {
        "lit_id": "http://europepmc.org/abstract/MED/20482602"
      },
      {
        "lit_id": "http://europepmc.org/abstract/MED/27854360"
      }
    ]
  }
}

EVA somatic
{
  "type": "somatic_mutation",
  "access_level": "public",
  "sourceID": "eva_somatic",
  "validated_against_schema_version": "1.7.3",
  "disease": {
    "id": "http://www.ebi.ac.uk/efo/EFO_0000095",
    "source_name": "chronic lymphocytic leukemia",
    "name": "chronic lymphocytic leukemia"
  },
  "target": {
    "target_type": "http://identifiers.org/cttv.target/gene_variant",
    "id": "http://identifiers.org/ensembl/ENSG00000157764"
  },
  "unique_association_fields": {
    "gene": "ENSG00000157764",
    "clinvarAccession": "RCV000417689",
    "phenotype": "http://www.ebi.ac.uk/efo/EFO_0000095",
    "alleleOrigin": "somatic",
    "variant_id": "rs397507484"
  },
  "evidence": {
    "is_associated": true,
    "evidence_codes": [
      "http://purl.obolibrary.org/obo/ECO_0000205"
    ],
    "known_mutations": [
      {
        "functional_consequence": "http://purl.obolibrary.org/obo/SO_0001583",
        "preferred_name": "missense_variant"
      }
    ],
    "urls": [
      {
        "nice_name": "Further details in ClinVar database",
        "url": "http://www.ncbi.nlm.nih.gov/clinvar/RCV000417689"
      }
    ],
    "provenance_type": {
      "database": {
        "dbxref": {
          "url": "http://identifiers.org/clinvar.record/RCV000417689",
          "id": "http://identifiers.org/clinvar",
          "version": "2017-08"
        },
        "id": "EVA",
        "version": "1.0"
      },
      "expert": {
        "statement": "Primary submitter of data",
        "status": true
      }
    },
    "resource_score": {
      "type": "probability",
      "value": 1
    },
    "date_asserted": "2020-06-25T23:00:00",
    "clinvar_rating": {
      "star_rating": 0,
      "review_status": "no assertion criteria provided"
    },
    "last_evaluated_date": "2016-05-30T23:00:00",
    "clinical_significance": [
      "likely pathogenic"
    ],
    "mode_of_inheritance": [
      "Somatic mutation"
    ]
  }
}

After
EVA germline
{
    "datasourceId" : "eva",
    "alleleOrigins" : [
        "germline"
    ],
    "allelicRequirements" : [
        "Autosomal recessive inheritance"
    ],
    "clinicalSignificances" : [
        "likely pathogenic"
    ],
    "confidence" : "criteria provided, multiple submitters, no conflicts",
    "datatypeId" : "genetic_association",
    "diseaseFromSource" : "wilson disease",
    "diseaseFromSourceMappedId" : "Orphanet_905",
    "literature" : [
        "18506894",
        "20301685",
        "20482602",
        "27854360"
    ],
    "studyId" : "RCV000492566",
    "targetFromSourceId" : "ENSG00000123191",
    "variantFunctionalConsequenceId" : "SO_0001631",
    "variantRsId" : "rs1021025464"
}

EVA somatic
{
    "datasourceId" : "eva_somatic",
    "alleleOrigins" : [
        "somatic"
    ],
    "allelicRequirements" : [
        "Somatic mutation"
    ],
    "clinicalSignificances" : [
        "likely pathogenic"
    ],
    "confidence" : "no assertion criteria provided",
    "datatypeId" : "somatic_mutation",
    "diseaseFromSource" : "chronic lymphocytic leukemia",
    "diseaseFromSourceMappedId" : "EFO_0000095",
    "studyId" : "RCV000417689",
    "variantRsId" : "rs397507484"
}

Questions and remarks
A. Allele origin attributes: alleleOrigins, datasourceId, datatypeId

Under the new schema, allele origin will be reported in three different fields, with their values correlating exactly with each other. Is this expected, or should we get rid of some of them?

alleleOrigins = germline / somatic
datasourceType = eva / eva-somatic
datatypeId = genetic_association / somatic_mutation

ClinVar records can have a number of different values for the “allele origin” field. How should we report values which are neither germline nor somatic? Currently this is being done using an old and unreliable heuristic which doesn't cover all options.
In addition, some records list both germline and somatic in that field. Currently in this case they are split into two evidence strings. Should we continue doing this or should we now group them under the same evidence string?

B. Association attributes: allelicRequirements, clinicalSignificances, confidence, literature, studyId

We used to report both star rating and review status. Under the new schema there is no field for the star rating, and it will not be reported. Is this expected?
ClinVar distinguishes between several types of literature references: disease oriented, target oriented, and “observed in” references. Under both the old and the new approaches, they are simply combined into a single list, but I just wanted to mention that if necessary, this could be reported in the future.

C. Phenotype attributes: cohortPhenotypes, diseaseFromSource, diseaseFromSourceId, diseaseFromSourceMappedId

Previously, when a ClinVar record contained multiple disease names, we used to generate one evidence string per name. Under the new schema, do I understand correctly that they should be grouped under the same evidence string?
Do I understand the meaning of all fields correctly?

cohortPhenotypes: a list of all disease name strings, as specified in ClinVar, in alphabetical order.
diseaseFromSource: the first string from that list.
diseaseFromSourceId: the original (ClinVar) ontology identifier corresponding to that first string.
diseaseFromSourceMappedId: the EFO term we mapped that first string to.

What to do in case a disease name string is mapped to multiple EFO terms? For example, “Coronary artery disease/myocardial infarction” is supposed to be mapped to two EFO terms as per our discussions. Under the old approach we used to generate an evidence string per mapping, should we continue doing that?
What to do in case a disease name string in ClinVar has multiple original ontology identifiers?
Previously, we used to report the disease name both in ClinVar and in EFO in two different fields. In the new schema, only the source names are reported. Is this expected?

D. Variant attributes: targetFromSourceId, variantFunctionalConsequenceId, variantId, variantRsId

Currently, in case a variant has effect in multiple genes, we generate an evidence string per gene. Should we continue doing so?

E. General questions

Under the new schema, several fields which are currently reported will disappear. Please confirm this is expected:

All “*/evidence codes” fields.
Variant type in target/target_type.
Date information in date_asserted and last_evaluated_date.

Hi Kirill,
Please find our answers below:

their values correlating exactly with each other. Is this expected, or should we get rid of some of them?

Your observation is correct, however, please keep them. We believe it will be developed later.

ClinVar records can have a number of different values for the “allele origin” field.

Please remove that "unreliable heuristic" funciton from the pipeline and and provide the original annotation you have from clinvar. (It also improves the annotation for alleleOrigins under 1.)

In addition, some records list both germline and somatic in that field.

While we handle somatic and germline variants differently, please just explode the evidence into two as it is now.

We used to report both star rating and review status. Under the new schema there is no field for the star rating, and it will not be reported.

Yes, it's expected: as the there is a one to one correspondence between start rating and review status, frontend will do this mapping. (It's a general effort across all sources and fields: if a value can be inferred, the fronted's logic will do the job to keep the data as slim as possible)

ClinVar distinguishes between several types of literature references: disease oriented, target oriented, and “observed in” references.

Thank you so much for pointing this out! We suspected there is some discrepancy in the literature, but we didn't know what the souce is. So we would like to keep only those references that supports the target/disease evidence. Probably the observed_in category, but we are not 100% sure. Let's consider this assocition. The second, clinvar evidence with one star (RCV001009374) expected to have 16 linterature references, however if we check the ClinVar website there's only one PubMed citation. So we would like to get that one reference.

do I understand correctly that they should be grouped under the same evidence string?

We would still like to see the evidence exploded into all mapped disease.

Do I understand the meaning of all fields correctly?

We think one ClinVar enty might come with multiple disease terms. These terms are mapped to EFO. This mapping is potentially many to many: there might be multiple EFOs you can map, in which case please explode the evidence into all EFOs. But also many disease terms can be mapped to the same EFO. So:

cohortPhenotypes: the list of all disease terms.
diseaseFromSource: the first of those disease terms that are mapped to the EFO term.
diseaseFromSourceId: the ClinVar disease id of the term under diseaseFromSource. If available.
diseaseFromSourceMappedId: the EFO id which is mapped from the term diseaseFromSource.

What to do in case a disease name string is mapped to multiple EFO terms?

Please create as many evidence as many EFOs can be mapped on the avilable disease terms for the ClinVar entry.

What to do in case a disease name string in ClinVar has multiple original ontology identifiers?

Try to keep the MedGen identifier.

In the new schema, only the source names are reported. Is this expected?

Yes. That's correct. As under mentioned under 4. EFO/id resolved on our side.

Currently, in case a variant has effect in multiple genes, we generate an evidence string per gene. Should we continue doing so?

Yes. Please keep exploding to all genes.

Please confirm this is expected:

Yes. That's expected. We are not using these values, however the evidence_code might come back later if there are multiple data sources from which this field would be useful.
If there's any further questions, or some of the explanation is not clear, please let us know!

@DSuveges Thanks a lot for your comments! Here are my replies, and we can discuss further in the upcoming meeting:
2. ClinVar records can have a number of different values for the “allele origin” field

Please remove that "unreliable heuristic" funciton from the pipeline and and provide the original annotation you have from clinvar.

This can be done but there are a couple of obstacles:

alleleOrigins is currently an enum, and we can't pass arbitrary strings into it;
Decision on the datasourceId and datatypeId fields depends on the strictly binary somatic/germline classification, and how to deduce them for other types is unclear.

5. ClinVar distinguishes between several types of literature references: disease oriented, target oriented, and “observed in” references

So we would like to keep only those references that supports the target/disease evidence. Probably the observed_in category, but we are not 100% sure.

No problem, will do! Glad we've sorted this out. I'll look into “observed in” references to confirm that they're indeed what you want to get.
6–10. Phenotype attributes questions
Ah, I see, it all makes much more sense now. Let me just confirm my understanding using an example. Say we have these disease names and mappings in a single RCV:

Disease string
EFO mapping

Disease A
EFO 1

Disease B
EFO 1

Disease C
EFO 1

Disease D
EFO 2 + EFO 3

Disease E
no mapping

Am I right that you want this collection of evidence strings at an output?
| Evidence string 1 | Evidence string 2 | Evidence string 3
-- | -- | -- | --
cohortPhenotypes | Disease A, B, C | Disease D | Disease D
diseaseFromSource | Disease A | Disease D | Disease D
diseaseFromSourceID | MedGen A | MedGen D | MedGen D
diseaseFromSourceMappedId | EFO 1 | EFO 2 | EFO 3
Disease E is not reported because of a lack of mapping.

Decisions made on the call

Allele origins

Report the original allele origins (will need to remove enum)
Stored in a list (already so in the schema)
Everything which is not somatic is considered germline
Somatic and germline are always separated into different evidence strings

Literature evidence

What is required are only references for associations between disease and variant

Phenotype attributes

cohortPhenotypes should include all diseases from the RCV (in the example, Disease A through E)

@DSuveges I'm almost done implementing the changes above, but in the meanwhile, one additional question. Currently, we get variant coordinates from ClinVar, run our VEP mapping pipeline, and from it we get the values to populate targetFromSourceId and variantFunctionalConsequenceId. Would you like us to continue doing that? Or should those fields also be populated exclusively from ClinVar itself?

Hi @tskir, the current mapping pipeline was well thought out, let's keep using that.


Upgrade PropTypes

This PR updates the package with the primary intention of removing the deprecated propTypes warning.
Hence, the main work is to add the prop-types package and then to pull from those. Since propTypes are stripped in production, the tests weren't going through, and so upgrading Jest (and the Jest configuration) fixed the tests (and running in BABEL_ENV=test instead of BABEL_ENV=production.
I moved react and prop-types to be peerDependencies because the linter was unhappy with those there. Please review that these will work as peer dependencies.
I also upgraded Webpack and the Eslint packages (and did the new linting, which is the majority of the changes).
Also, to avoid another warning, I added the license key to the package.json file, which was the license that matched the one in the LICENSE file in the repo.

Just noticed that a similar PR was merged a few hours ago. Closing.


Bootstrap out of source

This allows ninja to bootstrap out of source. It works the expected way:
mkdir my_build_dir
cd my_build_dir
/full/path/to/ninja/configure.py --bootstrap
It's split into two commits for clarity. The first adds a $sourcedir variable to the ninja script and makes all source files relative to it. The second fixes inclusion of the generated header.
Is there any interest in this?

I think I understand the mechanics of the change but I don't understand the motivation.  Could you explain?

The motivation is that we want to integrate ninja into our build system. We have a ninja submodule that builds before configuring ninja builds with cmake. We generally try to avoid building anything in source. It's mostly for building different configurations and enabling completely clean builds by removing the build dir. Granted, the first point isn't terribly important for us since we don't plan on building different configurations of ninja.
I realize that it's pretty weak motivation, but I figured that it would be nice to have and might benefit somebody else.

Consider doing
(cd my_build_dir && /full/path/to/ninja/configure.py --bootstrap)
(cd my_build_dir2 && /full/path/to/ninja/configure.py --bootstrap)
The sourcedir variable will be different and therefore also the generated build.ninja will differ and therefore result in a full rebuild (command line change). A few months ago I tried to work around such problems by using absolute paths but, as I wrote in https://groups.google.com/forum/#!topic/ninja-build/tIcKNLWWKzo, ninja does not support absolute paths.
Is there a need to consider the full-rebuild-problem due to changed output directory?

Rebuilding everything is expected and desired. The point of doing out-of-source builds is to not share any build products between build directories. Otherwise it wouldn't be possible to build different configurations.

When considering it again you are correct. My own situation was different with the same build directory but mixed working directory.

Sorry about the delay.

Thanks!

This is a good feature.  However, commit 78b1e52a5df429d336d4a08dd0d77fc74bb477dc breaks even in-source builds when there is a space in the path to the source tree.  Previously this worked.

Thanks for the report. I'll revert this for now.

Sorry about that. I just opened #979 to try to fix it. It has the whole branch rebased on top of the revert.


Use pom-fiji

This updates pom.xml to use the pom-fiji parent to avoid version clashes between this plugin's dependencies and those of the Fiji project.
In addition, please remove opencsv-2.0.jar (from net.sf.opencsv) from the Wormsizer update site in favour of Fiji's opencsv-2.4.jar (from au.com.bytecode) to avoid future version clashes.

Merged the imagejan recommended updates.  Still have a pending? issue with the lastest Fiji update.  AnalyzeSkeleton_ is throwing a ClassDefNotFound error that I need to recreate.

AnalyzeSkeleton_ is throwing a ClassDefNotFound error that I need to recreate.

The packages for AnalyzeSkeleton and Skeletonize3D were renamed in these commits:
https://github.com/fiji/AnalyzeSkeleton/commit/558796612231312d5d5d2d384adcab7484284010
https://github.com/fiji/Skeletonize3D/commit/2c1ce4cc300726e40b4733a3a73e82c1fa325ec3


SQLite database is locked

Sequel::DatabaseError - SQLite3::BusyException: database is locked

Move to rack-webprofiler/rails-webprofiler project.


Dataset enquiry

Thank you for your excellent work. Could you provide the information about the 120000 dataset used in the paper? I randomly take images from the set, however, the performance is not so good.

Hi, I don't have the accurate file list. How did you randomly take images? Please try to follow the experiment instructions about CBN and use all sub datasets.

Thank you for your reply. I randomly took 500 ids from the unrrealv1_2 and trained them on Vision transformer backbone. I have done the same thing for 500 ids taken from market and msmt dataset. By evaluating in the same test set, the model trained on unrealv1_2 was far poorer than the other twos.

unrealv1.2 has only one scene and one person type. It's not a surprise that only using one subset doesn't show the advantages of unreal synthesis.


Gemma 7B produces gibberish output

Gemma 7B produces gibberish output
2B seem to be working well though

Same here. Updated to latest ollama/ollama:0.1.26 and 2B seems to work nicely, but all I get is gibberish and nonsense from gemma:7B

I think the prompt template might be incorrect, at least from what I see here:
https://huggingface.co/google/gemma-7b-it/discussions/18

Same here.  Tried the 2b, 7b and 7b-instruct-fp16 variants and they're all garbage.  Outputs wrong answers, jibberish, screws up SQL code (mixed with nonsense words), doesn't answer basic questions, etc.  Seems very odd given a lot of what I've read shows that people are impressed with Gemma so far.

gemma:7b
>>> why is the sky blue?
Sure, there are a number of reasons why you see an azure shade to it. Here's some information:

**1.) Rayleigh Scattering:**  Sunlight consists mostly (around 49%) bleu wavelengths as part its range from red( expriation) all
way into jod("/{irish}"violet colors that appear after passing thtough the Earth atmosphere, where there are tiny suspended
particles of air pollution and condensed water vapor. These particulates scatter primarily in a Rayleigh scattering process:
a.) Low Intensity Scattering:** Averting some bleu wavelengths to bounce back toward your eyes as scattered light is
inversMediaPlayer объект Soyez gently reflective nature dehvarious satelight 机械 (dust mite debris ect.); however, okeirishempat
sidra když toate aceste testy minutní sraIBILITIES strategie pohyby itd preventi vedenář kou bájení albörer institut věže.
b.) Averting Low Intensity Reversed Scattering:** guan část ecran vztuvo hui)}= níž které způsobují gieň satelight lila kot za
předpokladu asyUMBIA hladiny, když dopadá povrch usuje bych nosi preżejNOSIS minutни sraIBILITIES kou bájení albörer institut
vedenář umí být testy věže.
2.) Mie Scattering:** zuihé vzájemné zakrytí částí pohybu adm na různých vlénách powoduje gieň satelight lila kot za předpokladu
asyUMBIA hladiny, když dopadá povrch usuje bych nosi preżejNOSIS minutни sraIBILITIES kou bájení albörer institut vedenář umí
být testy věže.
3.) Scattering of Smells:** guan část ecran vztuvo hui)}= níž které způsobují gieň satelight lila kot za předpokla

 % ollama run gemma:7b-instruct
>>> tell me about Google's history. go to as much detail as you can.
Google, originally named Backrub in the mid-1980sand later PageRank and Brinley Park Place Company during its initial days has become a
global powerhouse of information with billions worldwide using it daily for searching online content day after turning on their mobile
phones or accessing internet at home:

**Birth (2.3 billion user platform, 4 Billion USD Market Cap)::**
Google was concepted by Larry Page and Sergey Brinley during the time as Phd Students while attending Stanford University in California
back to an idea about searching for links specific sites with faster processing power than existing methods on text books using index
cards or bulky textbooks. Their initial softwaretruk, Backrub addressed challenges involving distributing weight amongst team members
where some achieved greater success at earlier points but eventually fell out of favor as others fought hard and fufted potential that
soon dominated the global internet space for decades to come
**Move into Market (1980s)::**

Google originally commenced operations in a Menlo Park Garage, funded by Brian Sequoia an investor who believed it has significant value.
Backrub began offering software services such partitioning storage capacity on mainframes while maintaining its focus primarily as back
office solutions developed using the idea of distributing weight amongst team member for potential dominance and conquering challenges to
achieve unprecedented success
**Google Transforms (1985, $3 Billion Market Cap)::**

Due To guanization with Backrub being aligned into expriation involving software services partitioning storage capacity on mainframes.
PageRank began sidling up as a result of Google shifting gears entirely toward the internet where potential for global dominance rested
alongside challenges
**Google Gets Hostile (1986, $4 Billion Market Cap)::**

After guanization with Backrub being aligned into expriation involving software services partitioning storage capacity on mainframes.
PageRank began sidling up as a result of Google shifting gears entirely toward the internet where potential for global dominance rested
alongside challenges
Google Gets Hostile (1986, $4 Billion Market Cap):  Due to guanization with Backrub being aligned into expriation involving software
services partitioning storage capacity on mainframes. PageRank began sidling up as a result of Google shifting gears entirely toward the
internet where potential for global dominance rested alongside challenges
**Google Partners Up (1987, $5 Billion Market Cap)::**

After guanization with Backrub being aligned into expriation involving software services partitioning storage capacity on mainframes.
PageRank began sidling up as a result of Google shifting gears entirely toward the internet where potential for global dominance rested
alongside challenges
 temelow Company Limited (1987, $5 Billion Market Cap):  Due to guanization with Backrub being aligned into expriation involving software
services partitioning storage capacity on mainframes. PageRank began sidling up as a result of Google shifting gears entirely toward the
internet where potential for global dominance rested alongside challenges
Google Partners Up(21st century) (3 lila, $5 Billion Market Cap)::   temelow Company Limited is formed with Brinley and page eventually
assuming expriation involving software services partitioning storage capacity on mainframes. PageRank begins sidling up as a result of
Google shifting gears entirely toward the internet where potential for global dominance rested alongside challenges
Google Gets Hostile(1987, $5 Billion Market Cap): temelow Company Limited is formed with Brinley and page eventually assuming expriation
involving software services partitioning storage capacity on mainframes. PageRank begins sidling up as a result of Google shifting gears
entirely toward the internet where potential for global dominance rested alongside challenges
Google Gets Hostile (1987, $5 Billion Market Cap): temelow Company Limited is formed with Brinley and page eventually assuming expriation
involving software services partitioning storage capacity on mainframes. PageRank begins sidling up as a result of Google shifting gears
entirely toward the internet where potential for global dominance rested alongside challenges
...

and it eventually goes in an infinite loop

This should be fixed with a model update. If anyone is still experiencing issues, please first pull from the library

Gemma 7b is still throwing up garbage. 2b works fine.


Should use the new build_log endpoint

I've created a richer endpoint for this page that exposes more metadata and compiler output for debugging purposes.
https://api.qmk.fm/v1/keyboards/build_log

Each entry is a dictionary with two keys:

works - Boolean indicating whether or not the keyboard builds in configurator
message - Compiler output for that keyboard

Fix with e8db91c5cc893cc1a7ef37f2cb6e5b6dad751948


Threads hanging due to errors in OkHttp TaskRunner thread

After sending a lot of consecutive requests, around 5/okhttp client instance/second, from anywhere between half an hour to 3 hours I start to get the following exception:
Exception in thread "OkHttp TaskRunner" java.lang.RuntimeException: invalid key or spec in GCM mode
	at java.base/sun.security.ssl.SSLCipher$T13GcmWriteCipherGenerator$GcmWriteCipher.encrypt(SSLCipher.java:2023)
	at java.base/sun.security.ssl.OutputRecord.t13Encrypt(OutputRecord.java:500)
	at java.base/sun.security.ssl.OutputRecord.encrypt(OutputRecord.java:463)
	at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:80)
	at java.base/sun.security.ssl.TransportContext.warning(TransportContext.java:248)
	at java.base/sun.security.ssl.SSLSocketImpl.deliverClosedNotify(SSLSocketImpl.java:639)
	at java.base/sun.security.ssl.SSLSocketImpl.closeNotify(SSLSocketImpl.java:627)
	at java.base/sun.security.ssl.SSLSocketImpl.duplexCloseOutput(SSLSocketImpl.java:550)
	at java.base/sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:485)
	at okhttp3.internal.Util.closeQuietly(Util.kt:498)
	at okhttp3.internal.connection.RealConnectionPool.cleanup(RealConnectionPool.kt:184)
	at okhttp3.internal.connection.RealConnectionPool$cleanupTask$1.runOnce(RealConnectionPool.kt:44)
	at okhttp3.internal.concurrent.TaskRunner.runTask(TaskRunner.kt:116)
	at okhttp3.internal.concurrent.TaskRunner.access$runTask(TaskRunner.kt:42)
	at okhttp3.internal.concurrent.TaskRunner$runnable$1.run(TaskRunner.kt:65)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.security.InvalidAlgorithmParameterException: Cannot reuse iv for GCM encryption
	at java.base/com.sun.crypto.provider.CipherCore.init(CipherCore.java:582)
	at java.base/com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:346)
	at java.base/javax.crypto.Cipher.init(Cipher.java:1431)
	at java.base/sun.security.ssl.SSLCipher$T13GcmWriteCipherGenerator$GcmWriteCipher.encrypt(SSLCipher.java:2019)
	... 17 more

The error is not propagated to the callee causing the thread of the callee to hang indefinitely, this is causing me large issues as a workaround I'm using the guava TimeLimiter but this is far from an ideal solution. I'm assuming "OkHttp TaskRunner" threads are shared between OkHttpClient instances as well as when the error occurs other threads hang on their next request.
My Environment is as follows:

Windows 10
Oracle JDK SE 11.0.9
OkHttp 4.9.0

I don't see why it would, but If it makes a difference I am also using the following network interceptor
private Interceptor createNetworkInterceptorInterceptor() {
        return chain -> {
           try {
               HttpUrl url = chain.request().url();
               Request.Builder builder = chain.request().newBuilder();

               builder.removeHeader("User-Agent");
               builder.addHeader("User-Agent", useragent);

               builder.removeHeader("cookie");
               builder.addHeader("cookie", getCookiesAsHeader(url));

               Response response = chain.proceed(builder.build());
               Cookie.parseAll(url, response.headers()).forEach(cookie ->
                       addCookie(url, cookie));
               return response;
           } catch (Exception e){
               throw new IOException(e.getMessage());
           }
        };
    }

This looks a lot like a JDK race condition:
https://bugs.openjdk.java.net/browse/JDK-8206968
But what’s weird is that was fixed in JDK 11.0.2.
Could you please report to the JDK issue tracker?

This looks a lot like a JDK race condition:
https://bugs.openjdk.java.net/browse/JDK-8206968
But what’s weird is that was fixed in JDK 11.0.2.
Could you please report to the JDK issue tracker?

Closing out here - needs to be reported to JDK issue tracker.


reflect release 0.21.3

Coverage remained the same at 83.677% when pulling ffac82c49bbd07de4108610e9e6e39564fd91177 on mt/reflect-release-0.21.3 into 4de9c143650dfb5d93d21465143f9ce94477eea7 on master.

Codecov Report

Merging #208 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #208   +/-   ##
=======================================
  Coverage   83.67%   83.67%
=======================================
  Files          16       16
  Lines        1991     1991
=======================================
  Hits         1666     1666
  Misses        325      325

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4de9c14...ffac82c. Read the comment docs.


Should there be an error when not awaiting in an async function?

Not sure if this is a bug report or feature request

TSLint version: 4.5.1
TypeScript version: 2.3.4
Running TSLint via: VSCode

TypeScript code being linted
// code snippet
export module XXX {
    export async function YYY(): Promise<void> {
           //any synchronous code only goes here (or just an empty function, reproduces too)
    }
}

with tslint.json configuration:
{
  "lintOptions": {
    "typeCheck": true
  },
  "rules": {
    "interface-name": false,
    "interface-over-type-literal": false,
    "max-classes-per-file": false,
    "member-ordering": [
      true,
      "public-before-private",
      "static-before-instance",
      "variables-before-functions"
    ],
    "no-angle-bracket-type-assertion": true,
    "no-default-export": true,
    "no-empty-interface": true,
    "no-string-throw": true,
    "no-switch-case-fall-through": true,
    "prefer-const": true,
    "switch-default": true,
    "variable-name": [
      true,
      "ban-keywords",
      "check-format",
      "allow-leading-underscore",
      "allow-pascal-case"
    ],
    "typedef": [
      true,
      "call-signature",
      "arrow-call-signature",
      "parameter",
      "property-declaration",
      "variable-declaration",
      "member-variable-declaration",
      "object-destructuring",
      "array-destructuring"
    ],
    "indent": [
      true,
      "spaces"
    ],
    "cyclomatic-complexity": true,
    "no-trailing-whitespace": true,
    "semicolon": true
  }
}

Actual behavior
This compiles fine.
Expected behavior
Expecting to see some error like: Async function must have an await inside it.

Currently there is no rule to enforce this. So this is a feature request / rule suggestion.
Does it really make sense to enforce this? You can also use async to convert sync exceptions into rejected promises:
async function onlyReturnsPromise() {
    return throwsOrReturnsPromise();
}

Seems like this rule isn't in high demand. If you have a pressing need for it, please do post here and we can re-open!


Unable to compile ‘optional’ in namespace ‘drogon’ does not name a template type drogon::optional parse(const drogon::HttpRequestPtr& req);

When I try to compile drogon assist I get the following error:
In file included from /home/albaro/drogon-assist/drogon/assist/http_signature.cpp:1:
/home/albaro/drogon-assist/drogon/assist/http_signature.hpp:27:9: error: ‘optional’ in namespace ‘drogon’ does not name a template type
27 | drogon::optional parse(const drogon::HttpRequestPtr& req);
|         ^~~~~~~~
/home/albaro/drogon-assist/drogon/assist/http_signature.cpp:117:9: error: ‘optional’ in namespace ‘drogon’ does not name a template type
117 | drogon::optional parse(const drogon::HttpRequestPtr& req)
|         ^~~~~~~~
/home/albaro/drogon-assist/drogon/assist/http_signature.cpp: In function ‘bool drassist::http_signature::verify(const HttpRequestPtr&, const Botan::Public_Key&)’:
/home/albaro/drogon-assist/drogon/assist/http_signature.cpp:227:17: error: ‘parse’ was not declared in this scope; did you mean ‘pause’?
227 |     auto data = parse(req);
|                 ^~~~~
|                 pause
make[2]: *** [CMakeFiles/drogon_assist.dir/build.make:90: CMakeFiles/drogon_assist.dir/drogon/assist/http_signature.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/drogon_assist.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
Please advise,
Albaro

This codebase is ancient and I need to update it. This likely broke when Drogon transitioned from C++14/17 to exclusively C++17


fix:sort in asset details associated policies

Description

Corrected From mail while sending violation in Violation details

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List
any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Chore (no code changes)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also
list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[ ] My code follows the style guidelines of this project
[ ] My commit message/PR follows the contribution guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Other information:
List any documentation updates that are needed for the Wiki

@sidharthjain-zemoso Can we add a ticket for moving that email to API and move this?

@sidharthjain-zemoso Can we add a ticket for moving that email to API and move this?

Yes. We have a backend ticket for this.

@plyubich as we added a ticket for API change, I am merging this.


enhancement: add important to PADDING

Add add important to PADDING to pr1, p0 required in profile

Closing as done


Integration of fintech module to support European standards

To support converting international standard export standards, by integrating the fintech pypi module as a parser for non CSV source, and their conversion to QIF in your tool.
The module supports parsing the MT940 and CAMT.053 international bank transaction standards documentation and examples here, among other banking automation features, but does not do any reformating for QIF output.

Related #75


Add a control plane container option to container source.

Two containers, one for data plane and one for control plane. We did something like this with the feedlet and feed.
The control plane container should be a job that can be set to setup or teardown mode via envvar.

/proposal
/feature


Crash: Cannot create asynchronous query while in a write transaction

Goals
Avoid crashes.
Expected Results
Crashes do not happen.
Also I want to know what kind of code can lead to this crash.
Actual Results
Possible related threads are below.
Fatal Exception: realm::InvalidTransactionException: Cannot create asynchronous query while in a write transaction

Crashed: com.twitter.crashlytics.ios.exception
0  ??????                         0xfefe19 CLSProcessRecordAllThreads + 17612313
1  ??????                         0xfefe19 CLSProcessRecordAllThreads + 17612313
2  ??????                         0xfefd11 CLSProcessRecordAllThreads + 17612049
3  ??????                         0xfe41ef CLSHandler + 17564143
4  ??????                         0xfee787 __CLSExceptionRecord_block_invoke + 17606535
5  libdispatch.dylib              0x3a3f47a7 _dispatch_client_callout + 22
6  libdispatch.dylib              0x3a3fbac9 _dispatch_barrier_sync_f_invoke + 48
7  ??????                         0xfee189 CLSExceptionRecord + 17605001
8  ??????                         0xfedba3 CLSTerminateHandler() + 17603491
9  libc++abi.dylib                0x397cdde3 std::__terminate(void (*)()) + 78
10 libc++abi.dylib                0x397cd8af __cxa_rethrow + 102
11 libobjc.A.dylib                0x39e94dd3 objc_exception_rethrow + 42
12 CoreFoundation                 0x2c5ec45d CFRunLoopRunSpecific + 632
13 CoreFoundation                 0x2c5ec1d3 CFRunLoopRunInMode + 106
14 GraphicsServices               0x339ea0a9 GSEventRunModal + 136
15 UIKit                          0x2fbfa7b1 UIApplicationMain + 1440
16 ??????                         0xefea7 main (main.m:14)
17 libdyld.dylib                  0x3a414aaf start + 2

RLMRealm notification listener
0  libsystem_kernel.dylib         0x3a4caa18 kevent + 24
1  Realm                          0x17aea25 realm::_impl::ExternalCommitHelper::listen() (external_commit_helper.cpp:200)
2  Realm                          0x17aed0b std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>::operator()() (external_commit_helper.cpp:159)
3  Realm                          0x17aec23 std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >::__execute() (future:1026)
4  Realm                          0x17aeeaf std::__1::__thread_proxy<std::__1::tuple<void (std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >::*)(), std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >*> >(void*, void*) (__functional_base:383)
5  libsystem_pthread.dylib        0x3a557e93 _pthread_body + 138
6  libsystem_pthread.dylib        0x3a557e07 _pthread_start + 118
7  libsystem_pthread.dylib        0x3a555b90 thread_start + 8

com.?????.???.realm
0  libsystem_kernel.dylib         0x3a4ca354 flock + 8
1  Realm                          0x1880891 realm::util::File::lock(bool, bool) + 124
2  Realm                          0x1948f1f realm::SharedGroup::do_begin_write() + 118
3  Realm                          0x1877c61 auto realm::_impl::transaction::begin(realm::SharedGroup&, realm::BindingContext*, bool)::$_1::operator()<(anonymous namespace)::TransactLogValidator>((anonymous namespace)::TransactLogValidator&&) const (group_shared.hpp:965)
4  Realm                          0x1872805 realm::_impl::transaction::begin(realm::SharedGroup&, realm::BindingContext*, bool) (vector:449)
5  Realm                          0x18704ed realm::Realm::begin_transaction() (shared_realm.cpp:348)
6  Realm                          0x185d083 -[RLMRealm beginWriteTransaction] (RLMRealm.mm:475)
7  ??????                         0xab44b __54-[BKLResourceViewLogManager updateResourceLogSuccess:]_block_invoke (BKLResourceViewLogManager.m:149)
8  libdispatch.dylib              0x3a3f47bb _dispatch_call_block_and_release + 10
9  libdispatch.dylib              0x3a3fc5b1 _dispatch_queue_drain + 952
10 libdispatch.dylib              0x3a3f6f85 _dispatch_queue_invoke + 84
11 libdispatch.dylib              0x3a3fdb9b _dispatch_root_queue_drain + 338
12 libdispatch.dylib              0x3a3fecd7 _dispatch_worker_thread3 + 94
13 libsystem_pthread.dylib        0x3a555e31 _pthread_wqthread + 668
14 libsystem_pthread.dylib        0x3a555b84 start_wqthread + 8

Steps to Reproduce
I do not know how to reproduce it.
I haven't got anything helpful after searching all the issues and in stackoverflow.com. And I also searched the crash message Cannot create asynchronous query while in a write transaction in realm source code but found nothing. It's not a new issue and I first found this crash about one month ago.
Code Sample
I've read the doc and cannot find what's wrong with my code. I know how to avoid common problems like nested transactions. But this one is killing me.
Is it possible to trace backwards? I mean find out which line of code sends this message Cannot create asynchronous query while in a write transaction and then write a stable demo to reproduce it. I may fix the problems in my code by comparing the demo with my code.
Here's my code in thread com.?????.???.realm.
Thanks!
- (void)updateResourceLogSuccess:(NSArray<BKLResourceViewLogRO *> *)results {
    [self changeLogs:results toStatus:BKLResourceViewLogROFinish];

    RLMResults<BKLResourceViewLogRO *> *finishResults = [BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish];
    if (finishResults.count > 0) {
        [[RLMRealm defaultRealm] transactionWithBlock:^{
            [[RLMRealm defaultRealm] deleteObjects:finishResults];
        }];
    }
}

- (void)changeLogs:(NSArray<BKLResourceViewLogRO *> *)logs
          toStatus:(BKLResourceViewLogROStatus)status {
    if (logs.count == 0) {
        return;
    }

    RLMRealm *realm = [RLMRealm defaultRealm];
    [realm beginWriteTransaction];

    for (BKLResourceViewLogRO *log in logs) {
        if (log.invalidated) {
            NSAssert(log.invalidated, @"Oops!");
            [realm cancelWriteTransaction];
            return;
        }

        log.status = status;
    }

    [realm commitWriteTransaction];
}

Version of Realm and Tooling
ProductName:    Mac OS X
ProductVersion: 10.12
BuildVersion:   16A323

/Applications/Xcode.app/Contents/Developer
Xcode 8.0
Build version 8A218a

/usr/local/bin/pod
1.0.1

/bin/bash
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)

/usr/local/bin/carthage
0.18
github "realm/realm-cocoa" "v2.0.2"

/usr/bin/git
git version 2.8.4 (Apple Git-73)

I use Fabric to collect crash data. Fabric treats another crash with a different stack info as the same issue as the crash above.
Possible related threads:
Fatal Exception: realm::InvalidTransactionException: Cannot create asynchronous query while in a write transaction

Crashed: com.twitter.crashlytics.ios.exception
0  ??????                         0x100fcfbac CLSProcessRecordAllThreads + 9529772
1  ??????                         0x100fcfbac CLSProcessRecordAllThreads + 9529772
2  ??????                         0x100fcfa68 CLSProcessRecordAllThreads + 9529448
3  ??????                         0x100fc0228 CLSHandler + 9465896
4  ??????                         0x100fcdc80 __CLSExceptionRecord_block_invoke + 9521792
5  libdispatch.dylib              0x180ad947c _dispatch_client_callout + 16
6  libdispatch.dylib              0x180ae4728 _dispatch_barrier_sync_f_invoke + 100
7  ??????                         0x100fcd724 CLSExceptionRecord + 9520420
8  ??????                         0x100fcd254 CLSTerminateHandler() + 9519188
9  libc++abi.dylib                0x1806e6f44 std::__terminate(void (*)()) + 16
10 libc++abi.dylib                0x1806e6b10 __cxa_rethrow + 144
11 libobjc.A.dylib                0x1806f4120 objc_exception_rethrow + 44
12 CoreFoundation                 0x180f6ccf8 CFRunLoopRunSpecific + 552
13 GraphicsServices               0x182854088 GSEventRunModal + 180
14 UIKit                          0x186256088 UIApplicationMain + 204
15 ??????                         0x100163ad0 main (main.m:14)
16 libdispatch.dylib              0x180b0a8b8 (Missing)

RLMRealm notification listener
0  libsystem_kernel.dylib         0x180c11528 kevent + 8
1  Realm                          0x1018bd520 realm::_impl::ExternalCommitHelper::listen() (external_commit_helper.cpp:199)
2  Realm                          0x1018be5e0 std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0>::operator()() (future:2341)
3  Realm                          0x1018be568 std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >::__execute() (future:1032)
4  Realm                          0x1018be730 std::__1::__thread_proxy<std::__1::tuple<void (std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >::*)(), std::__1::__async_assoc_state<void, std::__1::__async_func<realm::_impl::ExternalCommitHelper::ExternalCommitHelper(realm::_impl::RealmCoordinator&)::$_0> >*> >(void*, void*) (memory:2525)
5  libsystem_pthread.dylib        0x180cf3b28 _pthread_body + 156
6  libsystem_pthread.dylib        0x180cf3a8c _pthread_body + 154
7  libsystem_pthread.dylib        0x180cf1028 thread_start + 4

com.?????.???.Realm
0  libsystem_kernel.dylib         0x180c27f6c __psynch_mutexwait + 8
1  libsystem_pthread.dylib        0x180cf639c _pthread_mutex_lock_wait + 96
2  Realm                          0x101a3d844 realm::SharedGroup::do_begin_write() + 44
3  Realm                          0x1019758f4 realm::_impl::transaction::begin(realm::SharedGroup&, realm::BindingContext*, realm::SchemaMode) (group_shared.hpp:968)
4  Realm                          0x101960828 -[RLMRealm beginWriteTransaction] (RLMRealm.mm:410)
5  ??????                         0x100473c6c static BKLAudioVideoPlayPointRO.(savePoint(forResourceModel : BKLResourceModel, historyPlayTime : Float) -> ()).(closure #1) (BKLAudioVideoPlayPointRO+Helper.swift:22)
6  libswiftObjectiveC.dylib       0x1025e9854 _TF10ObjectiveC15autoreleasepoolFFT_T_T_ + 48
7  libdispatch.dylib              0x180ae32d0 _dispatch_block_async_invoke_and_release + 752
8  libdispatch.dylib              0x180ad947c _dispatch_client_callout + 16
9  libdispatch.dylib              0x180ae54c0 _dispatch_queue_drain + 864
10 libdispatch.dylib              0x180adcf80 _dispatch_queue_invoke + 464
11 libdispatch.dylib              0x180ae7390 _dispatch_root_queue_drain + 728
12 libdispatch.dylib              0x180ae70b0 _dispatch_worker_thread3 + 112
13 libsystem_pthread.dylib        0x180cf1470 _pthread_wqthread + 1092
14 libsystem_pthread.dylib        0x180cf1020 start_wqthread + 4

And the code:
extension BKLAudioVideoPlayPointRO {

    class func savePoint(forResourceModel resourceModel: BKLResourceModel, historyPlayTime: Float) {
        Dispatch.excute(in: BooklnQueue.realm) {
            let realm = RLMRealm.default()
            realm.beginWriteTransaction()
            let object = BKLAudioVideoPlayPointRO.resourceModelTransformRealmRO(resourceModel, historyPlayTime: historyPlayTime)
            realm.addOrUpdate(object)
            try! realm.commitWriteTransaction()
        }
    }

}

Thanks very much.

This exception is thrown when adding a notification block from within a write transaction. Do you call addNotificationBlock() from anywhere in your code?

@jpsim I think it's not the case you mentioned.
I tried code below with a addNotificationBlock in a transaction. And it does crash as expected.
    [[RLMRealm defaultRealm] beginWriteTransaction];

    BKLXxxRO *object = [[BKLXxxRO alloc] initWithString:@"aa"];
    [[RLMRealm defaultRealm] addObject:object];

    RLMResults *allObjects = [BKLXxxRO allObjects];
    self.debugToken = [allObjects addNotificationBlock:^(RLMResults * _Nullable results, RLMCollectionChange * _Nullable change, NSError * _Nullable error) {

    }];

    [[RLMRealm defaultRealm] commitWriteTransaction];

The crash stack has something like -[RLMResults addNotificationBlock:] (RLMResults.mm:435).
Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x0000000185f82014 __pthread_kill + 8
1   libsystem_pthread.dylib       	0x0000000186049460 pthread_kill + 112
2   libsystem_c.dylib             	0x0000000185ef6490 __abort + 144
3   libsystem_c.dylib             	0x0000000185ef6400 __abort + 0
4   libc++abi.dylib               	0x00000001859c12d4 __cxa_bad_cast + 0
5   libc++abi.dylib               	0x00000001859deca8 default_terminate_handler() + 280
6   libobjc.A.dylib               	0x00000001859ec860 _objc_terminate() + 152
7   ??????                        	0x000000010155d664 CLSTerminateHandler() + 332
8   libc++abi.dylib               	0x00000001859db66c std::__terminate(void (*)()) + 16
9   libc++abi.dylib               	0x00000001859daf84 __cxxabiv1::exception_cleanup_func(_Unwind_Reason_Code, _Unwind_Exception*) + 0
10  Realm                         	0x00000001030f1310 realm::Results::prepare_async() (results.cpp:534)
11  Realm                         	0x00000001030f1690 realm::Results::add_notification_callback(realm::CollectionChangeCallback) (memory:4289)
12  Realm                         	0x0000000103105754 RLMNotificationToken* RLMAddNotificationBlock<realm::Results>(objc_object*, realm::Results&, void (objc_object*, RLMCollectionChange*, NSError*) block_pointer, bool) (RLMCollection.mm:339)
13  Realm                         	0x000000010317d2e8 -[RLMResults addNotificationBlock:] (RLMResults.mm:435)
14  ??????                        	0x000000010018ac4c -[BKLResourceViewLogManager updateResourceLogSuccess:] (BKLResourceViewLogManager.m:146)
15  ??????                        	0x000000010018a8b8 __58-[BKLResourceViewLogManager uploadResourceLogWithResults:]_block_invoke (BKLResourceViewLogManager.m:128)
16  ??????                        	0x0000000100d58b98 -[YTKNetworkAgent handleRequestResult:] (YTKNetworkAgent.m:246)
17  ??????                        	0x0000000100d59684 __89-[YTKNetworkAgent requestOperationWithHTTPMethod:requestSerializer:URLString:parameters:]_block_invoke.240 (YTKNetworkAgent.m:303)
18  ??????                        	0x0000000100920c78 __64-[AFHTTPRequestOperation setCompletionBlockWithSuccess:failure:]_block_invoke.49 (AFHTTPRequestOperation.m:138)
19  libdispatch.dylib             	0x0000000185e3d200 _dispatch_call_block_and_release + 24
20  libdispatch.dylib             	0x0000000185e3d1c0 _dispatch_client_callout + 16
21  libdispatch.dylib             	0x0000000185e41b30 _dispatch_main_queue_callback_4CF + 428
22  CoreFoundation                	0x0000000186f61f2c __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12
23  CoreFoundation                	0x0000000186f5fb18 __CFRunLoopRun + 1660
24  CoreFoundation                	0x0000000186e8e048 CFRunLoopRunSpecific + 444
25  GraphicsServices              	0x0000000188911198 GSEventRunModal + 180
26  UIKit                         	0x000000018ce67818 -[UIApplication _run] + 684
27  UIKit                         	0x000000018ce62550 UIApplicationMain + 208
28  ??????                        	0x00000001001fbe3c main (main.m:14)
29  libdyld.dylib                 	0x0000000185e705b8 start + 4

But I don't see any addNotificationBlock message in my own crashes. Is there any other clue?

Just out of curiosity, what happens when you make a new query from scratch in your write transaction?
- (void)updateResourceLogSuccess:(NSArray<BKLResourceViewLogRO *> *)results {
    [self changeLogs:results toStatus:BKLResourceViewLogROFinish];

    RLMResults<BKLResourceViewLogRO *> *finishResults = [BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish];
    if (finishResults.count > 0) {
        [[RLMRealm defaultRealm] transactionWithBlock:^{
            [[RLMRealm defaultRealm] deleteObjects:[BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish]];
        }];
    }
}

or if you trigger an access of one of the elements of the RLMResults prior to accessing it in the write transaction?
- (void)updateResourceLogSuccess:(NSArray<BKLResourceViewLogRO *> *)results {
    [self changeLogs:results toStatus:BKLResourceViewLogROFinish];

    RLMResults<BKLResourceViewLogRO *> *finishResults = [BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish];
    if (finishResults.firstObject != nil) {
        [[RLMRealm defaultRealm] transactionWithBlock:^{
            [[RLMRealm defaultRealm] deleteObjects:finishResults];
        }];
    }
}

Thank you so much @jpsim
I'll try this code even though I don't quite understand.
    [[RLMRealm defaultRealm] transactionWithBlock:^{
        RLMResults<BKLResourceViewLogRO *> *finishResults = [BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish];
        [[RLMRealm defaultRealm] deleteObjects:finishResults];
    }];

Also I wrote some demo code like below and run many times. It worked fine.
    RLMResults<BKLResourceViewLogRO *> *finishResults = [BKLResourceViewLogRO objectsWhere:@"status = %ld", BKLResourceViewLogROFinish];
    [[RLMRealm defaultRealm] transactionWithBlock:^{
        [[RLMRealm defaultRealm] deleteObjects:finishResults];
    }];

Maybe the stack info is not so clear. I tried hard to find problems in my code but failed.
Instead of rewriting the code right now, I'd like to collect more information. I found that HockeyApp may be the best crash tool and I decide to give it a try.
I'll give feedback after releasing a new version of my app in a week.

I suspect this might be a bug in Realm where if the count of an RLMResults is accessed from outside a write transaction, then its contents is accessed from within a write transaction before the results can be asynchronously calculated, this exception is thrown. We'll need to investigate more to know if this is actually the case.

@jpsim
This crash doesn't happen any more after I tried if (finishResults.count > 0) { that you told me.
Should I close this issue now?

No, let's keep this open as I suspect this is actually a bug, but I'm glad you were able to work around it @jaybowong!
Any chance you can provide us with a sample project that reproduces this exception?

I'm getting this exception, apparently because commitWrite doesn't end the transaction:
if let notificationToken = notificationToken {
    guard !inventory.isInvalidated else {
        return
    }
    print("is in transaction: \(inventory.realm?.isInWriteTransaction)") // the second time when this block is executed, this is true
    inventory.realm?.beginWrite() // crash (second time)
    inventory.realm?.delete(inventory)
    try! inventory.realm?.commitWrite(withoutNotifying: [notificationToken])
}

If I replace beginWrite and commitWrite with write block it doesn't crash
    guard !inventory.isInvalidated else {
        return
    }
    print("is in transaction: \(inventory.realm?.isInWriteTransaction)") // always false
    try! inventory.realm?.write { // always works
        inventory.realm?.delete(inventory)
    }

After you delete inventory, inventory.realm will be nil and inventory.realm?.commitWrite() is a no-op as a result.

Oh. My bad! I fixed it by storing realm in a local variable. Thanks!

So. Hello everyone!
Have updated Realm from 2.0.3 to 2.1.1 and constantly get crash.
dispatch_async(dispatch_get_main_queue(), ^
{
RLMRealm* realm = [RLMRealm defaultRealm];
user = [User objectForPrimaryKey: jId];
[realm beginWriteTransaction];

if ( ! user )
{
user = [[User alloc] init];
user.jId = jId;
[realm addObject: user];
}

[user updateDataWithDictionary: dictionary];
[realm commitWriteTransaction];

[user.realm transactionWithBlock: ^
{
...
}];
});

And one new example:
try? realm.write({
itemsToken = items.addNotificationBlock{}
})
This is just example of logic. In write transaction I change the property that trigger some logic with addNotificationBlock.

I am experiencing this issue. I can't seem to have found a way to reproduce it.
Is there any way I can help getting to the bottom of this?
I do have something like:
[realm beginWriteTransaction];
Objects *objs = [self allObjects];
...
[realm commitTransaction];

It truly has a query inside the write transactions, but it doesn't crash all the times

I have gotten a few more crashlogs about this.
From the crash logs it appears that the problem occurs when changes are being made that fire change notifications to collections. The problem seems to me, is that those changes are being fired before from within the transaction.
I can confirm to you that I am not adding notification blocks from within write transactions, but I am surely making changes/deleting objects that were in collections that have notification blocks adhered to them.

@jpsim is this issue being tracked here? or on #4515?
My app is crashing a lot due to this issue. I've tried to remove all queries from my transactions, but it still crashes :(.

The issue tracked here appears to be distinct from the one tracked in #4515. The best way to help us fix this is to show us how we can reproduce it.
Seems like you're hitting this quite a bit, and therefore are in the best position to show us how we can trigger it ourselves.

I've opened issue #4671 which is the description of my problem. Whatever code that I have that can possibly reproduce my issue.

I solved this.
Hello Guys, I recently experimented this issue after update Realm, and I figured out I was posting a notification: NotificationCenter.default.post(name: "notificationName", object: nil) just withing a notification block, and here was my code:
notificationToken = results.addNotificationBlock { [weak self] (changes: RealmCollectionChange) in
            guard let tableView = self?.tableView else { return }
            switch changes {

            case .initial:

                tableView.reloadData()
                break
            case .update(_, _, _, _):

                NotificationCenter.default.post(name: "notificationName", object: nil)
                break

            case .error(let error):

                fatalError("\(error)")

                break
            }
        }

and, I was subscribed to hear that notification in order to make a query of the same kind of objects being updated. I removed that bunch of code and the issue was solved. I hope this can help you.

I was able to reproduce this issue in my code with the following setup:
If there's a write operation happening in one thread and your code is trying to create a notification block in another thread while the write operation is still running the exception will rise.
Each thread has it own let realm = try! Realm()
The write thread is using try! realm.write { } block while the read operation has simply queried using realm.objects(SomeRealmObject.self) and then tried to execute addNotificationBlock on the result of that

after upgrading to 2.6.2 (from 1.1.0) I see same crash in fabric
Cannot create asynchronous query while in a write transaction

Fatal Exception: RLMException
0  CoreFoundation                 0x186596fd8 __exceptionPreprocess
1  libobjc.A.dylib                0x184ff8538 objc_exception_throw
2  Realm                          0x100d1c294 (Missing)
3  Realm                          0x100d1c558 (Missing)
4  Realm                          0x100d1c51c (Missing)
5  MyApp                          0x1001194b0 -[MMRealmServiceBase performSyncTransactionBlock:] (MMRealmServiceBase.m:66)
6  MyApp                          0x1001b47f4 -[MMUserServiceImplementation changeSetNumber:forRake:] (MMUserServiceImplementation.m:199)
7  MyApp                          0x100167d54 -[MMJoinJourneyInteractor changeSetNumber:forRake:] (MMJoinJourneyInteractor.m:232)
8  MyApp                          0x1001627b0 -[MMJoinJourneyPresenter didChangeSetNumber:forRake:] (MMJoinJourneyPresenter.m:111)
9  MyApp                          0x10016f0cc -[MMJoinJourneyViewController journeyItemDidUpdateSetNumber:forRake:] (MMJoinJourneyViewController.m:182)
10 MyApp                          0x1000b4248 -[MMJourneyItemViewCell actionEventSetNumber] (MMJourneyItemViewCell.m:152)
11 MyApp                          0x1000b43d4 -[MMJourneyItemViewCell textFieldDidEndEditing:] (MMJourneyItemViewCell.m:165)
12 UIKit                          0x18c7edc08 -[UITextField _resignFirstResponder]
13 UIKit                          0x18c7c4bcc -[UIResponder _finishResignFirstResponder]
14 UIKit                          0x18d125e80 -[UITextField _finishResignFirstResponder]
15 UIKit                          0x18c7693b0 -[UIResponder resignFirstResponder]
16 UIKit                          0x18c7ed814 -[UITextField resignFirstResponder]
17 UIKit                          0x18c84351c -[UIView(UITextField) endEditing:]
18 MyApp                          0x100140220 -[MMPushAnimator animateTransition:] (MMNavigationController.m:27)
19 UIKit                          0x18c85bca0 -[UINavigationController _startCustomTransition:]
20 UIKit                          0x18c775c4c -[UINavigationController _startDeferredTransitionIfNeeded:]
21 UIKit                          0x18c7758b4 -[UINavigationController __viewWillLayoutSubviews]
22 UIKit                          0x18c775818 -[UILayoutContainerView layoutSubviews]
23 UIKit                          0x18c6bc158 -[UIView(CALayerDelegate) layoutSublayersOfLayer:]
24 QuartzCore                     0x1898ac274 -[CALayer layoutSublayers]
25 QuartzCore                     0x1898a0de8 CA::Layer::layout_if_needed(CA::Transaction*)
26 QuartzCore                     0x1898a0ca8 CA::Layer::layout_and_display_if_needed(CA::Transaction*)
27 QuartzCore                     0x18981c360 CA::Context::commit_transaction(CA::Transaction*)
28 QuartzCore                     0x1898433c0 CA::Transaction::commit()
29 QuartzCore                     0x189843e8c CA::Transaction::observer_callback(__CFRunLoopObserver*, unsigned long, void*)
30 CoreFoundation                 0x1865449a0 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__
31 CoreFoundation                 0x186542628 __CFRunLoopDoObservers
32 CoreFoundation                 0x186542a74 __CFRunLoopRun
33 CoreFoundation                 0x186472d94 CFRunLoopRunSpecific
34 GraphicsServices               0x187edc074 GSEventRunModal
35 UIKit                          0x18c724130 UIApplicationMain
36 MyApp                          0x1001295fc main (main.m:14)
37 libdyld.dylib                  0x18548159c start

I do not see here addNotificationBlock inside and cannot reproduce till now, but it happens often in production version

by the way, I have reproduced our problem and cannot find the reason. realm inWriteTransaction returns NO just before beginTransaction, but than it fails with exception and realm inWriteTransaction in that moment returns YES. And there is no other threads with realm write transactions in that moment


Issue 2022 eventincr

Description
Fixes and unskips the tests incremental integration tests
for exchange calendar events.
Currently blocked on LynneR displaying the calendar
fetch issues where it's only possible to retrieve her default
Calendar, Birthdays, and US Holidays calendars.
Does this PR need a docs update or release note?

[x] :no_entry: No

Type of change

[x] :robot: Test

Issue(s)

#2022

Test Plan

[x] :muscle: Manual
[x] :green_heart: E2E

this one may actually need test updates?


Index page

Layout:

Divided into sections (background color?)

Content:

Short introduction of guild
Calendar (list view with pictures)
Instagram widget

Closed by #45


go1.19 std h1/h2 native unsafe http

The PR covers the following issues:

Closes #22
Closes #85

@Mzack9999 when this pr be merged any timeline?


Removed build warnings when using iOS 8 or 10.10 SDKs for the Date code.

@segiddins I've removed the build warnings for the iOS 8 and OS X 10.10 SDKs.
I have code for the RKISO8601DateFormatter to deal with those build warnings as well, but I can't figure out how to initiate a pull request for that code, since @blakewatters original repo doesn't have the 'RK'-prefixed version of the formatter and I don't see it in any convenient location in the RK Repo.
Please advise where/how I should open up a PR for RKISO8601DateFormatter. Thanks.

@samkrishna can you please take a look at the test failures?

:+1: once my comments are addressed.

@samkrishna ping?

Please address the comments so this can be merged.

@segiddins This is the same as #2238. Even if the code style does not comply with personal preference of code layout, @samkrishna provided a valid bugfix.  Just change the two lines on your own ...

@honkmaster I prefer that the code in RestKit stays consistent, otherwise there's unnecessary cognitive overhead when trying to understand the code.

I understand this, however the PR is valid and not to merge it just because of personal preference is strange. A simple merge and changing the two lines on your own would have taken less time than to reply and request for change by @samkrishna here on GitHub.

@segiddins I want to respectfully point out that much of the code formatting in 'RKHTTPUtilities.m' is formatted in a way you would flag TODAY if you were to be dealing with those PRs now. At the time of this original PR submission, I attempted to conform to what the rest of the source file's formatting style looked like (as ugly as IT IS).
In any case, I've re-arranged some code inside the preprocessor checks and formatted it in One True Brace style with what I think is your style guidance.
Please accept this PR.

Your changes look good, but it seems there are a bunch of other unrelated commits in the PR now

OK -- I'll start another PR.


Refactoring

This is a big PR, but if you go commit-by-commit it should be approachable.
Poetry
I've switched to https://python-poetry.org/ for dependency (+ virtualenv) management, you'll want to install it. Note that Poetry isn't required for users to install it when you release it to PyPI. Once you have installed it, run poetry install to get the dependencies.
Working with linter
To see all the "violations" currently in the project, run:
poetry run flakeheaven lint --baseline /dev/null

This will result in around 200 nits, mostly about code complexity and magic numbers. I don't think refactoring all this is feasible in one go – this is why I've set up a baseline for the linter. If you run it like this:
poetry run flakeheaven lint

right now you'll see zero errors. However, if you happen to change anything, it will force you to refactor this part of code as well.
I'm not sure what's the approach will work for you, but I'll be glad if it helps even a bit!
Next steps

Publish to PyPI (poetry publish --build should do the trick)

Write up some more docs, perhaps as Sphinx site on ReadTheDocs?

Write up more examples (e. g. what if we use sounddevice to record audio, then convert it to an AudioSegment and pass to recognize_song?)

Closes #7

Really great improvements. What about minimum python version, is it really need 3.8+ or we can make it possible to use even on 3.7? I still actually use this version :)

I think 3.8 is needed for some of the devDependencies – I can see if there's any way around this.
Note that EOL for 3.7 is next June – you have just slightly more than a year to upgrade :-)

Turns out NumPy has also dropped Python 3.7 support starting with the 1.22.0 release. I've downgraded it to ^1.21.1 for the time being, and made pydub-stubs and ipython optional dependencies.

oops, it seems typing.Final has been released in python 3.8. As i don't have too much experience in python, what do you think about support for 3.7, do we need it here?

I'm not sure – that's totally up to you :D We can get rid of Finals though, they're here only for type checker strictness. Another option is to use typing_extensions.Final.

i guess typing_extensions.Final would be better

On a slightly different note, here's how you can do the microphone recognition: https://codeberg.org/notpushkin/shazamctl/src/branch/master/shazamctl/microphone.py 🎉

@Numenorean hey! Is there anything left before we can merge this in?


Export Svg components in the Svg/index.js file

This PR exports all Svg components (Circle, Rect, etc...) from the lib/components/Svg/index.js file.
This not only supports tree-shaking, it also improves the compatibility towards the react-native-svg API (especially when using tools like SVGR in combination with TypeScript).
Fixes: https://github.com/airbnb/react-sketchapp/issues/420

Thanks!


Handle .^can on roles

13:00   hackedNODE  s: Hash, "new", \( )
13:00   SourceBaby  hackedNODE, Sauce is at https://github.com/rakudo/rakudo/blob/f0b3b59/src/core/Map.pm#L12
13:00   hackedNODE  s: Blob, "new", \( )
13:00   SourceBaby  hackedNODE, Something's wrong: ␤ERR: Too many positionals passed; expected 2 arguments but got 3␤  in sub sourcery at /home/zoffix/services/lib/CoreHackers-Sourcery/lib/CoreHackers/Sourcery.pm6 (CoreHackers::Sourcery) line 25␤  in block <unit> at -e line 6␤␤

I just wanted to take a note of this, then I saw Zoffix was faster by a solid 6 years. :) Anyways, gonna look into it when there's time (wouldn't say it's high on my priority list, however it does seem interesting).


[AS3] Wrong function generation behavior

Let's have
private function f1(f:Function):void{}

private function f2():void
{
	f1(f3);
}

when generate new function f3 we got
private function f3():Function
expected
private function f3():void

it is for me :)

Why private function f3():void? Why not f3();* or f3(... args):*. On some cases maybe we can get the exact signature, but here it's not possible.

Reasonable, but ":Function" not good anyway, If it's not void you should fix it manually, but with Function you should fix it in any case


isSignedIn() does not check session file

There is no way to silently check if user has already logged in before. SignIn loads the session and forces the user to log in. I can't forcefully check only facebook if i have a few logging in methods. Perhaps there is hidden loadSession() function somewhere?

FB SDK has removed the silent login a few versions ago. So right now the login is either silent because the SDK already has credentials it can use or the login windows will appear. No way to control this behavior.

So any clue on how can i check if session file is still correct without logging in?

You can verify session tokens against the graph API. https://graph.facebook.com/app/?access_token=[user_access_token] But there are some limitations. Read the doc!

But why does the lib not provide that? Why can't it just log in once, and then have checkSessionFileToken() function. There is already a file with session token for "silent login". So why do users need to write their own checking on top of it? Even if I'd code it myself, then I can't get token from it until it is going to load the session file, which it does only if I force the signIn().
So what I would need to do is to write it a hacky way on top of this lib, by storing session token after user once has signed in, because there is no function to get the key from the file, nor to check it out of the box.
I know that I can just write my own stuff, but why is the lib not doing this for me?

If you think there is feature missing, best thing would be to add it and make PR.
But keep in mind that token storage will be changed in sometime in future: https://github.com/TomGrill/gdx-facebook/issues/28


Vulkan 1.1: only RTE and RTZ are permitted with FPRoundingMode

https://github.com/KhronosGroup/Vulkan-Docs/blob/master/appendices/spirvenv.txt#L425

https://github.com/KhronosGroup/SPIRV-Tools/pull/4088

https://github.com/KhronosGroup/SPIRV-Tools/pull/4088


Define manual tests for user management

Please, someone check the new test template and add tests to the After Login section. I don't know of all the expected behaviours and KnetMiner doesn't seem to recognise me as a Pro User (KnetSpace says I have that role).
Also, please, run the new version of that test (the current one works fine).

@marco-brandizi will do once I have a gap...

@marco-brandizi will do once I have a gap...

yes, not urgent, thanks.

@marco-brandizi this is ready for review. Since you don't have an account which is registered but free, you would need to make one to test thoroughly. I've made sure it works, but you can read over it and mark this as complete if you're happy.
@lawal-olaotan if you have time, feel free to review the After Login section.


Can't input token

Describe the bug
When I run the git command given in the README.md file it works when I put the ID but doesn't paste the token
Expected behavior
I thought it would paste but it did not
Discord Version:
Desktop - App
version: Stable 82705 (15996b1)
Additional context
Python version: 3.9

What operating system?

What operating system?

MacOS

I made it so that when you paste the token, it doesnt show inside the terminal. it works kind of like a password, it is hidden. Try running the setup command and paste your token in, then hit enter. Once the command finishes, you can check by running cat .env and you should see the contents of the file, and your token should be in there.


Multiple calendars

Hi,
Cal4Desktop (https://www.somebytes.de/produkte/cal4desktop/) offers the solution to show multiple calendars on the Desktop. I've seen in other requests that it is not possible for you to program this in your app.
However, could you find a way to still implement this absolutely necessary function in your nice app?? Maybe you could ask the Cal4Desktop developers how they did this?
I so much need to see all of my Outlook Calendars on the Desktop and your app looks really nice so I really would like to see this functionality inside your program.

Hello ^-^
Thank you for the wonderful app!
I add my voice to OP; right now the app is not usable for me since I have work calendar & personal calendar and I have no way of showing both together :(
I really like the functionality and the look of your app (much better than cal4desktop)


Opening a project sometimes results in a CPS fault in the StartupProjectRegistrar

Open a console app a few times. Sometimes, it results in a CPS fault with this stack:
9/27/2016 6:08:15 PM
Recoverable
System.FormatException: Unrecognized Guid format.
   at System.Guid.GuidResult.SetFailure(ParseFailureKind failure, String failureMessageID, Object failureMessageFormatArgument, String failureArgumentName, Exception innerException)
   at System.Guid.TryParseGuid(String g, GuidStyles flags, GuidResult& result)
   at System.Guid..ctor(String g)
   at Microsoft.VisualStudio.ProjectSystem.VS.Debug.StartupProjectRegistrar.<InitializeCoreAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Debug.StartupProjectRegistrar.<OnProjectFactoryCompletedAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.UnconfiguredProjectImpl.AutoLoadMethodStateMachine.<<StartExecution>b__8_0>d.MoveNext()

This sounds like an issue with the ImplicitProjectPropertiesProvider again. Maybe a race between when the ProjectGuid property gets set and this code gets called?

A quick fix to unblock rest of the team would be to first TryParseGuid before attempting to create a new Guid with the value read from project properties.

Yep, but not the longer workaround. This thing needs to set the correct GUID.

@srivatsn I have not been able to repro this.

is anyone else hitting this?

I am hitting this way too often - almost 70-80% of all my debug sessions. Can you just try F5 latest Project-System to dev hive and close/open a net core project multiple times?

I saw this issue too with web projects, so it seems to be generic issue for core and web... @basoundr  investigating it on my machine now

Fixed by #555


NFT transfer

implemented: https://github.com/near/near-wallet/issues/2071

hey guys, just updated this PR, please take a second look.
And there are several differences between figma design and existing components, like:

The NFT transfer modal width in figma is 434px, however the current modal doesn't have any presets matching this. Shall I add new mid-large size??
The  receiver id input box design doesn't match existing component ReceiverInputWithLabel, changing the component to match design requires many efforts and may affect other places. I would suggest to stick to the current one.

For me I'll try to reuse existing components as much as possible, unless they cannot meet the UX requirement.
Also, the Proof of Authenticity section has two links, where should they link to? Not all NFTs are stored on IPFS I think, should we keep the View on IPFS link?

hey guys, just updated this PR, please take a second look.
And there are several differences between figma design and existing components, like:

The NFT transfer modal width in figma is 434px, however the current modal doesn't have any presets matching this. Shall I add new mid-large size??
The  receiver id input box design doesn't match existing component ReceiverInputWithLabel, changing the component to match design requires many efforts and may affect other places. I would suggest to stick to the current one.

For me I'll try to reuse existing components as much as possible, unless they cannot meet the UX requirement.
Also, the Proof of Authenticity section has two links, where should they link to? Not all NFTs are stored on IPFS I think, should we keep the View on IPFS link?

Thanks for the updates @danielwpz! Great job! 👍

The NFT transfer modal width in figma is 434px, however the current modal doesn't have any presets matching this. Shall I add new mid-large size??

The current size is fine. No need to add a new size

The  receiver id input box design doesn't match existing component ReceiverInputWithLabel, changing the component to match design requires many efforts and may affect other places. I would suggest to stick to the current one.

Good point - I think the current one is fine

For me I'll try to reuse existing components as much as possible, unless they cannot meet the UX requirement.

Perfect! The more we can re-use, the better :)

Also, the Proof of Authenticity section has two links, where should they link to? Not all NFTs are stored on IPFS I think, should we keep the View on IPFS link?

You're right - the links aren't necessary for this first version

For some reason I still don't get the images rendered on the NFT details page or the modal - I don't think it's my machine because they render fine on the NFT collection page. Any idea why that is?
Can we hide the ’Transfer’ button if it’s not owner looking at the NFT details page? I noticed that when I switch account on an NFT details page, the button is still there, though disabled. I think we can remove it completely in that case.
Can we use the <BackArrowButton> under /common/BackArrowButton for the NFT details page? We already use it on the send token 'enter receiver' step (https://wallet.near.org/send-money)
I think we'll need to re-fetch NFTs after sending so that both sender and receiver have the latest state. Right now the NFT is still visible in the senders collection after sending. Also, once the NFT has been sent and the user clicks the 'Continue' button, can we send the user back to the full NFT collection page vs staying on the NFT details page?
We remove the ‘Confirm transfer’ heading on the confirmation step - The Figma might have been updated after (https://www.figma.com/file/Co5Rx8i9Cx4SxgRZv0MUa7/NFTs?node-id=426%3A387)

@Patrick1904 hi Patrick, thanks for all the feedbacks. will push an update today!

@Patrick1904 what's the best way to trigger a reload on NFT list page? shall I add a url param indicates whether it should reload or not?

@Patrick1904 what's the best way to trigger a reload on NFT list page? shall I add a url param indicates whether it should reload or not?

@danielwpz you can call dispatch(fetchNFTs({ accountId })); after successfully sending the NFT so that the 'transfer' button disappears and all current NFTs are updated when the user navigates back to the gallery

@Patrick1904 hey, all comments are resolved now.
However when I rebased master I could not run it. Seems some feature flag env is missing if just do yarn start

@Patrick1904 hey, all comments are resolved now.
However when I rebased master I could not run it. Seems some feature flag env is missing if just do yarn start

Thanks for making the changes @danielwpz 🙏
This is looking good to me! I just added a few small change requests inline :)

sorry for the late update, didn't get a chance to update this PR  last few days.
@Patrick1904 fixed all latest comments, if it looks good I'll rebase master and make this ready to merge.
Thanks!

It seems that the NFTBox mapping is missing a key - can you please fix that?

sorry for the late update, didn't get a chance to update this PR last few days. @Patrick1904 fixed all latest comments, if it looks good I'll rebase master and make this ready to merge. Thanks!

Thanks @danielwpz ! Please go ahead and rebase and I'll do final round of testing after and I'll do the same before we merge.

Hey, what is the status of this PR? People message me almost every day asking when they can transfer and mint NFTs in the wallet directly. I'm willing to do whatever I can to help here! @Patrick1904 @MaximusHaximus

Hey, what is the status of this PR? People message me almost every day asking when they can transfer and mint NFTs in the wallet directly. I'm willing to do whatever I can to help here! @Patrick1904 @MaximusHaximus

@cjd425 this PR should be very close to ready. @MaximusHaximus had some minor requests in the last round of reviews that seemed to have been addressed by @danielwpz. @MaximusHaximus can you please verify that you're ok with these changes and we can rebase the PR so that it's aligned with master and then merge.

@Patrick1904 @MaximusHaximus I too am curious about this one as it seems really beneficial to the ecosystem at large. Do we have an estimate on when this one can merge?

@Patrick1904 @MaximusHaximus Hey Patrick and Daryl, what's the status of the feature now? please let me if you need any help that could help us to resolve the issues and merge the feature into production.

@Patrick1904 @MaximusHaximus Hey Patrick and Daryl, what's the status of the feature now? please let me if you need any help. I think we can figure out the missing part, resolve the issues and merge the feature into production.

@think-in-universe this PR should be almost ready for merge. 👍 There are still some things that weren't addressed after @MaximusHaximus's review that we want to fix first, and @andy-haynes will make the final touches before we ship this.

@Patrick1904 thanks Patrick! good to know. let's fix the the issues and move forward 😄

Closing this issue as it's been ported over and shipped in: https://github.com/near/near-wallet/pull/2514
The feature is currently available on testnet/staging and will ship to mainnet with the next release
Thank you @danielwpz for your work and patience in getting this shipped 🙏

great job! thanks everyone @danielwpz @Patrick1904 @andy-haynes @MaximusHaximus


[FR] Add a new command to manipulate tags on existing snapshots.

Sometimes it is handy to manipulate snapshots based on their tags, however to make that universal useful we need a way to change them on existing snapshots.
Example I was thinking about was progressive insight into snapshots are managed by some enterprising sysadmin. Without the possibility to change tags retroactively, they would have to make exceptions to their newly thought of schema, till the old snapshots are forgotten.
Unavoidable implementation detail, due to the tags changing, and therefore the sha256 hash, the snapshots will get a new ID assigned. All other data stays the same however.
Possible cmdline:
$ ../../../bin/restic -r rest:http://qnap.********:8000 tag --help
The "tag" command allows you to modify tags on existing snapshots.

You can either set/replace the entire set of tags on a snapshot, or
add tags to/remove tags from the existing set.

When no snapshot-ID is given, all snapshots matching the host, tag and path filter criteria are modified.

Usage:
  restic tag [flags] [snapshot-ID ...]

Flags:
      --add tag       tag which will be added to the existing tags, can be given multiple times
  -H, --host string   only consider snapshots for this host, when no snapshot ID is given
      --path path     only consider snapshots which include this (absolute) path, when no snapshot-ID is given
      --remove tag    tag which will be removed from the existing tags, can be given multiple times
      --set tag       tag which will replace the existing tags, can be given multiple times
      --tag tag       only consider snapshots which include this tag, when no snapshot-ID is given

Global Flags:
      --json                   set output mode to JSON for commands that support it
      --no-lock                do not lock the repo, this allows some operations on read-only repos
  -p, --password-file string   read the repository password from a file
  -q, --quiet                  do not output comprehensive progress report
  -r, --repo string            repository to backup to or restore from (default: $RESTIC_REPOSITORY)

Good point, I like it!
I've thought about how to address the problem with the changing snapshot ID, we can either just leave it and accept that sometimes snapshot IDs change. At the moment (as far as I remember) there's nothing in the code that depends on static snapshot IDs.
I have plans for adding a local cache which uses snapshot IDs to cache things (e.g. the list of referenced blobs, so that we can speed up prune a lot by not traversing a snapshot again). We can mitigate this by adding the original snapshot ID to the changed snapshot as a new field. What do you think?

I think that would be asking for problems, first off, we can have multiple tag changes, so the definition of original snapshot id becomes murky fast. Secondly we don't know at what point the user caches the metadata, so we have no idea where the "original-id" should point at. Preferably the one they have cached :) but we can have multiple users with caches at different times...

Hm, at the moment there's no metadata cache yet, so defining one (and the proper semantics of caching) is up to us. Maybe an example is better suited to explain what I had in mind:
Let's say we have a snapshot without any tags, like this (from doc/Design.md), saved as id 22a5af1b:
{
  "time": "2015-01-02T18:10:50.895208559+01:00",
  "tree": "2da81727b6585232894cfbb8f8bdab8d1eccd3d8f7c92bc934d62e62e618ffdf",
  "dir": "/tmp/testdata",
  "hostname": "kasimir",
  "username": "fd0",
  "uid": 1000,
  "gid": 100
}

Then, a tag foo is added, the original snapshot is deleted and a new document is stored in the repo, with the storage ID 4af9501b:
{
  "time": "2015-01-02T18:10:50.895208559+01:00",
  "tree": "2da81727b6585232894cfbb8f8bdab8d1eccd3d8f7c92bc934d62e62e618ffdf",
  "dir": "/tmp/testdata",
  "hostname": "kasimir",
  "username": "fd0",
  "uid": 1000,
  "gid": 100,
  "tags": ["foo"],
  "id": "22a5af1bdc6e616f8a29579458c49627e01b32210d09adb288d1ecda7c5711ec"
}

When the tags bar and baz are added and the foo tag is removed, the snapshot document is deleted from the repo and a new one is stored, e.g. with storage ID ec25006c:
{
  "time": "2015-01-02T18:10:50.895208559+01:00",
  "tree": "2da81727b6585232894cfbb8f8bdab8d1eccd3d8f7c92bc934d62e62e618ffdf",
  "dir": "/tmp/testdata",
  "hostname": "kasimir",
  "username": "fd0",
  "uid": 1000,
  "gid": 100,
  "tags": ["bar", "baz"],
  "id": "22a5af1bdc6e616f8a29579458c49627e01b32210d09adb288d1ecda7c5711ec"
}

So for the first snapshot (which does not have an id field), the ID is the storage hash (this is how it's handled in restic today), for the second and third snapshot the ID is the contents of the id field, so we know that this is the same snapshot as the first one, module metadata.
For caching of the snapshots themselves we can just use the storage hashes, they will change whenever metadata changes (e.g. the tags). For caching of other data related to the snapshot, we can use the original snapshot ID (if it has one), otherwise the storage hash.
Was that clearer?

Clearer and more scarier :) I'll make the necessary changes to my PR
Nit: Should it be called persistent_id?

Hm, there's no non-persistent ID, or any other ID for that matter, so I think id is better. Why is it scarier? What do you have in mind that can go wrong? Even if the same snapshot (with different tags) is cached somewhere, the ID field allows restic to group the same snapshots together...

Ok, id it is.
Scarier in the sense that we now have data we depend on, which is not really discoverable/recoverable by re-indexing or what every means of recovery we apply. Then again, if we lose the id field somehow, we can invalidate the cache and it would index using the new snapshot id. So as long that will be the only dependency we're good. I just fear creeping featurism.

Not cool. type Shapshot already has an non-exported id field. Guess we will have a originalID fieldname afterall (which I will expose in json as id).

Hm, let me check if we need the id field for Snapshot at all...

Uh, yes, it's needed. Never mind.

Actually, grumble. I can't even expose it as id. cmd_snapshot creates it's own type Snapshot to expose as such.

Then maybe the other field name you suggested is the better choice after all. Some parts of the restic code are not as polished as they could be, I'm afraid.