Entity flickers when using availability

Open this Sandcastle, speed up the time, and pause it once you start to see flickering.
Came up on the forum. It looks like availability.contains(time) inside Entity.prototype.isAvailable  is returning true even after the time is outside the range. More strangely, it alternates between true and false even when the input time does not change?

@OmarShehata the use of setInterval in this sandcastle example doesn't make sense.  It's adding a new polyline each second the example is running.  I'm pretty sure the problem is with the end user code, not with Cesium.  I couldn't reproduce the problem using a for loop to add lines.

Congratulations on closing the issue! I found these Cesium forum links in the comments above:
https://groups.google.com/forum/#!topic/cesium-dev/wtcrYum0dTA
If this issue affects any of these threads, please post a comment like the following:

The issue at https://github.com/AnalyticalGraphicsInc/cesium/issues/7048 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.
:earth_africa: :earth_americas: :earth_asia:

@hpinkos why does using setInterval not make sense? The scenario this is simulating is data that comes from a server, so it wouldn't all be available from the beginning. Why does it matter when the entities are created? I feel like the fact that adding new entities causes other entities with availability whose end time has passed to be visible again is a bug.
I think this is the expected behavior. My workaround here is removing the entities after their availability is up.

@OmarShehata I'm sorry, you're right.  I was thrown off by how the example was written.  I've created a minimal version to reproduce the problem here: Example
Every time you press the add sample button it does show lines that are past the availability.

@hpinkos did you fix a similar bug that had to do with outdated show properties being used when the primitives are re-created?  This could be a similar issue where show isn't updated until after the first render instead of using the current value at creation time.

I'm the original poster on the forums.
This PR fixes a regression caused by #6835 which resolved issues #2909, #4714, #6830, and #5371.
I confirmed each of these issues and they are all fixed with this PR.
I was reluctant to create a pull request that removes this code because there must be a reason why it exists.  I still can't figure that out.  But I've run every test I can and everything seems to work OK.

Congratulations on closing the issue! I found these Cesium forum links in the comments above:
https://groups.google.com/forum/#!topic/cesium-dev/wtcrYum0dTA
If this issue affects any of these threads, please post a comment like the following:

The issue at https://github.com/AnalyticalGraphicsInc/cesium/issues/7048 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.
:earth_africa: :earth_americas: :earth_asia:


Memory use of genome digest init

with the new genome digest calculation during initialization, we're now using henge in the back-end. Unfortunatley when I ran this on some real data, soemf asta files (I presume the larger ones) are getting killed due to memory overages:
grep error ../genomes/submission/*.log
../genomes/submission/refgenie_Zea_mays_PH207_v1_0-fasta.log:slurmstepd: error: Detected 1 oom-kill event(s) in step 14541017.0 cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
../genomes/submission/refgenie_Zea_mays_PH207_v1_0-fasta.log:srun: error: udc-ba27-11: task 0: Out Of Memory
../genomes/submission/refgenie_Zea_mays_PH207_v1_0-fasta.log:slurmstepd: error: Detected 1 oom-kill event(s) in step 14541017.batch cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
../genomes/submission/refgenie_Ziziphus_jujuba__RefSeq_v1_1-fasta.log:yacman.exceptions.AliasError: Provided function '<lambda>' errored: KeyError('aliases',)
../genomes/submission/refgenie_Zostera_marina_JGI_v2_2-fasta.log:yacman.exceptions.AliasError: Provided function '<lambda>' errored: KeyError('aliases',)
../genomes/submission/refgenie_Zoysia_japonica_ssp__nagirizaki_Zoysia_Genome_Database_v1_1-fasta.log:yacman.exceptions.AliasError: Provided function '<lambda>' errored: KeyError('aliases',)

presumably, we could compute these digests without loading the whole fasta file into memory, or somehow otherwise make sure the memory use is more reasonable.

I've now tried these with 18 GB of memory and they are still getting out of memory errors (for about 3Gb genomes). So, this is a bigger problem than I thought; we really have to revise this approach to computing checksums.

alright, I've spent quite some time investigating and I think I've got it...
Performance comparison
from refgenconf.seqcol import SeqColClient
ssc = SeqColClient({})
c, a = ssc.load_fasta_old(fa_file="hg38.fa.gz", gzipped=True)

I used a gzipped hg38 FASTA (~3GB) to compare the old and new approach
old:

time: 35m24.652s
memory: ~8.7GB

fixed:

time: 1m13.119s
memory: ~3.1GB

Fixes
There were two issues that turned out to be completely unrelated:

Long execution time was caused by the pyfaidx.Fasta() call where we read the file in and  then iterate through the sequences again to process them before inserting to the Henge database. Fixed in a670e03, where I just decided to read the FASTA file line by line and process it on the fly, then insert to the Henge database, just as before.

Large memory usage was caused by lines like this (!!!!!):

https://github.com/refgenie/refgenconf/blob/d7db570ba245ee9217629d877e496f10a6dae45f/refgenconf/henge.py#L185
Python allocated huge chunk of memory for every item it was supposed to print, even if run in non debug mode... Removing debug lines solved the memory issue.

wow, that is appalling. nice sleuthing! 2 questions:

does this eliminate the need to have pyfaidx in the requirements?
here we have the issue with embedding seqcol/henge here -- what's your plan for maintaining these? Just keep the development effort here for now and then eventually paste it over those other repos?

yes, the old fasta parser is not required anymore, so pyfaidx requirement is gone
I didn't think about it, but this seems to be the best option for now. Alternatively, we could just update henge and seqcol every time we release stable version of refgenconf.

and the new approach handles gzipped fasta files (with out .gz extensions) correctly, right?
https://github.com/refgenie/refgenconf/pull/102/commits/d198ea5e31c6f7f9cb33ddd9a417ad6bfb33861d

yes, as long as you determine whether it is gzipped or not gzipped file upfront.

the signature and returned value did not change, the changes are internal.

This has improved things, but I think there may be something wrong in your calculation, as my jobs are being given 6GB of memory but they are getting killed by slurm with out-of-memory errors on genomes that have only 3Gb of disk
See this one for example:

zipped fastq size: 2.8Gb
time: 3m +
memory: ~10.7GB

the file I used is this one: /project/shefflab/www/refgenie_plantref/Triticum_turgidum_Svevo_v1_0-fasta-fasta

Interesting -- I got similar numbers to you on the hg38 genome, though:

it only used about 1/4 of the memory. Must have something to do with size of the chromosomes? That plant genome has only 14 chromosomes

what is the size of that plant genome unzipped? hg38 is 2.9GB unzipped and 833.4MB zipped.
Maybe the plant one unzipped is around 10GB and the memory scales with the file size as expected?

Ah, that makes sense. Ok.
Interesting -- when you run refgenie, the memory use continues to increase, it goes past 12Gb. So some other stuff after that function call consumes more memory. Is there any way we can garbage collect or something?

It seems like it should be possible to compute the genome checksum without loading the whole thing into memory. We only need to consider one chromosome at a time.

The command I'm running is this one:
bulker run databio/refgenie:0.7.6 refgenie build Triticum_aestivum_Ensembl_Plants_TGACv1/fasta --genome-description 'Triticum aestivum Ensembl Plants TGACv1' --files fasta=/project/shefflab/www/refgenie_plantref/Triticum_aestivum_Ensembl_Plants_TGACv1-fasta-fasta

it took more than 16 minutes, and more than 12Gb of memory to initialize the genome here. So, most of the time, and the max memory mark, are actually outside of that function all.
So,
ssc.load_fasta:

3 min
10.7 GB mem

entire initialize process:

~17 min
~14 GB memory

are you sure you're referring to the same genome in both cases?
I tried building fasta asset for Triticum_aestivum_Ensembl_Plants_TGACv1 and then running just the load_fasta method in iPython and got exactly the same results close to 17mins and 13GB RAM usage. Here is a screenshot from the load_fasta method call:

you are correct. I was mistaken. Thanks. This explains the discrepancy.
But this point still stands: It seems like it should be possible to compute the genome checksum without loading the whole thing into memory. We only need to consider one chromosome at a time.
It's unfortunate that the initialization procedure takes upwards of 20 minutes and 14 GB of RAM for a real-world use case genome. The actual FASTA asset building step takes only 3 or 4 seconds and no RAM :). Not sure we can solve this completely, but it's worth thinking about.

It seems like it should be possible to compute the genome checksum without loading the whole thing into memory. We only need to consider one chromosome at a time.

right, for this particular use case (to determine a digest and create annotated sequence digest list that we use for genome compatibility checks) it is possible to split it by chromosome and spare memory. In general, if you wanted to store the sequences for a downstream retrieval the RAM use has to be >FASTA file size, unless you back the object with a DB.
Soo.. with the recent commit the algorithm now computes the digest prior to inserting the data to the Henge and indeed it reduces RAM use, which is now a function of single sequence size, not FASTA file size.
As for time, I don't think there's anything we can do. The calculations that need to happen are pretty much the same in case you consider the FASTA file as a whole or split it by sequence.
Results
For hg38 (3GB):

time: ~1min
memory: high water mark of around 900MB

The performance boost is even more apparent in case of Triticum_aestivum_Ensembl_Plants_TGACv1-fasta-fasta file, RAM < 1GB:
	Command being timed: "python test.py"
	User time (seconds): 591.65
	System time (seconds): 3.86
	Percent of CPU this job got: 98%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 10:02.98
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 874760
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 665638
	Voluntary context switches: 2846
	Involuntary context switches: 9604
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

yeah I noticed -- this appears to have made things much more efficient; at least for this use case. you're right that we can't use this approach if we're actually interested in storing the sequences in the back-end -- but in that case we should still be able to insert them one at a time.
But I'm getting a bunch of new issues now with file locks; not sure if this is unrelated:
I'm seeing this in dozens of results now:
cat ../genomes/submission/refgenie_Zoysia_japonica_ssp__nagirizaki_Zoysia_Genome_Database_v1_1-fasta.log
Compute node: udc-aj37-15c1
Start time: 2020-08-27 10:15:42
The lock has been created in the split second since the last lock existence check. Waiting
Bulker config: /project/shefflab/rivanna_config/bulker.yaml
Activating crate: databio/refgenie:0.7.6
Using 'default' as the default tag for 'Zoysia_japonica_ssp__nagirizaki_Zoysia_Genome_Database_v1_1/fasta'
Recipe validated successfully against a schema: /home/ns5bc/.local/lib/python3.6/site-packages/refgenie/schemas/recipe_schema.yaml
Building 'Zoysia_japonica_ssp__nagirizaki_Zoysia_Genome_Database_v1_1/fasta:default' using 'fasta' recipe
Initializing genome: Zoysia_japonica_ssp__nagirizaki_Zoysia_Genome_Database_v1_1
Loaded AnnotatedSequenceDigestList (11786 sequences)
Traceback (most recent call last):
  File "/home/ns5bc/.local/bin/refgenie", line 10, in <module>
    sys.exit(main())
  File "/home/ns5bc/.local/lib/python3.6/site-packages/refgenie/refgenie.py", line 713, in main
    refgenie_build(gencfg, asset_list[0]["genome"], asset_list, recipe_name, args)
  File "/home/ns5bc/.local/lib/python3.6/site-packages/refgenie/refgenie.py", line 543, in refgenie_build
    alias=ori_genome)
  File "/home/ns5bc/.local/lib/python3.6/site-packages/refgenconf/refgenconf.py", line 1348, in initialize_genome
    with self as rgc:
  File "/home/ns5bc/.local/lib/python3.6/site-packages/yacman/yacman.py", line 110, in __enter__
    self.make_writable()
  File "/home/ns5bc/.local/lib/python3.6/site-packages/yacman/yacman.py", line 216, in make_writable
    create_lock(filepath, getattr(self, WAIT_MAX_KEY, DEFAULT_WAIT_TIME))
  File "/home/ns5bc/.local/lib/python3.6/site-packages/ubiquerg/files.py", line 240, in create_lock
    wait_for_lock(lock_path, wait_max)
  File "/home/ns5bc/.local/lib/python3.6/site-packages/ubiquerg/files.py", line 156, in wait_for_lock
    "file still exists.".format(wait_max))
RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
Waiting for file lock: /project/shefflab/rivanna_config/lock.bulker.yaml .Waiting for file lock: /project/shefflab/deploy/plantref/plantref/config/lock.refgenie_config.yaml ................................................................... File unlocked

is it possible that this system is locking the config file while computing the checksum?

89 out of 156 runs failed due to this issue:
grep maximum ../genomes/submission/*.log
../genomes/submission/refgenie_Amborella_trichopoda_Amborella_V1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reac
hed and the lock file still exists.
../genomes/submission/refgenie_Amborella_trichopoda__JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached a
nd the lock file still exists.
../genomes/submission/refgenie_Ananas_comosus_JGI_v3_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the
lock file still exists.
../genomes/submission/refgenie_Apostasia_shenzhenica_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and t
he lock file still exists.
../genomes/submission/refgenie_Arabidopsis_lyrata_JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
the lock file still exists.
../genomes/submission/refgenie_Asterochloris_sp__Cgr_DA1pho_v2_0_JGI_7_45_13-fasta.log:RuntimeError: The maximum wait time (60) has
 been reached and the lock file still exists.
../genomes/submission/refgenie_Bathycoccus_prasinos_Ghent_University-fasta.log:RuntimeError: The maximum wait time (60) has been re
ached and the lock file still exists.
../genomes/submission/refgenie_Brassica_oleracea__Ensembl_Genomes_v2_1-fasta.log:RuntimeError: The maximum wait time (60) has been
reached and the lock file still exists.
../genomes/submission/refgenie_Carica_papaya_Hawaii_Agriculture_Research_Center-fasta.log:RuntimeError: The maximum wait time (60)
has been reached and the lock file still exists.
../genomes/submission/refgenie_Cenchrus_americanus_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the
 lock file still exists.
../genomes/submission/refgenie_Chenopodium_quinoa__JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
 the lock file still exists.
../genomes/submission/refgenie_Citrus_clementina__JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
the lock file still exists.
../genomes/submission/refgenie_Citrus_sinensis_JGI_v1_assembly_and_v1_0_annotation-fasta.log:RuntimeError: The maximum wait time (6
0) has been reached and the lock file still exists.
../genomes/submission/refgenie_Coccomyxa_subellipsoidea_C-169_JGI_2_0_Phytozome_10_2-fasta.log:RuntimeError: The maximum wait time
(60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Cucumis_melo__Melonomics_v3_5-fasta.log:RuntimeError: The maximum wait time (60) has been reached an
d the lock file still exists.
../genomes/submission/refgenie_Cucumis_sativus_L___JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
 the lock file still exists.
../genomes/submission/refgenie_Dictyostelium_discoideum__ENSEMBL_protist_release_28-fasta.log:RuntimeError: The maximum wait time (
60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Ectocarpus_siliculosus_Ghent_University-fasta.log:RuntimeError: The maximum wait time (60) has been
reached and the lock file still exists.
../genomes/submission/refgenie_Elaeis_guineensis_EG5_1-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the
lock file still exists.
../genomes/submission/refgenie_Emiliania_huxleyi__ENSEMBL_protist_release_28-fasta.log:RuntimeError: The maximum wait time (60) has
 been reached and the lock file still exists.
../genomes/submission/refgenie_Glycine_max_JGI_Glyma1_1_annotation_of_the_chromosome-based_Glyma1_assembly-fasta.log:RuntimeError:
The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Glycine_max__JGI_Wm82_a2_v1-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
the lock file still exists.
../genomes/submission/refgenie_Gossypium_raimondii_JGI_annotation_v2_1_on_assembly_v2_0-fasta.log:RuntimeError: The maximum wait ti
me (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Helicosporidium_sp___Illinois_University_1_0-fasta.log:RuntimeError: The maximum wait time (60) has
been reached and the lock file still exists.
../genomes/submission/refgenie_Hevea_brasiliensis__NCBI_Genomes_v1-fasta.log:RuntimeError: The maximum wait time (60) has been reac
hed and the lock file still exists.
../genomes/submission/refgenie_Homo_sapiens_ENSEMBL_release_81-fasta.log:RuntimeError: The maximum wait time (60) has been reached
and the lock file still exists.
../genomes/submission/refgenie_Hordeum_vulgare_Ensembl_Genomes_ASM32608v1-fasta.log:RuntimeError: The maximum wait time (60) has be
en reached and the lock file still exists.

grep maximum ../genomes/submission/*.log | wc -l
89

So, there must be something wrong with when the config file is getting locked

that's odd, I'm 100% sure nothing has changed in that regard. Maybe failing jobs left the lock in place and that's the issue.I noticed there are a couple of logs with errors other than the RuntimeError, did you look into these?
For example: /project/shefflab/deploy/plantref/genomes/submission/refgenie_Arabidopsis_lyrata__JGI_v2_1-fasta.log
There has been a FileExistsError raised, that I need to resolve anyway, even if it's not related.

I bet you're right that it's failing jobs that left the lock in place or something. that's a problem.

I don't know what is happening, but on a clean slate, I get the exact same set of errors:
grep maximum ../genomes/submission/*.log
../genomes/submission/refgenie_Brassica_oleracea__Ensembl_Genomes_v2_1-fasta.log:RuntimeError: The maximum wait time (60) has been
reached and the lock file still exists.
../genomes/submission/refgenie_Caenorhabditis_elegans_ENSEMBL_release_81-fasta.log:RuntimeError: The maximum wait time (60) has bee
n reached and the lock file still exists.
../genomes/submission/refgenie_Capsella_rubella_JGI_annotation_v1_0_on_assembly_v1-fasta.log:RuntimeError: The maximum wait time (6
0) has been reached and the lock file still exists.
../genomes/submission/refgenie_Carica_papaya__JGI_ASGPB0_4-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
the lock file still exists.
../genomes/submission/refgenie_Chenopodium_quinoa__JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and
 the lock file still exists.
../genomes/submission/refgenie_Chlamydomonas_reinhardtii_JGI_5_5_Phytozome_10_2-fasta.log:RuntimeError: The maximum wait time (60)
has been reached and the lock file still exists.
../genomes/submission/refgenie_Chlamydomonas_reinhardtii_JGI_v5_0_assembly_annot_v5_3_1_based_on_Augustus_u11_6-fasta.log:RuntimeEr
ror: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chlamydomonas_reinhardtii__JGI_v5_5-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.

Something is locking the file for more than 60 seconds. It's not leftover from previous runs. The file is eventually getting unlocked.

I can confirm that in my test, the config file is getting locked during the entire build process.

I narrowed it down to fasta asset builds. So it needs to be a side effect of some recent change in genome initialization

ok, I believe it's fixed. You can give it a try

the issue persists for me:
grep maximum ../genomes/submission/*.log
../genomes/submission/refgenie_Cajanus_cajan__Penguin_Genomics_v5_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Capsella_rubella_JGI_annotation_v1_0_on_assembly_v1-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chenopodium_quinoa__JGI_v1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chlamydomonas_reinhardtii_JGI_5_5_Phytozome_10_2-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chlamydomonas_reinhardtii__JGI_v5_5-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chlorella_sp_NC64A_JGI_1_0-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Chondrus_crispus_ENSEMBL_protists__release_28-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Citrullus_lanatus_Cucurbit_Genomics_Database-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.
../genomes/submission/refgenie_Coffea_canephora__Coffee_Genome_Hub_v1-fasta.log:RuntimeError: The maximum wait time (60) has been reached and the lock file still exists.

I'm watching the file lock -- it is not locking and unlocking immediately the way it should. whatever is locking it is keeping it locked for seconds, I think sometimes minutes at a time.

for testing, you can use this to watch the lockfile and see when it gets replaced:
watch -n 0.2 'ls -l --time-style=full-iso config/lock.refgenie_config.yaml'

Yes, I can confirm, in my hands, the config file is locked during the entire duration of the asset build process. this is a mistake. I am using the latest refgenie and refgenconf.

could it be because the initialize_genome is not in a context manager, so it's getting locked, but not unlocked after initializing the genome?

See line: DEBU 16:59:26 | yacman.yacman:yacman:227 > Made object writable
then it goes straight into the build
EBU 16:59:26 | refgenconf.refgenconf:refgenconf:103 > Config version is compliant: 0.4
**DEBU 16:59:26 | yacman.yacman:yacman:227 > Made object writable**
DEBU 16:59:26 | refgenconf.refgenconf:refgenconf:2242 > checking existence of: c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f/Non
e:None
DEBU 16:59:26 | attmap.attmap:attmap:105 > Transforming map-like: PathExAttMap: {}
DEBU 16:59:26 | yacman.alias:alias:228 > Added aliases (c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f: ['Nelumbo_nucifera__LOTUS
-DB_v1_1'])
INFO 16:59:26 | refgenconf.refgenconf:refgenconf:1274 > Set genome alias (c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f: Nelumbo
_nucifera__LOTUS-DB_v1_1)
DEBU 16:59:26 | yacman.yacman:yacman:168 > Wrote to a file: /project/shefflab/deploy/plantref/plantref/config/refgenie_config.yaml
DEBU 16:59:26 | yacman.yacman:yacman:168 > Wrote to a file: /project/shefflab/deploy/plantref/plantref/config/refgenie_config.yaml
INFO 16:59:26 | root:refgenie:413 > Saving outputs to:
- content: /project/shefflab/deploy/plantref/genomes/data/c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f
- logs: /project/shefflab/deploy/plantref/genomes/data/c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f/fasta/default/_refgenie_bui
ld
DEBU 16:59:26 | logmuse:est:266 > Configured logger 'logmuse' using logmuse v0.2.6
DEBU 16:59:26 | logmuse:manager:1131 > Logger set with logmuse.logger_via_cli
Removed existing flag: '/project/shefflab/deploy/plantref/genomes/data/c0cc9804e43d0e05b1266f52a29e701958975ec136ef863f/fasta/defau
lt/_refgenie_build/refgenie_failed.flag'
INFO 16:59:26 | logmuse:manager:1134 > ### Pipeline run code and environment:

INFO 16:59:26 | logmuse:manager:1134 > *              Command:  `/home/ns5bc/.local/bin/refgenie --verb 5 build Nelumbo_nucifera__L
OTUS-DB_v1_1/fasta --genome-description Nelumbo nucifera  LOTUS-DB v1.1 --files fasta=/project/shefflab/www/refgenie_plantref/Nelum
bo_nucifera__LOTUS-DB_v1_1-fasta-fasta -R`
INFO 16:59:26 | logmuse:manager:1134 > *         Compute host:  udc-ba34-36
INFO 16:59:26 | logmuse:manager:1134 > *          Working dir:  /sfs/qumulo/qproject/shefflab/deploy/plantref/plantref
INFO 16:59:26 | logmuse:manager:1134 > *            Outfolder:  /project/shefflab/deploy/plantref/genomes/data/c0cc9804e43d0e05b126
6f52a29e701958975ec136ef863f/fasta/default/_refgenie_build/
INFO 16:59:26 | logmuse:manager:1134 > *  Pipeline started at:   (08-27 16:59:26) elapsed: 0.0 _TIME_
INFO 16:59:26 | logmuse:manager:1134 >
### Version log:

INFO 16:59:26 | logmuse:manager:1134 > *       Python version:  3.6.6
INFO 16:59:26 | logmuse:manager:1134 > *          Pypiper dir:  `/sfs/qumulo/qhome/ns5bc/.local/lib/python3.6/site-packages/pypiper
`
INFO 16:59:26 | logmuse:manager:1134 > *      Pypiper version:  0.12.1
INFO 16:59:26 | logmuse:manager:1134 > *         Pipeline dir:  `/sfs/qumulo/qhome/ns5bc/.local/bin`
INFO 16:59:26 | logmuse:manager:1134 > *     Pipeline version:  None
INFO 16:59:26 | logmuse:manager:1134 >
### Arguments passed to pipeline:

INFO 16:59:26 | logmuse:manager:1134 > * `asset_registry_paths`:  `['Nelumbo_nucifera__LOTUS-DB_v1_1/fasta']`
INFO 16:59:26 | logmuse:manager:1134 > *             `assets`:  `None`
INFO 16:59:26 | logmuse:manager:1134 > *            `command`:  `build`
INFO 16:59:26 | logmuse:manager:1134 > *        `config_file`:  `/home/ns5bc/.local/lib/python3.6/site-packages/refgenie/refgenie.y
aml`
INFO 16:59:26 | logmuse:manager:1134 > *             `docker`:  `False`
INFO 16:59:26 | logmuse:manager:1134 > *              `files`:  `[['fasta=/project/shefflab/www/refgenie_plantref/Nelumbo_nucifera_
_LOTUS-DB_v1_1-fasta-fasta']]`
INFO 16:59:26 | logmuse:manager:1134 > *             `genome`:  `None`
INFO 16:59:26 | logmuse:manager:1134 > *      `genome_config`:  `None`
INFO 16:59:26 | logmuse:manager:1134 > * `genome_description`:  `Nelumbo nucifera  LOTUS-DB v1.1`
INFO 16:59:26 | logmuse:manager:1134 > *             `logdev`:  `False`
INFO 16:59:26 | logmuse:manager:1134 > *          `new_start`:  `False`

I was able to reproduce this issue previously in my test fasta build locally. Then I fixed it here: 3c3738e and it resolved the issue in my test.
With your recent commits you've reverted this "fix", so I guess is it still does not work

Can you explain what you mean by "you've reverted this fix" ? I did not intend to revert anything and I see no revert commits.
It wasn't working for me with your fix, and it's still not working for me even with my context manager update. none of these things is solving the issue in my test...

ok, I just checked out your commit and that did work this time. I'm confusing myself.
I have to go now but I'll double-check later. I do not know what I did wrong. Perhaps it depends on if the genome is already initialized or not.

Can you explain what you mean by "you've reverted this fix" ?

I might've used wrong word here. By revert I meant: you brought that part of code back to the state similar that it was before by "fix" -- the context manager is there (my "fix" removed it, because the file locking is handled by a context manager in the set_genome_alias method).

I can't explain what happened, but now with your change it worked. However, watching the lock files, it's clear that something is still locking the file for periods of time. I think it worked one time and didn't work another time may have something to do with random issue on rivanna with file system speed or something.
Anyway, I've reverted my changes (I never seem to be able to figure out how to do that well) -- I just forced update to set back to where you had it, hope it doesn't cause trouble for you on the history rewrite


yum repository for version 2.x can't access from AWS EC2 instance.

I tried to apply this cookbook my EC2 instance to install td-agent 2.1.5-0 by pinning attribute, but error raised like below.
[2015-03-12T17:51:14+09:00] ERROR: yum_repository[treasure-data] (td-agent::default line 76) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[yum-makecache-treasure-data] (/var/chef/cache/cookbooks/yum/providers/repository.rb line 55) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of yum -q makecache --disablerepo=* --enablerepo=treasure-data ----
STDOUT:
STDERR: http://packages.treasuredata.com/2/redhat/latest/x86_64/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Trying other mirror.

I checked these things...

The URL http://packages.treasuredata.com/2/redhat/latest/x86_64/repodata/repomd.xml respond NoSuchKey xml.

The yum repository does not have latest entry. (here)
/etc/yum.conf on my EC2 instance contains releasever=latest line. (code)

Should I change releasever=7 on my EC2 instance or install by other method ?
(I hope that the repository has latest entry on http://packages.treasuredata.com/2/redhat/...)

What distro are you using? Amazon Linux?

What distro are you using? Amazon Linux?

Yes.
The EC2 instance created with Amazon Linux AMI 2014.09.2 (HVM) - ami-18869819.

@repeatedly What if we release RPM packages to latest as well? I think it would help improving td-agent2's support for Amazon Linux.
In my understanding, Amazon Linux is one of popular distro on EC2 and we should improve the support for them.

@yyuu One concern is Amazon Linux doesn't have binary compatibility with RHEL/CentOS because they have own version policy since latest release.
It causes hard to guarantee our RPM work on Amazon Linux.
In addition, we don't have a Amazon Linux image so automatic packaging is too hard.
I tried to create redirection rule for Amazon Linux but S3 + CloudFront combination has a limitation for redirection(latest -> 7).
I'm not sure we should create RPM for Amazon Linux because it needs extra cost.
BTW, setup latest directory is not difficult.

@repeatedly I understood that the difficulty to support Amazon Linux is binary compatibility issue.... To deal with this issue for now, I think there would be 2 ways. I believe the former is better. Which one would you like, or other options?

Support Amazon Linux unofficially with using as same binary as RHEL7/CentOS7

Create copy of binaries in latest directory

Remove support for Amazon Linux from chef-td-agent

After tried https://github.com/mitchellh/vagrant-aws, we should select 1.
I'm in fossasia event so I can't test above right now.

td-agent 2.2.0 provides Amazon Linux RPM.
This issue should be resolved.

Thanks !
This cookbook works with td-agent 2.2.0.


Nuke: wrong "star at" value on render load

Describe the bug
When a render doesn't match the shot length, for example shorter CG renders, it will load into nuke starting from the shot start frame, rather than start frame of the actual render.
We should be placing it on the actually frames coming in from the rendered footage instead.
In this screen shot, the render was placed at 1001, but should be starting on 1013

This behavior is by design. We will need to add loading options which will disable this.


Fix form validations on Orgs form

Fixes #1860  .
Changes proposed in this PR:

Update .gitignore file
Add validations for contact_name field
Remove validations for links fields

I think we may need to restore some JS validation for the links (used here and on the template details page when the org is a funder).
The links aren't required, but when one is provided it must have both a URL and Text. The URL should be validated to ensure that its a valid URL.

Lets refactor that afterward (along with the CSS changes you've made). Orgs don't update their links very often. We'll have to add a cleanup task though to capture any unvalidated links that get through between releases.


Additional NULL check in find_

If I'm searching a point that is not contained by the tree, I'll get a segfault, because of quadtree.c line 92.
If get_quadrant_ returns NULL, then in the next find_ -> quadtree_node_isleaf call, will crash.

The idea behind this is right, outside of the spelling error, can you fix?
Thanks!

On Aug 1, 2017, at 12:41 AM, Isty001<EMAIL_ADDRESS>wrote:
If I'm searching a point that is not contained by the tree, I'll get a segfault, because of quadtree.c line 92.
If get_quadrant_ returns NULL, then in the next find_ -> quadtree_node_isleaf call, will crash.
You can view, comment on, or merge this pull request online at:
https://github.com/thejefflarson/quadtree/pull/4
Commit Summary
Update quadtree.c
File Changes
M src/quadtree.c (3)
Patch Links:
https://github.com/thejefflarson/quadtree/pull/4.patch
https://github.com/thejefflarson/quadtree/pull/4.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Oh, yeah, thanks for pointing out!


Have an issue on IOS device when getting queryPastPurchases()

final QueryPurchaseDetailsResponse purchaseResponse = await _connection.queryPastPurchases();
Error Domain=SKErrorDomain Code=0 "UNKNOWN_ERROR" UserInfo={NSLocalizedDescription=UNKNOWN_ERROR, NSUnderlyingError=0x600002a25b30 {Error Domain=ASDErrorDomain Code=507 "Error decoding object" UserInfo={NSLocalizedDescription=Error decoding object, NSLocalizedFailureReason=Attempted to decode store response}}}

@propertychernetsov
I am assuming this is related to in_app_purchases ?
If so, can you provide flutter doctor -v and a complete minimal reproducible code sample that shows this issue ?

I am having similar issues
/// Get all products available for sale
 Future<void> _getProducts() async {
   try {

     ProductDetailsResponse response = await iap.queryProductDetails(ids);

      if (response.notFoundIDs.isNotEmpty) {
        debugPrint(
            '#PurchaseService.loadProductsForSale() notFoundIDs: ${response
                .notFoundIDs}');
      }
      if (response.error != null) {
        debugPrint(
            '#PurchaseService.loadProductsForSale() error: ${response.error
                .code + ' - ' + response.error.message}');
      }
     products = response.productDetails;

     loader.isLoading = false;
   } catch (e) {
     print(e.toString());
   }
 }

Response:

flutter: #PurchaseService.loadProductsForSale() error: storekit_getproductrequest_platform_error - UNKNOWN_ERROR

flutter doctor -v
flutter doctor -v
[✓] Flutter (Channel dev, 1.27.0-8.0.pre, on macOS 11.1 20C5048l darwin-x64, locale en-NG)
    • Flutter version 1.27.0-8.0.pre at /Users/chiziaruhoma/flutter
    • Framework revision b7d4806243 (5 weeks ago), 2021-02-19 09:22:45 -0800
    • Engine revision 6993cb229b
    • Dart version 2.13.0 (build 2.13.0-30.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/chiziaruhoma/Library/Android/sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google
    Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.54.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.20.0

[✓] Connected device (2 available)
    • iPhone 12 Pro Max (mobile) • D74B99D5-2821-4EAB-B825-9DF28675952D • ios        •
      com.apple.CoreSimulator.SimRuntime.iOS-14-4 (simulator)
    • macOS (desktop)            • macos                                • darwin-x64 • macOS 11.1 20C5048l
      darwin-x64
    ! Error: Chizi’s iPhone is not connected. Xcode will continue when Chizi’s iPhone is connected. (code -13)

! Doctor found issues in 1 category.

libri_app on  master [⇡!+] took 12s

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.


⚠ Forum - Funcraft has degraded performance

In 8695b18, Forum - Funcraft (https://community.funcraft.net/) experienced degraded performance:

HTTP code: 200
Response time: 596 ms

Resolved: Forum - Funcraft performance has improved in 73209dc after 12 minutes.


Get eslint-plugin-jsonc version via command line

I need a way to get eslint-plugin-jsonc version from command line to generate correct information is eslint-plugin-jsonc page on Mega-Linter. Currently i'm just able to retrieve eslint version.
Is it existing ? :)
Thanks !

Thank you for question.
This plugin is not used directly from the CLI, so there are no commands.
If you have a mechanism to get the version of @typescript-eslint/eslint-plugin from the CLI, I think you can get the version of this plugin in the same way.

Thank you for question.
This plugin is not used directly from the CLI, so there are no commands.
If you have a mechanism to get the version of @typescript-eslint/eslint-plugin from the CLI, I think you can get the version of this plugin in the same way.

Solved by https://github.com/eslint/eslint/issues/14031 :)
npm list eslint-plugin-jsonc :)

Solved by https://github.com/eslint/eslint/issues/14031 :)
npm list eslint-plugin-jsonc :)


fix : Gssoc project repolink

When i was searching through projects and found this project the redirecting repo throws you on the site not the github repo
issue under GSSOC -23

Thanks..
Ticket has been created for same..


Finish README

And mention buddies SA and the whole world

Please correct me where im wrong .im new into this


Fixing cmake configuration for multi-config generators

The assumption made throughout ospray cmake files is that CMAKE_BUILD_TYPE will always contain the build config type. The problem is that in multi-config generators (such as vs and xcode), the build type is only selected during build time, and setting CMAKE_BUILD_TYPE should have no impact.
This gets problematic if the user selects Debug during build time through --config, while CMAKE_BUILD_TYPE is not set, so it takes the default value of Release, causing a mismatch between the actual build type and the one set through CMAKE_BUILD_TYPE. Another scenario is when a user switches from Debug to Release builds, then they need to switch the CMAKE_BUILD_TYPE as well to keep it consistent with the actual config type, which is not the usual process in multi-config.
The solution proposed here relies on the cmake generator-expression ($) to derive the config type for both single and multi configuration types. It also enables debugging for windows since it was disabled before, for no good reason that I am aware off. The final impact is that we remove the default setting for CMAKE_BUILD_TYPE since it should use whatever is default for the generator.
The only downside comes to single-config builds, which is that we had to remove some variables from the UI (the cmake_build_type through cycling CONFIGURATION_TYPES), and also remove OSPRAY_BUILD_RELEASE/DEBUG/RELWITHDEBINFO from the config files, since we cannot use configure_file to support these as they rely now on generator expressions which are evaluated post config.
This change has been tested with both single (make) and multi-config (vs) and worked fine in both cases.
If not taking this pull request, I suggest at least removing the win32 condition from ispc debug, and clarifying in the documentation that multi-config generators (such as vs) need to set the CMAKE_BUILD_TYPE to be consistent with the chosen config type, otherwise some inconsistencies will happen (including the case where CMAKE_BUILD_TYPE  is not set at all).

Just wanted to say thanks for this! We are busy with SIGGRAPH preparations right now, but this will get incorporated ASAP after that...we haven't forgotten about it. :)

Sure thing, Jeff! Looking forward to the magic you guys will show us at Siggraph this year.


Files are not getting stored or maybe I can't locate them...

I've a problem that I can't see any files, photos, etc sent from my mobile phone.
To figure out where the files should be located I used this code in hook.php
echo $telegram->getUploadPath();
Output from Browser:
/var/www/website.tld/mybot/php-telegram-bot/src/../Upload
So the location for uploaded files should be there /var/www/website.tld/mybot/php-telegram-bot/src/Upload right?
All the folder permissions should be correct.  Per example audio looks like this:
{"update_id":86454387, "message":{"message_id":601,"from":{"id":123456789,"first_name":"myFirstname","last_name":"myLastName"},"chat":{"id":123456789,"first_name":"myFirstname","last_name":"myLastName","type":"private"},"date":1465411058,"voice":{"duration":1,"mime_type":"audio\/ogg","file_id":"AwADBAADBQAD3QozDt94YZV9NlHoAg","file_size":2750}}}
When I try to debug with the setCustomInput function and call the hook.php via browser I don't get an error.
No related log entry in debug.log or error.log is visible.
Has someone an idea what could cause this issue?
Thanks for help!
PHP Version:
PHP 5.6.20-0+deb8u1 (cli) (built: Apr 27 2016 11:26:05)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
php-telegram-bot version: 0.33

The uploaded files aren't saved anywhere automatically at the moment...
What you could do in the meantime, is create a custom command, say /upload, and then save the uploaded file then.
I'll put together the command for you to use, stay tuned!
If you feel like exploring a bit in the meantime, take a look at the SurveyCommand.php file and look here how to save an uploaded file:
https://github.com/akalongman/php-telegram-bot/issues/123#issuecomment-224523310

@MBoretto @jacklul @akalongman
Should every file that gets uploaded to the bot automatically be saved to the Upload folder?
Would make sense, right?
At the moment the Upload folder isn't even used!

Not by default.
I noticed on my bot there is a lot of people who just try to send files to the bot, most likely viruses and stuff.

Mhm it shouldn't be activated per default. My bot I do create is for a game (mrX). Only user with a specific /join  can join respectively will be able to use all commands. For some challenges in the game they need to upload pictures, which an admin later analyse manually and if challenge passed they get credits.

@jacklul You're absolutely right, by default isn't a good idea. We could add a setting for those people who would want that, or at least a helper to make file downloads less painful.
@xtrahost Voilà, save as UploadCommand.php in your custom commands folder. Being a UserCommand, anyone can use it! So you might want to change it to an AdminCommand.
<?php
/**
 * This file is part of the TelegramBot package.
 *
 * (c) Avtandil Kikabidze aka LONGMAN<EMAIL_ADDRESS> *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

namespace Longman\TelegramBot\Commands\UserCommands;

use Longman\TelegramBot\Request;
use Longman\TelegramBot\Conversation;
use Longman\TelegramBot\Commands\UserCommand;
use Longman\TelegramBot\Entities\ForceReply;
use Longman\TelegramBot\Entities\ReplyKeyboardHide;
use Longman\TelegramBot\TelegramLog;

/**
 * User "/upload" command
 */
class UploadCommand extends UserCommand
{
    /**#@+
     * {@inheritdoc}
     */
    protected $name = 'upload';
    protected $description = 'Upload and save files';
    protected $usage = '/upload';
    protected $version = '0.0.1';
    protected $need_mysql = true;
    /**#@-*/

    /**
     * Conversation Object
     *
     * @var Longman\TelegramBot\Conversation
     */
    protected $conversation;

    /**
     * {@inheritdoc}
     */
    public function execute()
    {
        $message = $this->getMessage();

        $chat = $message->getChat();
        $user = $message->getFrom();

        $chat_id = $chat->getId();
        $user_id = $user->getId();

        //Preparing Response
        $data = [
            'chat_id'      => $chat_id,
            'reply_markup' => new ReplyKeyBoardHide(['selective' => true]),
        ];

        if ($chat->isGroupChat() || $chat->isSuperGroup()) {
            //reply to message id is applied by default
            $data['reply_to_message_id'] = $message->getMessageId();
            //Force reply is applied by default to so can work with privacy on
            $data['reply_markup'] = new ForceReply([ 'selective' => true]);
        }

        //Conversation start
        $this->conversation = new Conversation($user_id, $chat_id, $this->getName());

        if (in_array($message->getType(), ['Audio', 'Document', 'Photo', 'Video', 'Voice'])) {
            $doc = call_user_method('get' . $message->getType(), $message);
            ($message->getType() === 'Photo') && $doc = $doc[0];

            $file_id = $doc->getFileId();
            $file = Request::getFile(['file_id' => $file_id]);
            if ($file->isOk() && Request::downloadFile($file->getResult())) {
                $data['text'] = $message->getType() . ' file is located at: ' . $this->telegram->getDownloadPath() . '/' . $file->getResult()->getFilePath();
            } else {
                $data['text'] = 'Failed to download.';
            }

            $this->conversation->notes['file_id'] = $file_id;
            $this->conversation->update();
            $this->conversation->stop();
        } else {
            $data['text'] = 'Please upload the file now';
        }

        return Request::sendMessage($data);
    }
}

Thanks a lot @noplanman works fine. Except that it will store only the smallest photo version.

Right, my bad, sorry.
The line in question is this one:
($message->getType() === 'Photo') && $doc = $doc[0];

Replace it with this one for the highest resolution:
($message->getType() === 'Photo') && $doc = end($doc);

Just amazing! Thank you, in my view we can close the issue.

Ok 👍
We'll be working on a better implementation of this, but until then you can use the /upload command as a workaround.

hi guys!
thank you noplanman for answering this issue!
but i got some error on your code. I've tried to fix it but I was not successful!
these are what I got in error log:
[21-Dec-2016 20:46:52 UTC] PHP Deprecated:  Function call_user_method() is deprecated in /home/mybotpro/public_html/Project/vendor/longman/telegram-bot/src/Commands/UserCommands/SendImgCommand.php on line 82
[21-Dec-2016 20:46:52 UTC] PHP Fatal error:  Call to a member function getFileId() on array in /home/mybotpro/public_html/Project/vendor/longman/telegram-bot/src/Commands/UserCommands/SendImgCommand.php on line 85
what can I do to fix this?

Could you post the code of your SendImgCommand.php please?

 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

namespace Longman\TelegramBot\Commands\UserCommands;

use Longman\TelegramBot\Request;
use Longman\TelegramBot\Conversation;
use Longman\TelegramBot\Commands\UserCommand;
use Longman\TelegramBot\Entities\ForceReply;
use Longman\TelegramBot\Entities\ReplyKeyboardHide;
use Longman\TelegramBot\TelegramLog;
/**
 * User "/cancel" command
 *
 * This command cancels the currently active conversation and
 * returns a message to let the user know which conversation it was.
 * If no conversation is active, the returned message says so.
 */
class SendImgCommand extends UserCommand
{
    /**
     * @var string
     */
    protected $name = 'sendimg';

    /**
     * @var string
     */
    protected $description = 'ارسال عکس';

    /**
     * @var string
     */
    protected $usage = '/sendimg';

    /**
     * @var string
     */
    protected $version = '0.2.0';

    protected $conversation;

    /**
     * {@inheritdoc}
     */
    public function execute()
    {
		$servername = "mybotprovider.ir";
		$username = "mybotpro_admin";
		$password = "1a2a3a4197";
		$dbname = "mybotpro_telegram";

		$mysql_credentials = [
		'host'     => $servername,
		'user'     => $username,
		'password' => $password,
		'database' => $dbname,
		];

	$this->telegram->enableMySQL($mysql_credentials);

        $message = $this->getMessage();
        $chat_id  = $message->getChat()->getId();
		$from       = $message->getFrom();
        $user_name    = $from->getUsername();
		$user_id    = $from->getId();

        //Preparing Response
        $data = [
            'chat_id'      => $chat_id,
        ];

            $doc = call_user_method('get' . $message->getType(), $message);
            ($message->getType() === 'Photo') && $doc = $doc[0];

            $file_id = $doc->getFileId();
            $file = Request::getFile(['file_id' => $file_id]);
            if ($file->isOk() && Request::downloadFile($file->getResult())) {
                $data['text'] = $message->getType() . ' file is located at: ' . $this->telegram->getDownloadPath() . '/' . $file->getResult()->getFilePath();
            } else {
                $data['text'] = 'Failed to download.';
            }

        return Request::sendMessage($data);
    }
}

`<?php
/**

This file is part of the TelegramBot package.

(c) Avtandil Kikabidze aka LONGMAN<EMAIL_ADDRESS>
For the full copyright and license information, please view the LICENSE
file that was distributed with this source code.
*/

namespace Longman\TelegramBot\Commands\UserCommands;
use Longman\TelegramBot\Request;
use Longman\TelegramBot\Conversation;
use Longman\TelegramBot\Commands\UserCommand;
use Longman\TelegramBot\Entities\ForceReply;
use Longman\TelegramBot\Entities\ReplyKeyboardHide;
use Longman\TelegramBot\TelegramLog;
/**

User "/cancel" command

This command cancels the currently active conversation and

returns a message to let the user know which conversation it was.

If no conversation is active, the returned message says so.
/
class SendImgCommand extends UserCommand
{
/*

@var string
*/
protected $name = 'sendimg';

/**

@var string
*/
protected $description = 'ارسال عکس';

/**

@var string
*/
protected $usage = '/sendimg';

/**

@var string
*/
protected $version = '0.2.0';

protected $conversation;
/**

{@inheritdoc}
*/
public function execute()
{
$servername = "mybotprovider.ir";
$username = "mybotpro_admin";
$password = "1a2a3a4197";
$dbname = "mybotpro_telegram";
$mysql_credentials = [
'host'     => $servername,
'user'     => $username,
'password' => $password,
'database' => $dbname,
];

$this->telegram->enableMySQL($mysql_credentials);
 $message = $this->getMessage();
 $chat_id  = $message->getChat()->getId();
 $from       = $message->getFrom();
 $user_name    = $from->getUsername();
 $user_id    = $from->getId();

 //Preparing Response
 $data = [
     'chat_id'      => $chat_id,
 ];

     $doc = call_user_method('get' . $message->getType(), $message);
     ($message->getType() === 'Photo') && $doc = $doc[0];

     $file_id = $doc->getFileId();
     $file = Request::getFile(['file_id' => $file_id]);
     if ($file->isOk() && Request::downloadFile($file->getResult())) {
         $data['text'] = $message->getType() . ' file is located at: ' . $this->telegram->getDownloadPath() . '/' . $file->getResult()->getFilePath();
     } else {
         $data['text'] = 'Failed to download.';
     }

 return Request::sendMessage($data);

}
}`

also i got this error when i was trying to connect to mysql for creating a conversation.
[21-Dec-2016 20:36:52 UTC] PHP Warning:  date(): It is not safe to rely on the system's timezone settings. You are required to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /home/mybotpro/public_html/Project/vendor/longman/telegram-bot/src/DB.php on line 245
[21-Dec-2016 20:36:52 UTC] PHP Warning:  PDOStatement::execute(): SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (mybotpro_telegram.conversation, CONSTRAINT conversation_ibfk_1 FOREIGN KEY (user_id) REFERENCES user (id)) in /home/mybotpro/public_html/Project/vendor/longman/telegram-bot/src/ConversationDB.php on line 107

I got the second error in this code:
`<?php
/**

This file is part of the TelegramBot package.

(c) Avtandil Kikabidze aka LONGMAN<EMAIL_ADDRESS>
For the full copyright and license information, please view the LICENSE
file that was distributed with this source code.
*/

namespace Longman\TelegramBot\Commands\UserCommands;
use Longman\TelegramBot\Commands\UserCommand;
use Longman\TelegramBot\Conversation;
use Longman\TelegramBot\Entities\Keyboard;
use Longman\TelegramBot\Request;
use Longman\TelegramBot\Telegram;
/**

User "/cancel" command

This command cancels the currently active conversation and

returns a message to let the user know which conversation it was.

If no conversation is active, the returned message says so.
/
class SendImgCommand extends UserCommand
{
/*

@var string
*/
protected $name = 'sendimg';

/**

@var string
*/
protected $description = 'ارسال عکس';

/**

@var string
*/
protected $usage = '/sendimg';

/**

@var string
*/
protected $version = '0.2.0';

protected $conversation;
/**

{@inheritdoc}
*/
public function execute()
{
$servername = "mybotprovider.ir";
$username = "mybotpro_admin";
$password = "1a2a3a4197";
$dbname = "mybotpro_telegram";
$mysql_credentials = [
'host'     => $servername,
'user'     => $username,
'password' => $password,
'database' => $dbname,
];
$telegram->enableMySQL($mysql_credentials);
$message = $this->getMessage();
$chat = $message->getChat();
$user = $message->getFrom();
$chat_id = $chat->getId();
$user_id = $user->getId();
//Preparing Response
$data = [
'chat_id'      => $chat_id,
];
//Conversation start
$this->conversation = new Conversation($user_id, $chat_id, $this->getName());
if (in_array($message->getType(), ['Audio', 'Document', 'Photo', 'Video', 'Voice'])) {
$doc = call_user_method('get' . $message->getType(), $message);
($message->getType() === 'Photo') && $doc = $doc[0];
 $file_id = $doc->getFileId();
 $file = Request::getFile(['file_id' => $file_id]);
 if ($file->isOk() && Request::downloadFile($file->getResult())) {
     $data['text'] = $message->getType() . ' file is located at: ' . $this->telegram->getDownloadPath() . '/' . $file->getResult()->getFilePath();
 } else {
     $data['text'] = 'Failed to download.';
 }

 $this->conversation->notes['file_id'] = $file_id;
 $this->conversation->update();
 $this->conversation->stop();

} else {
$data['text'] = 'Please upload the file now';
}
return Request::sendMessage($data);
}
}`

hey noplanman!
thanks a lot for your response.
I'm waiting for your answer. as you know, there aren't many sources about creating telegram bot on php.
so I'm really looking forward to hearing an answer from you.
my main problems are:

I'm unable to connect to mysql from longman source.
I don't know how could i upload a photo from bot to host. that's because of that I can't get FileId.

I'm waiting ... so please don't let me down.

First of all, the MySQL connection info shouldn't be here, it should be in your hook.php.
Also, the type is all lowercase since version 0.36 (I think).
Some lines need to be changed, namely:
...
if (in_array($message->getType(), ['audio', 'document', 'photo', 'video', 'voice'], true)) {
    $doc = call_user_func([$message, 'get' . ucfirst($message->getType())]);
    ($message->getType() === 'photo') && $doc = $doc[0];
...

Does that help enough?

thank you so mush noplanman! you actually have saved my life !!!
now i can get filepath without any problem.
but about mysql connection i also have some problems.
I set mysql connection method inside hook.php and then unset webhook and reset it. but when i put this line:
protected $need_mysql = true;
inside SendImgCommand.php and run it, I get this message in my bot:
Sorry no database connection, unable to execute "sendimg" command.
this is my hook.php:
`<?php
// Load composer
require DIR . '/vendor/autoload.php';
$API_KEY = '287930194:AAHnR1JB5H2ygwHTspppSvyqnVTrnOrUe7g';
$BOT_NAME = 'Artoan';
try {
// Create Telegram API object
$telegram = new Longman\TelegramBot\Telegram($API_KEY, $BOT_NAME);
// Handle telegram webhook request
$telegram->handle();

$mysql_credentials = [

'host'     => 'mybotprovider.ir',
'user'     => 'mybotpro_admin',
'password' => '1a2a3a4197',
'database' => 'mybotpro_telegram',
];
$telegram->enableMySQL($mysql_credentials);

} catch (Longman\TelegramBot\Exception\TelegramException $e) {
// Silence is golden!
// log telegram errors
// echo $e;
}
?>`
So what do you think about it? what is the problem?

You need to enable MySQL before handling the request! 😉
(on a side note, when you post code, use the triple backticks with PHP syntax highlighting. Edit your comment above to see what I mean and read more about it here)

thank you so much noplanman!
you did the best for me, but I can't do anything for you except being Grateful and have best wishes for you!
thanks you a lot!

You're so welcome 😊
Happy to hear it's working now.

I know it is necroposting..but I've tried this command on the new version of Php Telegram bot and it does not download my sent file. Seems that $file->getResult() is null.
Is there another way to download the photo sent to bot?
Thanks


fix: SpriteAnimation errors listed in Dart Analysis and app won't run

Description
When using the VGV CLI to create a Flame game there are 13 errors listed in Dart Analysis and the template app won't run.
Steps To Reproduce

From terminal run: very_good create flutter_app very_good_flutter_app --desc "A Flutter app built using Very Good Ventures Very Good CLI"
Open project in IntelliJ
Look at Dart Analysis tab
See 13 errors - most of which are pertaining to Sprite Animation.
There are also 6 hints, including that HasTappables and Camera have been deprecated (I know Camera deprecation is in the last week).

Expected Behavior
No Dart Analysis errors to be listed.
Screenshots

Additional Context
Macbook Pro M1
macOS 13.3.1
IntelliJ IDEA 2022.3.3 (Community Edition)
Flutter 3.10.0 (tried rolling back. Still gave errors)
Dart Dart 3.0.2 (tried rolling back. Still gave errors)
pubspec.yaml was not updated so entries remained as shown below.  Even updating them didn't solve issue.  I also tried rolling back. Still gave errors)

class Unicorn extends PositionedEntity with HasGameRef {
Unicorn({
required super.position,
}) : super(
anchor: Anchor.center,
size: Vector2.all(32),
behaviors: [
TappingBehavior(),
],
);
@visibleForTesting
Unicorn.test({
required super.position,
super.behaviors,
}) : super(size: Vector2.all(32));
SpriteAnimation? _animation;
late SpriteAnimationTicker _ticker;
@visibleForTesting
SpriteAnimation get animation => _animation!;
@override
Future onLoad() async {
_animation = await gameRef.loadSpriteAnimation(
  Assets.images.unicornAnimation.path,
  SpriteAnimationData.sequenced(
    amount: 16,
    stepTime: 0.1,
    textureSize: Vector2.all(32),
    loop: false,
  ),
);

_ticker = animation.createTicker();

resetAnimation();
_ticker.onComplete = resetAnimation;

await add(SpriteAnimationComponent(animation: _animation, size: size));

}
/// Set the animation to the first frame by tricking the animation
/// into thinking it finished the last frame.
void resetAnimation() {
_ticker
..currentIndex = _animation!.frames.length - 1
..update(0.1)
..currentIndex = 0;
}
/// Plays the animation.
void playAnimation() => _ticker.reset();
/// Returns whether the animation is playing or not.
bool isAnimationPlaying() => _ticker.isFirstFrame;
}

Tracking on https://github.com/VeryGoodOpenSource/very_good_cli/issues/770


Show urls for modal domains and custom domains for web endpoint methods on new-style classes without method placeholders

Show urls for modal domains and custom domains for web endpoint methods on new-style classes without method placeholders

@prbot approve


Futures - Publish Contract Atomic Tx

A variant on the Managed Smart Property creation, this transaction creates two new smart properties that are to be part of a series, and carry a special categorization as being Futures Contracts
Parameters:
Blocks between Expiration - from the block the new contract series if published, the first contract will expire N blocks later, and the second contract will expire 2N blocks later. At Block N, a new contract comes into existence which expires 3N blocks after this transaction was published, and so on ad infinitum.
Note the smart property IDs for a contract series needs to follow a numeration where 1000s of contracts can come in and out of existence over time without collisions. So futures contract series should occupy a large header number, such as 410000, so that if a contract expires every day, after ten years all the contracts in that series still occupy 41xxxx numeration.
There should be a minimum time between contract expirations, perhaps 50 blocks (~6hr contract) or perhaps 150 (~24hrs).
Collateral Currency - a smart property ID # for the smart property used to both back and settle the contracts. Eg. #1 for OMNI backed contracts that pay some amount of OMNI to winners.
Collateral amount - the # of units of a reserve currency locked up per absolute value number of contracts held long or short. Eg. 10 OMNI per contract means if you have -10 contracts or 10, then your address must have had 100 OMNI to enter the positions. The amount is reserved until settlement for any amounts that are net profits or loss. The amounts can be freed up by reducing the absolute value of one's contract positions, less any losses, which remain reserved until settlement.
Quote Currency - a smart property ID # used to calculate settlement value. Eg. Tether; if the collateral currency were OMNI and the quote currency were Tether on a OMNI/USDT contract, then the OMNI value of a contract would become larger as the price declined, with the USDT value fixed, providing a perfect hedge. Or conversely, imagine an OMNI/BTC contract that is collateralized and quoted in OMNI, it would be a quanto contract that provides convex gains and losses to those who are long/short is respectively, because the losses for the short-side would be fixed in OMNI as the value of OMNI rises. Hence 10% rise means 11% loss.
Notional Value - the value of each contract in the quote currency. Eg. 1 contract is worth 10 OMNI or 1 contract is worth $100 worth of OMNI.
Settlement Reference - either an array containing two smart property IDs or a bitcoin address where a data feed is regularly published by some party. The prior option has settlement logic that looks for the Merkle branches of the two properties, tallies all the spot Dex trades between them in the preceeding N blocks, and uses the ratios to devise a price. The latter option has nodes looking at the Op_return data of transactions broadcast from the data feed address, with some contingency logic for a non-timely publication of a settlement price, that looks at preceeding prices and averages them, and includes a grace period of 2-5 blocks before doing so in case the publisher is a bit late on the update. Perhaps that grace period and the number of preceeding data feed points to average could be parameters in this transaction, perhaps we simply go with 6 and 6 as hardcoded. The benefit of that latter approach is, traders can see if a feed is not being kept up consistently or has been overtly manipulated and know what they are getting into (we can have such statistics included in UI designs for Omniwallet/OmniJ), and it balances with the publisher's profit motive and professional reputation to provide an imperfect but recoverable dynamic.

Hey Patrick,
I was chatting to @CraigSellars about these features and I definitely agree the first step is to get a workable spec.  Paging @marv-engine whom I believe is going to help you with this.
From what I can gather so far there is quite some complexity to what's being requested, so it's going to be a fairly lengthy development effort I think as all of it requires extra ways of storing and processing data etc in the reference client (it's not a simple A+B consensus rule change) - though we can estimate time required better once we have a workable spec (again @marv-engine is your man here and the sooner we get to a workable spec the sooner we can start prototyping :) ).
Thanks for all you're doing & for the ideas
Z

@patrickdugan I need some help understanding this. Assume that I know virtually nothing about what business transaction you're trying to specify. Can you describe the transaction(s) in layman's terms and/or point me toward an existing description?
When writing a spec, I prefer to start with objective and quantifiable functional requirements. If nothing else, that provides a framework for testing. Design and then implementation come only after all the stakeholders understand and agree on the reqs.

Thanks @marv-engine - really appreciate your help on this :)

You bet on changes in price using a fraction of the value of the bet. At a fixed time the bet will pay based on a reference price. So I put up 10 OMNI to take a bet on something that is worth 50 OMNI, and you have the same amount, and when the price changes and I trade off my contract to someone else or wait for expiration, if the price went up and I'm long, I pay you, and if the price went down and you were short, you pay me.

@patrickdugan, @zathras-crypto, @dexX7  - just a thought about one detail (which may be too early in the process):
If I understand how a contract series will work, we can consider using (or extending) the Property Type and Previous Property ID fields to indicate the sequence of SP id's - effectively, generating a linked list. That would eliminate the need(?) to reserve a block of SP id's.
It's TBD if append or replace or something new is the correct Property Type description of the relationship between the new and old contracts' SP id's.

some very basics for the newbies like myself following this:
https://www.youtube.com/watch?v=nwR5b6E0Xo4

Thanks @stebansaa!

I think it's better to keep track of the specification or feature proposals in the spec repository:
https://github.com/OmniLayer/spec
Please feel free to open a new issue over there.


app crash on inheritance

Single wallet created , No vault on pleb subscription ... Check Version history below
https://github.com/bithyve/bitcoin-keeper/assets/107177634/3b32ebbe-9c22-407f-8f8d-327c0aeab744
Version history

Home Screen

Not able to reproduce resolved


Use creator in extra field when there's no creator in the usual places?

Discussed in https://github.com/retorquere/zotero-better-bibtex/discussions/1871

Originally posted by Jmuccigr July 13, 2021
I've just discovered that Zotero has no "editor" option for the creator of a web page, but the usual "cheat" of putting one into the extra field works fine. However BBT seems to want nothing to do with using that field to create a cite key. I put [extra=editor:lower][year][0] | [auth:(unknown)][year][0] in the format, and it ignores the editor field. If I use another name for it, like "blah" or tex.editor, it works fine. Of course that also means that Zotero no longer recognizes it as a creator and so it won't end up in the exported citations.
There's also the issue of the format Zotero needs for names there, that is "Last || First", which I suppose I could handle with regex so BBT doesn't process the whole thing.
Am I missing the proper way to do this?

@Jmuccigr I hadn't thought to use these names for key generation. If you could right-click such an item and send a support log from the popup menu, I can make it work.

🤖 this is your friendly neighborhood build bot announcing test build <IP_ADDRESS>3 ("fixes #1873")
Install in Zotero by downloading test build <IP_ADDRESS>3, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

Yep, this seems to work for me with my format string...as long as I don't have a date. Once I insert one, it does the "unknown" thing.
But if I change the format to remove the date with the editor: [extra=editor:lower][0] | [auth:(unknown)][year][0], then it will work with the date and also insert it after the editor name, which doesn't seem consistent with the format (though maybe I'm not understanding how the format works). Also, if I change the format string back to include the date part, the key now includes the date (but still not the editor).
I submitted two logs, changing the format string in between:
Log without the date in the format: CACCRLN6-refs-euc
Log with the date in the format: 2UFBHLIC-refs-euc
PS The log message refers to a "Next" button, but it's "Continue" on my Mac.

OK, figured something out. The | seems only to affect the first element in the format string, so if I use [extra=editor:lower] | [auth:(unknown)][year][0] all is well. That doesn't seem to jibe with the documentation.

With build 1203 you can use [edtr:lower][year][0] | [auth:(unknown)][year][0] rather than [extra=editor:lower][year][0] | [auth:(unknown)][year][0]. editor isn't one of the recognized fields by the extra function -- it only knows about the ones listed here, but that wasn't on the site yet I see. I've scheduled an update of the site to add that link. That list has "simple fields" -- basically, those fields that are a single line of text in Zotero. editor is a creator-type field and those can appear multiple times, they're treated differently, but they weren't previously picked up by auth and edtr and so on. 1203 fixes that.

OK, figured something out. The | seems only to affect the first element in the format string, so if I use [extra=editor:lower] | [auth:(unknown)][year][0] all is well. That doesn't seem to jibe with the documentation.

The | means "if the preceding pattern did not generate any output, proceed to the next pattern". Since the date generates output, the pattern is accepted. You could use something like [edtr:lower][>0][year][0] | [auth:(unknown)][year][0] where the [>0] means "if there is no output at this point, skip to the next pattern".

OK, figured something out. The | seems only to affect the first element in the format string, so if I use [extra=editor:lower] | [auth:(unknown)][year][0] all is well. That doesn't seem to jibe with the documentation.

The | means "if the preceding pattern did not generate any output, proceed to the next pattern". Since the date generates output, the pattern is accepted. You could use something like [edtr:lower][>0][year][0] | [auth:(unknown)][year][0] where the [>0] means "if there is no output at this point, skip to the next pattern".

No, what I mean is that if I have the format as [extra=editor:lower] | [auth:(unknown)][year][0], I get the year in the citekey when there's an editor in the extras (and no author) even though there's no year in the first part of the format. But in fact [extra=editor] doesn't work at all, but now [auth:(unknown)][year][0] by itself now works just fine and will grab the author if there is one, then the extra editor, if there is one.
One more thing, there was a brief a difference between Editor and editor (initial cap), where the former resulted in a pinned cite key, the latter did not, but that seems to have stopped.

It's a little confusing because both CACCRLN6-refs-euc and 2UFBHLIC-refs-euc have pinned citekeys, which will always take precedence. The pinned key in 2UFBHLIC-refs-euc has only a year, and as long as that line is present, it doesn't matter what your pattern is set to. If there is a Citation Key: line in the extra field, BBT will always take that verbatim, and will not use the generator.
Can you remove the Citation Key: line from extra, send a new log, and describe what you see happening that you don't want to there?

There should not be a difference between editor: and Editor:, and that hasn't changed in a very long time. If you see a difference, please open a new issue for that.

OK, so with no authors, a date of 1999, and only editor: Darwin || Charles in the extra field:
[extra=editor:lower][year][0] | [auth:(unknown)][year][0] yields 1999.
[extra=editor] does not yield any output, because extra= only does single-line fields, and editor: ... is a (potentially) multi-line field, as there can be more than one editor:  line. But [year] does yield output, so BBT never gets to the 2nd pattern.
[extra=editor:lower][0] | [auth:(unknown)][year][0] yields Darwin1999.
Because there is no output from the first pattern as the year is no longer present, BBT moves on to the next. There it finds [auth], that will take authors if it can find any, and if not, will take  the first of editors, translators, collaborators that are present. With 1203, editors: is recognized and used as "native" creators are, so it will find Darwin through the fallback, uses that, and adds the year.
Documentation was at least incomplete for this behavior, and when I merge 1203 that will also update the docs, but the behavior of | is consistent.

I do, and when I test I use the “unpin citation” menu item to get it to redo the cite key. Seems to work. Let me know what I can test.

I do, and when I test I use the “unpin citation” menu item to get it to redo the cite key. Seems to work. Let me know what I can test.

That's fine. Does your experience match either of the scenarios above (because those are in-spec and aren't likely to going to change)? If not, I'd need you to set up the scenario you have in mind, try to regenerate the key, and then send a new log for that item.

OK, a few things:
Most importantly, my current format string is [auth:(unknown)][year][0] which seems to behave the way I want, i.e., it grabs an editor in the extra field.

No, what I mean is that if I have the format as [extra=editor:lower] | [auth:(unknown)][year][0], I get the year in the citekey when there's an editor in the extras (and no author) even though there's no year in the first part of the format.

I cannot reproduce this behavior with 1203. Can you set this up and send a new log for it?

To be clear on this, the first part, [extra=editor:lower], just doesn't seem to work at all. The log for that all by itself as the format is VDFKK8GZ-refs-euc.
Now, I'll add the second part to get this format, [extra=editor:lower] | [auth:(unknown)][year][0], which works fine: 99PKAZYH-refs-euc
Finally with just the second part, [auth:(unknown)][year][0]: UWIXSAFW-refs-euc. So it seems pretty clear that the extra field isn't working for me in the format.
In each case I'm unpinning the cite key, which seems to delete it and then BBT re-creates it. I'm not quite sure I understand what you've written above about extra only doing single-line fields. Perhaps it's that pinning the citekey results in multi-line extra fields?

Most importantly, my current format string is [auth:(unknown)][year][0] which seems to behave the way I want, i.e., it grabs an editor in the extra field.

That's what 1203 and later have added. [edtr] and [auth] will now find creator-type lines in the extra field.
Long story short: [edtr:lower][0] | [auth:(unknown)][year][0] should get you what you want.
The rest is just my attempt to explain BBTs behavior, which wrt the extra field is fairly complicated. I'm in the process of creating a visual guide through this labyrinth, which I'll put up on the site when done, but it's slow going.

To be clear on this, the first part, [extra=editor:lower], just doesn't seem to work at all. The log for that all by itself as the format is VDFKK8GZ-refs-euc.

I think my attempts to explain how extra-field handling works have muddied things. [extra=editor] will never generate output, and that's by design. During key generation, BBT makes a copy of the extra field and does the following (all on a copy of the item, not the actual item, so these don't actually change the item you see in Zotero):

lines that can be recognized as Zotero creator-type fields are added to the creators list, and are removed from extra
lines that can be recognized as a Zotero non-creator-type field that is not multi-line (so extra and abstract are excluded) are taken as fallbacks for missing fields, and removed from extra

"fallback" here just means that if you have a DOI field filled out in the item, and a DOI: bla bla in the extra field, that is stripped and ignored, because Zotero GUI fields take precedence, but for items where you haven't filled the DOI field or the item type doesn't have a DOI field, this value is used as the item's DOI.
So an extra field that looks like
editor: Darwin || Charles
publisher: me
doge: much wow

will leave the key generator with this setup wrt the data in extra
creators: [
  { firstName: 'Charles', lastName: 'Darwin', type: 'editor' }
]
fallback:
  publisher: me
remainder:
  doge: much wow

the extra= function only gets values from fallback and remainder, so [extra=editor] will never generate output, as it is claimed as a creator-type. This was already the case before 1203 as it is just the work that the extra-parser does before key generation. 1203 and later likewise strip the extra field, but makes the stripped data available to key generation.
What you can do instead (post-1203) is [edtr:lower][0] | [auth:(unknown)][year][0]

In each case I'm unpinning the cite key, which seems to delete it and then BBT re-creates it. I'm not quite sure I understand what you've written above about extra only doing single-line fields. Perhaps it's that pinning the citekey results in multi-line extra fields?

No, I was trying to explain rule 1/2 above -- I shouldn't have mentioned the key pinning, I mentioned that because I said before that "your keys are pinned, that takes precedence" and then realized you had not manually pinned them to stay that way. Anyhow, editor is treated as creator-type; even if it weren't, it couldn't play the same role as publisher and doge above, as this is a valid scenario:
editor: Darwin || Charles
publisher: me
editor: Wallace || Alfred Russel
doge: much wow

so it's not clear what the return value should be for [extra=editor] as there are multiple lines that would match. The auth and edtr functions can deal with multiple creator-type values, and they will now also find the creator-type fields in the extra, in addition to the regular creator-type fields Zotero offers in the GUI.
I'm truly sorry that extra field handling is so complicated, but it's just simply complicated. Zotero has planned to add more "real" fields in the GUI (e.g a DOI on all item types) and as these are added, the extra handling should become less and less necessary. This shows a small part of the mapping process, that's going to be cleaned up and part of the visual guide, but the actual graph is much larger.

Is your issue resolved with [edtr:lower][0] | [auth:(unknown)][year][0] on 1203?

I'm good with just the auth part. Thanks for the help.
(Sorry for the delay - traveling)


Add option to keep multiple copies of backups

Coming from GSM and it's laginess I find this tool really good. Now, one thing that would make it even better is if it could keep multiple copies of the backups and backup only if files have changed, just like in GSM a checksum is made then if checksum has changed or there are new files it performs the backup otherwise it skips it.

Hi! I'm glad you've found it helpful :D Regarding the different parts of this:

Skip if not changed: This is already implemented. Ludusavi won't re-copy any files that are the same as in the existing backup. I would definitely keep that behavior for compressed backups as well.
Compression: Tracked by #47.
Multiple copies: Agreed, this would be good to add. I briefly mentioned it in #47, but it's good to track it explicitly since it could be implemented independently.

So I've been testing Ludusavi, and apparently it does copy files that already exist in the backup, cause disk write speeds are high when I'm backing up the same fiiles. Shouldn't it be just read speeds?

@DragonightFury How is your merge option configured? When it's disabled, the backup directory is deleted first.
If it's enabled, then I'm not sure why you'd see that. This is the check for identical files:
https://github.com/mtkennerly/ludusavi/blob/edb55fdb1d22693e5f76bc8da8646351ad0ac848/src/prelude.rs#L694-L703
It will only proceed to the copy/write code if the files have different content (Ok(false)). I tried adding some logging to confirm, and I do see it consistently following the Ok(true) branch when I repeatedly back up the same game.

@mtkennerly I'm using ludusavi-playnite plugin. On their page it says that it has this option enabled. I'm still not sure because if I delete some file from the saves directory, after i backup, it gets deleted too

Gotcha, yeah, the Playnite plugin always uses merging. The full behavior is:

Keep the existing backup folder
If any files are missing or have different content in the backup folder, then copy them over
If any outdated/irrelevant files exist in the backup folder (e.g., a backed up save that no longer exists in the game's live save location), then remove them

Is the third point what you're referring to?

First of all, thank you for creating Ludusavi and using 🦀 to make it a reality. Seeing Rust in use always make me happy. However there are some features I consider missing from Ludusavi, multiple backups being one of them. Here I outline my thoughts on how this feature should be implemented and what I think would be a cohesive user experience.
I'll start by suggesting that when pressing the Back up button, the user should be presented the backup method options: Merge and Recreate. This elevates the Merge option to be on par with Recreate as a standalone backup method.
If backup methods and their options were presented at the top of the pop-up shown after Back up is pressed, then the descriptive text could be placed below and dynamically change to display what actions will occur based on the currently selected choices. Once the user is happy with their choices and the described result, they can press Continue to begin the process.
This kind of UX flow would keep all the backup decision-making on a single page and allow for it to be expanded once a multiple backups feature is introduced. I think the backup methods should be presented to the user categorised by type and method (this assumes future features):

Backup type
Backup method 1
Backup method 2

One backup per game
Merge (merge with backup location)
Recreate (delete and recreate backup location)

Multiple backups per game
Full backups only
Full backups with differentials

Assuming its implementation, Full backups only would be a full backup like Recreate but instead of deleting and recreating the configured Back up to directory, simply increment the subdirectories contained within it by appending some new value to them, a good candidate being be the date and time, up to the current second, at the moment that the process starts (e.g. "Game Name [20220721-131259]"). I think I read somewhere that you'd be moving to making all backups be compressed archives, which I agree with, but I also think an Other settings option to keep using plain folders (i.e. skip archiving altogether) would be appreciated for those unconcerned with the storage used or those who want immediate access to the backed up files.
I propose that the settings which determine how Full backups only and Full backups with differentials work should be controlled at a global level via the Other tab. There we could set some values:
Other > Full backups only:

A limit on the number of full backups to create before Ludusavi starts to prune them. Could be set to 0 for an unchecked number of full backups.

Other > Full backups with differentials:

A limit on the number of diff backups to create post the full backup before a new chain is created (a chain being a full backup plus its diffs).

An option to only ever store one diff in a chain to keep storage usage down (a diff would be created and the old one deleted each backup until the parent option's limit was reached and a new chain begins).

The number of full backup chains to keep. When the limit of diffs is reached, and if some num_prev_chains_to_keep were set to "2", then the previous two full backups and their related diffs will always be kept. If a third chain existed and was the eldest, delete that when starting a new chain.

With multiple backup methods to choose from, an option in the Other tab(?) of Ludusavi could define the preferred method so that it would be preselected for us each time we press Back up (but still be overridable). Recreate (with Merge enabled) should likely be the default because it's the simplest, and it would prevent accidental loss which is possible with Merge disabled.
Problems I've not thought about:

What happens when you perform a Full backups only on a directory that was used as a Recreate backup previously? Maybe the One backup per game and Multiple backups per game backup types should have separate Back up to paths?

I'm sure these ideas will need some work or restructuring when it comes to actual implementation. I was attempting to adhere to how things currently operate, but perhaps a more refined approach will become more obvious once you attempt to code. Hopefully some of what I've written is food for thought.
Thanks.

In case you aren't familiar with incremental and differential backups: A Differential backup is a backup containing the difference between itself and its related full backup. This means any diff backups created before the latest one can be deleted without consequence as long as the full backup remains present (assuming that restoring the most up-to-date files is your goal); 1 full + latest diff = latest backed up files. A full plus a diff can be considered a chain, although I think this term is more suitable for incremental backups which require all previous incs and the full backup to exist to be able to restore the most up-to-date backed up files. A loss of an inc means that the chain is now broken resulting in a partial loss of data post the breaking point of the chain. Using diff backups is more resilient but consumes more storage, whereas the inverse is true for incs. Diffs are the way to go for simplicity's sake in the context of a games user data backup utility.

This is now implemented! Preview:

https://user-images.githubusercontent.com/4934192/184405927-5474cea5-aa01-40dc-b06b-0770a72207d9.mp4

Thanks @mtkennerly!


Added MongoDB default port to constants

Thanks!

My pleasure.  Thanks for your library.
On 6 Nov 2016 12:54 am, "Mark Peek"<EMAIL_ADDRESS>wrote:

Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/cloudtools/troposphere/pull/608#issuecomment-258652869,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAnaO85nnYB93mHF53g5YPiWEorE_QdNks5q7SVOgaJpZM4KozD7
.


Migrating to AndroidX

1.3.0

Support Android X. There might be build issue when using this version. If it so, you either migrate to Android X or using previous version.
Breaking change. Migrate from the deprecated original Android Support Library to AndroidX. This shouldn't result in any functional changes, but it requires any Android apps using this plugin to also migrate if they're using the original support library.

As we are not sure when this will be merged, you can adjust your pubspec.yaml in the meantime:
flutter_full_pdf_viewer:     git:       url: https://github.com/faisalramdan17/flutter_full_pdf_viewer.git       ref: android-x

The version number in the pubspec.yaml file is 1.0.3

okay thanks


Downloading songs from Spotify where the accompanying Youtube video is private crashes SpotDL

Environment

System OS: <Linux, but might also be possible on other platforms-->

Python Version: 3.8.6 (most recent version on Ubuntu 20.10)

Install Source:
PIP, python3 -m pip install spotdl

Version OR Commit hash:
3.3.1

Expected Behaviour

I was expecting when SpotDL sees a private video after scraping the data, that it would skip it
Actual Behaviour

SpotDL crashed to the commandline and stopped downloading the Spotify playlist.
Steps to Reproduce

Have a playlist with https://open.spotify.com/track/6c9EGVj5CaOeoKd9ecMW1U in it (haven't tested a standalone download of only this track)
Run spotdl
See it crash, stopping the playlist download if the song was in it.

Traceback

Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 8, in <module>
    sys.exit(console_entry_point())
  File "/usr/local/lib/python3.8/dist-packages/spotdl/__main__.py", line 112, in console_entry_point
    downloader.download_multiple_songs(songObjList)
  File "/usr/local/lib/python3.8/dist-packages/spotdl/download/downloader.py", line 91, in download_multiple_songs
    self._download_asynchronously(songObjList)
  File "/usr/local/lib/python3.8/dist-packages/spotdl/download/downloader.py", line 355, in _download_asynchronously
    self.loop.run_until_complete(asyncio.gather(*tasks))
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/spotdl/download/downloader.py", line 350, in _pool_download
    return await self.download_song(song_obj)
  File "/usr/local/lib/python3.8/dist-packages/spotdl/download/downloader.py", line 172, in download_song
    youtubeHandler = YouTube(
  File "/home/ddg/.local/lib/python3.8/site-packages/pytube/__main__.py", line 104, in __init__
    self.prefetch()
  File "/home/ddg/.local/lib/python3.8/site-packages/pytube/__main__.py", line 203, in prefetch
    self.check_availability()
  File "/home/ddg/.local/lib/python3.8/site-packages/pytube/__main__.py", line 137, in check_availability
    raise VideoPrivate(video_id=self.video_id)
pytube.exceptions.VideoPrivate: p_d6rh2ArNc is a private video

Any other information:

The private video is https://www.youtube.com/watch?v=p_d6rh2ArNc, which as seen, is indeed private

The song you provided is not available on spotify.?

The song you provided is not available on spotify.?

Resolved on Discord:
https://discord.com/channels/771628785447337985/796571887635267614/802314386978897931
Duplicate of #902.

Resolved on Discord:
https://discord.com/channels/771628785447337985/796571887635267614/802314386978897931
Duplicate of #902.


AKS cluster creation: dropdown list for supported Kubernetes versions does not populate

Setup

Rancher version: 2.8.3
Rancher UI Extensions: none
Browser type & version: latest Firefox and Chrome

Describe the bug
When creating an AKS cluster the dropdown list for supported Kubernetes version does not populate - despite the backend apparently responding with the correct list.

To Reproduce

add an Azure Cloud credential
home -> Create
continue in the wizard

Result
Expected Result
When I click on the Kubernetes Version dropdown, there are options that can be chosen.
Screenshots
See above

This should be resolved in 2.9.0 with the work to port this form from Ember to Vue https://github.com/rancher/dashboard/pull/9669

as mentioned above this is resolved. Verified in 2.9


11ty

Converted site to Eleventy and using TailwindCSS.
Fingers crossed this will auto-deploy when merged too!
Preview of the site (link not guaranteed to work forever): https://inspiring-williams-3ac8d9.netlify.app/events/

@tamouse approved in Slack

This fixes #1, fixes #6, fixes #18, and fixes #19 as well 😅


Using voronoi tessellation to select closest line in multiline charts

When making interactive multiline charts it is often necessary to provide some way to select a specific line. This can be nicely done using an invisible voronoi tessellation—when hovering the cell of a point, the line this point belongs to is marked active.
Example: Multi-Line Voronoi.
Has anyone done this using vx? Or could this be integrated into vx?

Yes this is possible with vx. @williaster built this in a reusable chart form in data-ui. Example:

https://williaster.github.io/data-ui/?selectedKind=xy-chart&selectedStory=LineSeries&full=0&addons=0&stories=1&panelRight=0
vx provides the components to make it (voronoi, linePath) but doesn't have the concept of a chart so it's not an out of the box thing at the moment.

That's great! I thought that vx-voronoi was only used as a visualization component and not as accessor.


Fix when trying to use validator's key to sign messages

Related Github tickets

a list of github issues that are relevant for this PR

Background
Initial assumption (due to the fact that I am still learning cosmos ecosystem) was that we can use validator's keys for signing messages. We can't :) (I mean, we can, but that would be a hack^2).
Testing completed

[x] wrote tests
[x] tested this with Sparrow to see that it works as intended

What does Gravity use for validators to sign messages?

This is something that I've taken from gravity
https://github.com/VolumeFi/gravity-bridge/blob/0bfefb9238213d92f6d950729e1824cb7e124ce0/module/x/gravity/types/msgs.pb.go#L695-L697. They register the eth accounts which they use to verify signatures. I've taken that approach and applied it here.


delay report

my contact sensor gets a lot of double taps when opening and closing giving a lot of double reports. can you add delay, if so How. Please help.

Done!
Go to the folder 'homebridge-contactsensor' that is in the folder node_modules and type ' sudo nano watchpins.py ' into your terminal. Delete everything in there and copy and paste this new code (bottom of this reply, recommend  ssh or config ui to copy paste) you can change number the line ' delay = 0.5 ' to what delay you want in seconds. save it and restart. Hope this helps. please make a backup before doing this in case you accidentally put in something wrong.
Please let me know if it doesn't work or does i am a newbie and is keen to learn (it works for me and will be using it)
watchpins.txt


Using the widget library, grow out a more comprehensive repertoire of widgets.

There needs to be a core widget library containing a comprehensive repertoire of widgets folks can use by default "out of the box".

Involve Toga.


CSA プロトコルに Keepalive と空行送信に関する設定を追加

説明 / Description

チェックリスト / Checklist

MUST

[x] npm test passed
[x] npm run lint was applied without warnings
[x] changes of /docs/webapp not included (except release branch)
[x] console.log not included (except script file)

RECOMMENDED (it depends on what you change)

[ ] unit test added/updated
[x] i18n

Codecov Report
Attention: 50 lines in your changes are missing coverage. Please review.

Comparison is base (e5add79) 52.55% compared to head (2466e36) 52.60%.

Files
Patch %
Lines

src/common/settings/csa.ts
54.34%
11 Missing and 10 partials :warning:

src/background/csa/client.ts
63.41%
10 Missing and 5 partials :warning:

src/background/csa/socket.ts
36.36%
7 Missing :warning:

src/common/i18n/text_template.ts
0.00%
7 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #721      +/-   ##
==========================================
+ Coverage   52.55%   52.60%   +0.05%
==========================================
  Files         107      107
  Lines       17024    17144     +120
  Branches     1576     1592      +16
==========================================
+ Hits         8947     9019      +72
- Misses       6514     6547      +33
- Partials     1563     1578      +15

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


volar 0.37.8 + vue 2 error. Property 'props' does not exist on type '{}'.ts(2339)

Type cannot be resolved by hovering to a variable on template
Property 'props' does not exist on type '{}'.ts(2339)
This issue did not occur in volar 0.37.7.
version
volar 0.37.8
vue 2
vscode 1.68.0
sample project:
https://github.com/juner/vue_volar_error_sample
pattern

volar_vue
vue 2
vue 3

volar 0.37.7

volar 0.37.8

+1

Replace "experimentalCompatMode": 2 with "target": 2  in tsconfig.json

@kingyue737 Thank you. It was very helpful.