Strip Go binary debug symbols

Description
Remove debug symbols and reduce binary file size. In general, the final binary is around 30% less and use less time to load into system memory.
Related issues/PRs

Fixes https://github.com/projectcalico/calico/issues/6649.
Related to https://github.com/projectcalico/calico/pull/6686..

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

TBD

Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

Debug symbols are very useful for... well... debugging!  If we do this we need to start storing off the debug symbols in a way that lets them be retrieved and used with the debugger.

Debug symbols are very useful for... well... debugging! If we do this we need to start storing off the debug symbols in a way that lets them be retrieved and used with the debugger.

Yeah. I will hold-merge on this.


Support timeout for Cypress transactions

Transactions with too large timeouts are bad since they can hold a lock for too long. Let's add a (configurable) limit for the transaction timeout.
https://github.com/ytsaurus/ytsaurus/blob/main/yt/yt/server/master/transaction_server/transaction_manager.cpp#L326 -- this function is called when transaction is started, so the timeout check can be added here.

@gritukan I see that there is already a timeout in the function signature. Need to make it customizable via config ?

@gritukan I see that there is already a timeout in the function signature. Need to make it customizable via config ?

For now, transaction timeout is passed from the client during transaction start. However, transactions with very big timeouts can be bad, so I want to add a maximum timeout in config and throw an error during transaction start if passed timeout is bigger than limit.


Ensemble VCF Variant Phasing and split_biallelic

Hi All,
We are trying to debug losing phased variants produced by mutect2 and vardict when we run the Callers in the order: [vardict, mutect2, gatk-haplotype].
Question 1 - phased variants
We are seeing that when we have the variant callers in the above order, phased variants (MNVs, and delins variants) are not chosen above SNVs for the final ensemble vcf file. We do see that even if a vardict SNV has less support than a gatk variant for example, the vardict variant is chosen.
Question 2- split biallelic:
We found that in bcbio/variants/normalize.py the "normalize" function sets split_biallelic=True by default and bcbio/variants/ensemble.py does not parse options to set this value to False.
vrn_files = [normalize.normalize(f, data, passonly=passonly, rerun_effects=False, remove_oldeffects=True, nonrefonly=True, work_dir=utils.safe_makedir(os.path.join(base_dir, c)))
Example Case, ensemble only shows unphased gatk variant even though vardict was called first:
sample-ensemble-annotated.vcf:2   212426699       .       C       A       235.0   PASS    AC=1;AF=0.2763;CALLERS=vardict,mutect2,gatk-haplotype;DECOMPOSED;LEN=1;TYPE=snp        GT:DP:VD:AD:AF:RD:ALD   0|1:257:71:183,71:0.2763:94,89:35,36
sample-ensemble-annotated.vcf:2   212426700       .       C       A       235.0   PASS    AC=1;AF=0.2763;CALLERS=vardict,mutect2,gatk-haplotype;DECOMPOSED;LEN=1;TYPE=snp        GT:DP:VD:AD:AF:RD:ALD   0|1:257:71:183,71:0.2763:94,89:35,36
sample-gatk-haplotype-annotated.vcf:2     212426699       .       C       A       1168.6  PASS    AC=1;AF=0.5;AN=2;BaseQRankSum=-3.062;ClippingRankSum=-2.521;DP=207;ExcessHet=3.0103;FS=0;MLEAC=1;MLEAF=0.5;MMQ=60,60;MQ=60;MQ0=0;MQRankSum=0;QD=5.65;ReadPosRankSum=2.772;SOR=0.682;EPR=pass  GT:AD:DP:GQ:PL  0/1:141,66:207:99:1176,0,5819
sample-gatk-haplotype-annotated.vcf:2     212426700       .       C       A       2382.6  PASS    AC=1;AF=0.5;AN=2;BaseQRankSum=-2.969;ClippingRankSum=-2.318;DP=211;ExcessHet=3.0103;FS=0.536;MLEAC=1;MLEAF=0.5;MMQ=60,60;MQ=60;MQ0=0;MQRankSum=0;QD=11.29;ReadPosRankSum=2.694;SOR=0.752;EPR=pass     GT:AD:DP:GQ:PL  0/1:142,69:211:99:2390,0,3841
sample-mutect2-annotated.vcf:2    212426699       .       CC      AA      .       PASS    CONTQ=93;ClippingRankSum=-1.983;DP=141;ECNT=1;FS=4.592;GERMQ=70;MBQ=45,34;MFRL=134,127;MMQ=60,60;MPOS=44;MQ=60;MQ0=0;MQRankSum=0;POPAF=7.3;ReadPosRankSum=2.148;SAAF=0.273,0.242,0.281;SAPP=0.008253,0.028,0.964;TLOD=125.1;EPR=pass  GT:AD:AF:DP:F1R2:F2R1   0/1:100,39:0.284:139:41,0:59,39
sample-vardict-annotated.vcf:2    212426699       .       CC      AA      235.0   PASS    SAMPLE=NTP19-001706_S9_R1_001_val_1;TYPE=Complex;DP=257;VD=71;AF=0.2763;BIAS=2:2;REFBIAS=94:89;VARBIAS=35:36;PMEAN=36.8;PSTD=1;QUAL=38.3;QSTD=1;SBF=0.78145;ODDRATIO=1.086;MQ=60;SN=142;HIAF=0.2851;ADJAF=0.035;SHIFT3=0;MSI=2;MSILEN=1;NM=1.2;HICNT=71;HICOV=249;LSEQ=CTTGTGCTCGTGGACATACT;RSEQ=AACAGGCAGCCATGGGGCAT;DUPRATE=0;SPLITREAD=0;SPANPAIR=0;EPR=pass      GT:DP:VD:AD:AF:RD:ALD   0/1:257:71:183,71:0.2763:94,89:35,36
In this post, another user found that the "passonly" was set as a default to False  in this same line so that the "True" option could not be passed. Maybe a similar solution could be implemented for this? https://github.com/bcbio/bcbio-nextgen/issues/2196
Below is our yaml template file as well.
Thank you!
Lauryn Keeler
Template for paired (tumor/normal) variant calling

details:

analysis: variant2
genome_build: GRCh37
metadata:
batch: your-batch-name
phenotype: tumor
sex:
algorithm:
aligner: bwa
background: /isilon/prod/apps/bcbio-data/genomes/Hsapiens/GRCh37/variation/1000G_omni2.5.vcf
ensemble:
numpass: 1
use_filtered: false
mark_duplicates: true
indelcaller: false
min_allele_fraction: 1
remove_lcr: true
umi_type: fastq_name
variant_regions: true
variantcaller: [vardict,mutect2,gatk-haplotype]
coverage_interval: amplicon
effects: false

Hi Lauryn,
Thanks for the question. Regarding question 1) : It looks like it is doing the right thing-- vardict is first in the list so vardict calls will get prioritized over the others. Swapping the order will get GATK calls prioritized, etc. The variant in the ensembl VCF file is the vardict one but decomposed into two variants.
@chapmanb will have to chime in regarding 2) though.

We would prefer to keep the phased variant and not decompose. Is there a current option to disable the decomposition?
Thank you!
Lauryn

Lauryn and Rory;
Thanks much for this question and discussion. For ensemble calling we don't have a way to disable decomposition and retain MNPs. The ensemble combining method is not intelligent enough to be able to group variants with different representations like the MNPs from VarDict and MuTect2 in your example versus the HaplotypeCaller single variant. Without decomposition, it would report both variants giving double calls at that region.
It's a much harder problem to try and handle this different representations generally and merge intelligently and the current ensemble implementation isn't set up to do that. Our best suggestion is to use individual caller outputs if you want to use non-decomposed inputs. Hope this helps.

Hi @chapmanb,
I noticed you recommended here to use a single caller in order to retain non-decomposed MNPs. Based on my experience (as described in https://github.com/bcbio/bcbio-nextgen/issues/2825), running Freebayes as the single variantcaller still ends up splitting MNPs into multiple SNPs tagged as DECOMPOSED. This happens both with germline and with somatic (paired) VC.
Is there a way to retain the original MNPs?

Sorry for the confusion. Single caller output versus multiple ensemble output was referring to the splitting of multiallelic calls, which we do need for ensemble. We don't currently have an option for disabling MNP splitting. We had this in the past but FreeBayes output can look quite different than GATK or other callers with respect to variant output and it caused a lot of confusion in interpretation so we decided to focus on a single set of decomposed outputs as a help for using different methods within bcbio.
If someone wanted to revisit this we'd need to add a tools_off: [mnp_decomposition] option and factor out the post cleaning done to FreeBayes output:
https://github.com/bcbio/bcbio-nextgen/blob/bafc19aa6b391ae3c59abc01fe22e711453b0929/bcbio/variation/freebayes.py#L136
Thanks again for the discussion.

I see, thanks for clarifying. In the end it's fine for me to leave things as they are, since I managed to get WhatsHap working with the output of Freebayes and also with an ensemble VCF file.


feat: support external event queue

This helps nushell to implement commandline edit --accept, the same thing that zle accept-line does for zsh.
See this issue.

This looks interesting. Thanks! Generally, we'd like to not have unwrap() as much as possible. I'll have fun trying this out later.

I updated the code to not use unwrap

Is there another immediate use for this kind of event injection besides for https://github.com/nushell/nushell/issues/11065 / https://github.com/nushell/nushell/pull/13728 ?
If there is a simpler solution with clear semantics that is easy to document/test I would prefer that to trying to be too general and drilling more holes into the internals. (e.g. now the ordering in the event loop could become load bearing to some behavior a library user may come up with, bikeshedding of Arc<Mutex<Vec<E>>> vs channel)
If I understand the goal of running the output from Nushell commandline immediately this could be achieved in a variety of ways. The requirement could be boiled down to a binary choice/enum. When commandline is executed we are outside of Reedline::read_line modifying the buffer, so if I grok it right there is not even a need to synchronize across threads from reedlines perspective.

Is there another immediate use for this kind of event injection besides for nushell/nushell#11065 / nushell/nushell#13728 ?

Fascinating question. I also thought about this. And honestly, probably not.

If there is a simpler solution with clear semantics that is easy to document/test I would prefer that to trying to be too general and drilling more holes into the internals. (e.g. now the ordering in the event loop could become load bearing to some behavior a library user may come up with, bikeshedding of Arc<Mutex<Vec<E>>> vs channel)

I feel like, I don't fully understand what you mean. Reedline is single-threaded, right? Then the lock will be trivial to acquire. Arc<Mutex<X>> is the only way to have something like shared memory. I tried using a channel, but I struggled to make it work. I you wish, I would give it another try.
Something else, I figured after implementing: maybe I don't need the two queues. Currently, there is the local queue and a shared queue. Whenever the local queue is read, I append all the events from the shared queue beforehand. Maybe having one single queue at the first place is also a cleaner implementation?

If I understand the goal of running the output from Nushell commandline immediately this could be achieved in a variety of ways. The requirement could be boiled down to a binary choice/enum. When commandline is executed we are outside of Reedline::read_line modifying the buffer, so if I grok it right there is not even a need to synchronize across threads from reedlines perspective.

Yes, that is the goal. And yes, I also don't see a need for synchronization, except that Rust requires a Mutex to allow the lifetime and access from different locations, basically a borrow check-proof shared memory. Feel free to propose an alternative implementation and I will give it a shot - I am a bit impatient about the feature and eager to implement it.

How do I test this?

it seems to work for "print 'hello'" but not for "!!" or any of the other ! commands.

Because I didn't implement it for the !-commands yet. I'd do that in a separate PR, because I'd prefer to have it as an opt-in and accordingly add a config option

I added the two lines of code that immediately accept the bashisms. What are the maintainers' opinions on this? Obviously, a breaking change. But is it intended that we have to enter twice to accept a bash-expansion? Should this be configurable? I feel like an immediate accept is very sane default behavior.

good move, dialing back the scope creep on both your prs.

OK looking at the way commandline works currently (and also your interim proposal for the change to the ! expansions) I still think there is no current need for a whole separate event queue. I feel that an overly broad public interface like that invites spooky action at a distance or unforeseen states that are harder to reason about.
This could be a simple bool execute_immediately, with an associated method to set it on the Reedline instance or something entirely happening in the nu-cli/EngineState machinations.
Looking at the curent requirements:

Only required event Submit -> no need for a full ReedlineEvent queue, if it is just counting Submits
It would only ever execute one event in the current use with commandline and Submit (with potential for what happens if more events got enqueued) -> just an (atomic) bool
There is no need for Send/Sync logic because of the pedestrian implementation of commandline (I would rather be concerned if implementers of Completer or Menu wrongly think they would through such a channel get access to an event queue to influence Reedlines internal execution, but its order is not determinate or not really sensible.)

I don't particularly like how commandline is implemented and interacting with reedline but I am still inclined to say that the case for this kind of event queue is not particularly strong even if we come up with a better API for commandline shenanigans.
Nushell commandline itself never communicates in parallel with Reedline execution, everything there is happening on the engine state.
Hooks are executed only by nushell and if a ExecuteHostCommand keybinding is reached the Reedline instance yields from read_line as if a buffer content were submitted to the normal REPL loop. -> commandline never executes its code while Reedline has exclusive access.
Buffer content and cursor position is read before the hook evaluation through two methods on Reedline and then moved into the engine state.
The writing of new buffer content coming from commandline executions is managed rather hackily by running a few edit commands in the REPL loop. (This latter part is still a poor design to deal with illegal cursor positions.)
All of this is currently accomplished without shared state with reedline but rather happening on EngineState.
I am open to a better API on reedline's side to make synchronization within the REPL less of a careful dance, but I think this should choose its public API to the clear requirements from commandline instead of providing overly general access before it is actually needed.

Thank you for your thorough introduction to the overall architecture. Now this is much clearer and I understand, why this is a bad implementation.
As soon as I am done with my contributions over at tree-sitter-nu, I'll  come back to this and incorporate my new insights into this implementation :)

Just making this draft until we get back around to it. Thanks for working on this.


tls: remove obsolete openssl version check and fix libressl build

Two more version checks have to be removed.

Removed the OPENSSL_VERSION_NUMBER >= 0x10100000L check completely.


Oops! We encountered an unexpected error. Please try again.

Hi, getting that error every time. With and without proxy.
Captcha found, registering account ...
Oops! We encountered an unexpected error. Please try again.
Username: turnaboutgrand019
Username: turnaboutgrand019 Password:mpQHexKeYD_MK32 BDay: 1973/10/2
Getting email
Solving captcha
Using 2Captcha.
Request sent to the 2captcha wait a bit, Request ID:<PHONE_NUMBER>5
Wating to reslove ... (15 - 300 sec)
Thanks in advance.

i get same

Same happening for me, gonna try to see why

Got the same over here, can't find why

No fix for this yet? Im getting the same issue...

Thanks, could you add me for better contact - xyss#5770

Sure no problem.

I checked that,
Now Twitch using: https://www.kasada.io/
The issue is about Client-Integrity.
For registration, We need an integrity_token.
To get integrity_token, We need to send the request to passport.twitch.tv with x-kpsdk-cd and x-kpsdk-ct.
I know how to get x-kpsdk-ct but I don't know about x-kpsdk-cd.
So the project is not working for now ...
If I figure out how to solve that, I will commit to solving the problem.

no solution found?

I checked that, Now Twitch using: https://www.kasada.io/ The issue is about Client-Integrity. For registration, We need an integrity_token. To get integrity_token, We need to send the request to passport.twitch.tv with x-kpsdk-cd and x-kpsdk-ct. I know how to get x-kpsdk-ct but I don't know about x-kpsdk-cd. So the project is not working for now ... If I figure out how to solve that, I will commit to solving the problem.

an you tell me how to obtain x-kpsdk-ct? I am also trying to figure this out and any information may be helpful.

I checked that, Now Twitch using: https://www.kasada.io/ The issue is about Client-Integrity. For registration, We need an integrity_token. To get integrity_token, We need to send the request to passport.twitch.tv with x-kpsdk-cd and x-kpsdk-ct. I know how to get x-kpsdk-ct but I don't know about x-kpsdk-cd. So the project is not working for now ... If I figure out how to solve that, I will commit to solving the problem.

I know how to get x-kpsdk-cd, tell me how to get it and I will help you with x-kpsdk-cd

Ok now project work, I need to close the issue.


🤗 [REQUEST] - uukuguy/speechless-starcoder2-7b

Model introduction
https://huggingface.co/uukuguy/speechless-starcoder2-7b
speechless-starcoder2-7b
Code: https://github.com/uukuguy/speechless
Use the following dataset to fine-tune bigcode/starcoder2-7b in order to improve the model's reasoning and planning abilities.
Total 986k samples.

teknium/OpenHermes-2.5
TokenBender/python_eval_instruct_51k
Spider
codefuse-ai/Evol-instruction-66k

How to Prompt the Model
This model accepts the Alpaca instruction format.

Instruction:
Implement a linked list in C++
Response:
Model URL
https://huggingface.co/uukuguy/speechless-starcoder2-7b
Data type
float16
Additional instructions (Optional)
No response
Author
Yes
Security

[X] I confirm that the model is safe to run which does not contain any malicious code or content.

Integrity

[X] I confirm that the model comes from unique and original work and does not contain any plagiarism.

HE+:

MBPP+:


bug in "cautious" and "pedantic" read_data

I think there is a bug in the current version of the our Database.read_data method when run in anything but "promiscuous" mode.
The offending section is lines 495-510 of database.py copied below:
        # 1. build metadata as dict
        md = Metadata()

        # 1.1 read in the attributes from the document in the database
        for k in object_doc:
            if k in exclude_keys:
                continue
            if mode == "promiscuous":
                md[k] = object_doc[k]
                continue
            # FIXME: note that we do not check whether the attributes' type in the database matches the schema's definition.
            # This may or may not be correct. Should test in practice and get user feedbacks.
            if read_metadata_schema.is_defined(k) and not read_metadata_schema.is_alias(
                k
            ):
                md[k] = object_doc[k]

What this section is doing is copying all the contents of a document read from MongoDB (the object_doc symbol) to a Metadata container.
When I was rewriting this section for the cleanup of Database I'm currently working on I recognized it was bad for the code to silently drop any attributes that failed the is_defined(k).  i.e. notice that that loop will silently not copy any doc values to md if the key (k) is not defined in the schema.   At the very least we need a fix like this patch in my rewrite of this section:
        md = Metadata()
        dropped_keys=[]

        for k in doc:
            if k in exclude_keys:
                continue

            if metadata_schema.is_defined(k) and not metadata_schema.is_alias(
                k
            ):
                md[k] = doc[k]
            else:
                dropped_keys.append(k)
        if len(dropped_keys) > 0:
            message = "While running with mode={} found {} entries not defined in schema\n".format(mode,len(dropped_keys))
            message += "The attributes linked to the following keys were dropped from converted Metadata container\n"
            for d in dropped_keys:
                message += d
                message += " "
            elog.log_error("doc2md",
                              message,
                              ErrorSeverity.Complaint)

Noting in this revision elog is a return of a function that contains this revision I called doc2md.
The revised code is failing on test reading gridfs data for Seismogram objects in "cautious" mode because the key "gridfs_id" is not defined in the metadata_schema used here.  When I look at the yaml file I see it is true that for a Seismogram the attribute gridfs_id is not defined for Metadata.  I note, however, that it is defined for the  DatabaseSchema for Seismograms.
I am not sure how to fix the revision.   I wonder if the original code meant to use a reference to the database schema in the section above instead of the metadata schema?   @wangyinz can you remind me of your recollection of the original thinking about "cautious" and "pedatic" that they should just drop attributes not defined (is_defined(k) == False).
I think our schema file and this section of database.py need to at least be reconciled.   I cannot actually understand why the existing code actually run the test in test_db_no_spark_dask.py called test_save_read_data.   For my new implementation it fails running "cautious" reads when the input is gridfs.

Update on this.   Adding gridfs_id, cardinal, and orthogonal to the yaml file definition for Seismogram in the schema made the failure of the test in my revised code work.  I reiterate that I do not comprehend how the same test worked for the current version of Database.   It doesn't totally matter because I think the new code does what it should, which is log a complaint error before dropping the data the attribute contained.  At least then the user has hope of figure out why a read failed.
What this does raise is a new suggestion.  I think for the new implementation I will add a new optional parameter which is "kill_if_undefined".   It would kill data any datum with failures of the is_defined schema method conditional.  What do you think?

hmmm.... I don't think it is needed to read the gridfs_id into Metadata, because that is something only useful in the database. Similarly, we didn't have the dir or dfile in Metadata neither. Also, with gridfs_id dropped, it should not fail because the reading of the waveform is not dependent on the content of the Metadata, which is like this:
https://github.com/mspass-team/mspass/blob/23898f45753603606c529b0034e3bac7eaf80ac2/python/mspasspy/db/database.py#L664-L665
For the behavior of the cautious and pedantic mode, I guess we probably should add something that warns what's being dropped, but we just didn't implement that. But the behavior is still correct because for the more strict mode, we want to handle the data only with what's defined in the schema.
I think the "kill_if_undefined" mode that you are suggesting is actually one step further as the older modes will never kill data on read anyway. I am fine with adding that, or make that the default behavior in the pedantic mode maybe.

I guess I still don't understand the thinking of why keys like gridfs_id, dir, or dfile shouldn't be defined for Metadata?   gridfs_id is a case in point as if it isn't defined the update_data method cannot know how to replace the data stored there.  Same would be true if we implemented a similar functionality for storage_mode == "file"- we don't support that currently and probably shouldn't.   That aside, there is definitely a mistake in the metadata schema definition for Seismogram as "cardinal" and "is_orthogonal", which are required by constructor were not defined in the Metadata schema.
For this branch I'm working on I'm going to add gridfs_id, dir, and dfile to the Metadata schema to try to get the current tests to run successfully.   At our next meeting we can discuss whether we want to make that change part of release 2.0.
I also want to point out that I think this particular issue is a case in point about why the rewrite I'm doing is necessary to make MsPASS more maintainable down the road.   The logic if the current read_data method is so convoluted from not breaking the algorithm into well defined functions that it is really hard to understand how it works let alone why the particular test that led to this issue was breaking.   You can look at the cleanup_database branch to see the details, but it might be useful to summarize here what I have done to read_data for this new version:

Functionally, read_data can be used to read both atomic data and ensembles.  It uses the type of arg0 as a hint for the expected data type.  If arg0 is an ObjectId or a dict it assumes it should read atomic objects.  If arg0 is a cursor it assumes it should construct an ensemble.  TimeSeries versus Seismogram is determined from the collection argument as we do currently. Note in contrast that the current read_data only supports input with a dict or ObjectId.
The reader now has two main blocks:  one for reading atomic data and the other for ensembles.
For atomic data there is a new function, doc2md, that converts the document (dict) from MongoDB to a Metadata.  There is a similar function doclist2mdlist that does the same thing to convert a list of documents into a list of Metadata containers.   Note that is currently done with a for loop over the cursor.  We might want to add a list of dict containers as an input option.  Would be easy to do.  doclist2mdlist is conceptually little more than a function that loops over the list running doc2md on each component.  What the doc2md function does that is important for maintenance is standard the way we handle the "mode" argument - the topic of this issues page.  All the logic of "promiscuous", "cautious", and "pedantic" should now only be done through this function.
Each block has an associated private method it calls to construct the type of data object the algorithm infers using the logic above.   The new methods are called _construct_atomic_object and _construct_ensemble where the names describe what they do.  Like doc2md these methods standardize how a datum is to be constructed from MongoDB input.
The method for doing "normalization" has fundamentally changed.   Rather than drive the normalization by a list of normalize collection the new  reader uses a list of BasicMatcher class instances.   I've made the new reader backward compatible by allowing the "normalize" argument to be a list of strings which are converted to database driven id matchers - what we do now.   I just realized I hadn't finished this for ensembles, but will do so soon.  Ensembles have an added complexity that the normalization has to be defined as for the ensemble or for members.   I propose we add a new method argument, normalize_ensemble, that is a list of BasicMatcher instances to apply to ensemble Metadata and we use normalize for ensemble members.  That takes a bit more coding, but seems more consistent to me.
The current draft only handles the load_history option for atomic data.   I see I need to fix that too.  This one was just an omission and does not require any api change.
The new implementation introduces the concept of an "abortion".   An abortion is a datum that is killed before it is born (constructed).   Documentation changes I'm working on in parallel with this explain the concept in detail along with how it is handled.
As noted elsewhere, I think, I've extended the Undertaker class to mesh with these new changes.  The main changes were extensions to allow it to handle ensembles and deal with the new concept of "abortions".   Undertaker is another piece of code that standardizes a single concept:  how to handle "dead" data - just like the name implies.

yeah... I think you got a point here. Now thinking about it, I find the original design is mixing concepts. The question here is: what is the different between the schema for database and that for the metadata. The old design seems to have some distinctions between the two, but now I think they are probably the same thing. i.e., database stores the metadata.
Anyway, I think the right approach is that we should treat the original schema definitions in the yaml file as one way to represent the database and metadata, but that shouldn't be the only way to handle the two. But, yes, in that schema, there is no straightforward way to do the update.

In yesterday's meeting we discussed this issue.   We made a tentative decision to put special handling of attributes creating this problem in the revision of read_data and write_data and leave the default schema definition as is.   When I started to try to do this, I concluded that was not a good idea for several reasons:

Hard coding names like dir, dfile, foff, nbytes, format, and storage_mode into the python code somewhat defeats the purpose of having a configurable schema defined in the yaml file.  If we lock them into the code this way the names become keywords, which is a maintenance issue and an unnecessarily dogmatic viewpoint.
It turns out the way my revised code works hard coding these attributes into the writer is not hard, but the reader is a completely different story.   Why is an implementation detail, but the solution requires a major change in the algorithm I used to handle ensembles.  (Atomic data isn't hard and note our current ensemble readers don't handle this consistently anyway.)   Can be done, but the other points I'm making here convince me it is not wise.
On reflection, I think it is actually an error in the current schema to think of the set of attributes here as not being something that should  always be posted to Metadata anyway.  I think that thinking comes from thinking of each attribute in isolation.   e.g. a dfile attribute by itself is nonsense as it is insufficient to define the provenance of a piece of data.  On the other hand, if the Metadata of an object contained all of the following:  storage_mode=="file", dir, dfile, foff, and format=="binary" it is clear that datum was constructed from a file defined by dir and dfile and reading binary float64 values from position foff.  Adding something like format=="mseed" and a value for nbytes means about the same thing but that the data came from a miniseed file.  Other formats would be similar.  For formatted data, in particular, that detail is important as if there are Metadata problems downstream it is essential to know these details to track the problem.  The counter argument is that in principle these same things could be find by querying the database with the datum's ObjectId, BUT the value of a ObjectId of the source waveform can be invalidated by an intermediate save.

So, unless there is strong objection I am going to back up to changing the Metadata schema to add the fields storage_mode, gridfs_id, dir, dfile, foff, and format.  As we get more experience with cloud services from Earthscope I suspect we will need to do additional additions to Metadata to support other possible definitions of storage_mode.
What this issue does reveal, however, is that we may need to do some more work to more cleanly handle variations in what we have been handling with the attribute keyword "storage_mode".   What I know for sure can be an issue is that it is very easy to get conflicting and/or stale attributes stored in the database when the method of storage changes.  For example, if one initiates a workflow with data read from a miniseed file, the Metadata loaded would normally contain at least the following attributes: storage_mode, format, dir, dfile, foff, nbytes.   If that datum is processed and then saved with storage_mode="gridfs" without editing the Metadata the result will have the following attributes that are relics of the parent:  format, dir, dfile, foff, nbytes.   I think we need to design a generic solution to the relic attribute problem for writers.  Making it generic is the challenge.  I could do this right now for a specific example like the one above, but the same solution might not work, for example, if the parent was some other format.   Defining the generic problem is a design issue we need to consider before fixing this related problem.


Significant perfomance regression in 0.2.4

Hi! After 0.2.3 -> 0.2.4 upgrade I experienced a significant increase of startup (not compile!) time (6-7 times) in application with >250 layers. Currently I can't reproduce this with reasonably short example, I will continue investigation

Can't try 0.2.5 due to zio/zio#4809 (it forces to keep zio 1.0.1)

Are you still experiencing this? It's very strange that this would occur. Perhaps there are layers that aren't being memoized. Are all your layers vals?

I'm forced to stay on 0.2.3 due to this issue. Some of my layers are defs, but all layers passed to fromMagic are vals (some layers are still composed by hand)

New version should 🤞 fix this hopefully. 0.3.0

Thank You @kitlangton! I will take a try immediately after zio/zio#4809 fixed

Any luck with the newer versions?

Ah, sorry, it was, in fact, a ++-related issue, now we're fine, let's close this)


Provide a JSON API endpoint for events

The endpoint URL can be something simple like /api/events. You should be able to filter by the same query string filters available on the homepage.
The controller for this endpoint can go here: app/Http/Controllers/Api/EventController.php

The endpoint URL can be something simple like /api/events. You should be able to filter by the same query string filters available on the homepage.
The controller for this endpoint can go here: app/Http/Controllers/Api/EventController.php

I can take on this issue.

@imacrayon do you mean something like this?
api/events?after=2019-11-11 00:00:00&before=2019-10-18 00:00:0
also should the endpoint be versioned?

also should the event come from the local db or an external source

@lacasera yes, that example URL looks correct. The JSON info would come from the local database. The code will probably be similar to the index method in the existing EventController except that it should  return a JSON response instead of an Inertia view

pr to fix #28


⚠ ClamAV has degraded performance

In e18cf9a, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 71 ms

Resolved: ClamAV performance has improved in 2f7c936.


Add relation between res user and payment gateways #6992

Coverage increased (+0.52%) to 73.56% when pulling f0ffe758cb9bec0079b0169f603113ac6075d9c6 on aroraumang:feature/task-6992- into c3f79080727175e8885eae75c80ce05933315dba on openlabs:3.2.


Feature/ Support graphql interfaces

defineByDocumentFile
  "./assets/simple.gql"
  [gql|
    query SomeQuery($arg1: String!) {
    ...
    }
  |]

# simple.gql
interface Node {
  nodeId: ID!
}
type SimpleInterfaceType implements Node {
  nodeId: ID!
  name: String!
}

Interface and implements are treated as unknown keywords while parsing.
@nalchevanidze mentioned building tuples of (interfaceName, ...interfaceFields) that can be checked against during validation.

no. i think the best solution is to support interfaces only for schema Validation.
we will have raw and valid GQLSchema ,

RawSchema: that will include interfaces too. and will be validated before Morpheus interpreter consumes it.
ValidSchema: only validated Types. interfaces are excluded (there is no need  to represent them in Haskell, they will be used only for Validation).

after validation every:
   type SimpleInterfaceType implements Node {
      nodeId: ID!
      name: String!
 }

will be converted to just  object type
   type Node {
      nodeId: ID!
      name: String!
 }


Add advanced configuration options for piwik analytics

In order to use one Piwik instance for tracking several sub domains, some advanced piwik configuration options have to be set. This kind of configuration should be part of the administration UI of Rocket.Chat.
This issue implements the following options
Prepend the site domain to the page title when tracking
_paq.push(["setDocumentTitle", document.domain + "/" + document.title]);
Track visitors across all subdomains
_paq.push(["setCookieDomain", "*.your.domain.tld"]);
In the 'Outlinks' report, hide clicks to known alias URLs
_paq.push(["setDomains", ["*.your.domain.tld"]]);

Lgtm.
Merging later. Please PR to Rocket.Chat fork.

In https://github.com/RocketChat/Rocket.Chat/issues/233 is shortly discussed why RC won't implement all Piwik Config options. Might be a reason for decline the pull request.
Also I'm not sure if the second commit: https://github.com/mrsimpson/Rocket.Chat/commit/89df55c33f8b0d93d8390b20b9a39ea43e12b915 meets the intend of Piwik. "set a custom URL" leads for incorrect links in Piwik UI. But was the only way I found to tweak the sub domain handling.

Lessons learned: A piwik website configuration has settings for subdomains. Using this settings makes the commit unnecessary https://github.com/mrsimpson/Rocket.Chat/commit/89df55c33f8b0d93d8390b20b9a39ea43e12b915

Piwik supports tracking the same website into two different piwik instances.
@see https://piwik.org/blog/2017/02/complete-guide-tracking-websites-web-apps-multiple-piwiks-easily-efficiently/
Would be nice if that would be configureable in Rocket.Chat Analytics Admin UI.
For this feature I will open a new Issue. Posting it here, soon.

https://github.com/mrsimpson/Rocket.Chat/issues/17

merged and deployed.

make a pull request to RC core.


GitHub Workflows security hardening

This PR adds explicit permissions section to workflows. This is a security best practice because by default workflows run with extended set of permissions (except from on: pull_request from external forks). By specifying any permission explicitly all others are set to none. By using the principle of least privilege the damage a compromised workflow can do (because of an injection or compromised third party tool or action) is restricted.
It is recommended to have most strict permissions on the top level and grant write permissions on job level case by case.

I merged #2819 over this since it had slightly narrower permissions set. Thanks!


Can you tell me how to change the currency of virtual currency?

Can you tell me how to change the currency of virtual currency?
I hope I can do some experiments.

Please review the FAQ on our website: https://brd.com or contact support if your answer is not there<EMAIL_ADDRESS>


TypeScript def: statusCode and more

HttpResponseMessage.statusCode is a number not a string. (It's copied from XMLHttpRequest.status, which is a unsigned short according to MDN).

Still a problem in the beta 1.

Feel free to submit a PR. Should only take a minute.


Refactor various dialog components' trigger prop

Converted the trigger prop on dialog components the use the technique to be a render function instead of an element.  The trigger render function takes onClick as an argument and needs to assign that event passed handler to the correct tag for the dialogs to open.
Applied the refactor to:

DiskImageEditor
NicEditor
DeleteConfirmationModal
RestoreConfirmationModal

Fixes: #1123
NOTE: The Action component in src/components/VmActions/Action.js uses a very similar pattern but with the children prop.  I'm not changing the Action component at this time since its use looks ok.

What about action-delete for both NICs and Disks? It seems that the issue is reproduced for delete button as well...

What about action-delete for both NICs and Disks? It seems that the issue is reproduced for delete button as well...

Good catch.  I'll get those updated as well.

Does it integrate with the ...rest prop on OverlayTooltip?

@yodem - That change on OverlayTooltip will not impact or be impacted by this change.  This change really just makes it very explicit where the events handlers are coming from (the onClick parameter give to the trigger render function) and how they're attached to DOM elements (onClick={onClick})

(rebased)

@leistnerova when do you plan to test this?

@leistnerova when do you plan to test this?

I did not know, we should test it. So no plan yet.

ci build please


Layouts lose content scroll position on browser back button

Here is a quick way of reproducing this:

Visit this template page http://www.getmdl.io/templates/dashboard/index.html
Resize your browser height to be small enough to see scroll bar of the main content.
Browse to a new address.
Press browser back button.
The main content of the layout has lost its scroll position, and is reset to be at top.

This could be an annoying, and indeed blocking, problem for long contents with links to visit other pages.

This is a duplicate of #1120.
TL;DR: It will hopefully get fixed with the layout rewrite, which can only happen for v2. Before that, we have to live with that behavior now :confused:


Could not instantiate controller with $scope injected

angularjs Could not instantiate controller with $scope injected when i use the new router v0.5.3
the same controller is working without $scope injected
here is a Plunker : http://plnkr.co/edit/Fl128QOUpPOJ9vCIdQVB?p=preview

No new release yet, working on master if you build from scratch using gulp.

you are not alone on that one!
http://stackoverflow.com/q/30167898/599991
http://jsbin.com/yegowi/1/edit?html,js,output

If anyone's running into this in a spot where building from source isn't practical, we've made an interim npm package that fixes this:
npm install<EMAIL_ADDRESS>
(Note that there are some potentially breaking changes, such as renaming ng-viewport to ng-output, and moving the $componentMapperProvider classes into a factory, which may need to be dealt with). :fish:

Have you made a test with 0.5.4 you can load it from https://rawgit.com/loomio/router/master/dist/router.es5.js in your http://plnkr.co/edit/Fl128QOUpPOJ9vCIdQVB?p=preview

Thanks for the tip, @ZeGregg; I've updated the dist files in v0.5.7, which should work better for ya.
Cheers!

Here is a working version http://plnkr.co/edit/dSdiKr?p=preview

this workaround is unfortunately not working for me.
would be nice to see some progress here soon.

If there's an interim package available, could this please be added into the main release?

The child scope (the controller's scope) is created when the activate method is being called - that is - after the actual compilation of the view is finished. Therefore, it is simple impossible the inject $scope - it is simple created later.
Long story made short...
There is a better, built-in solution for this.

Instead of assigning your properties to the non-existing-yet $scope, simply assign them on the "this" variable - the controller instance variable. Then on the view, you can use them by referring the controller name:
angular.module('app.home').controller('HomeController', HomeController);

function HomeController() {
    this.name = "tomer";
}

And now on the template html:

{{home.name}}

Makes sense?

The child scope (the controller's scope) is created when the activate method is being called - that is - after the actual compilation of the view is finished. Therefore, it is simple impossible the inject $scope - it is simple created later.
Long story made short...
There is a better, built-in solution for this.

Instead of assigning your properties to the non-existing-yet $scope, simply assign them on the "this" variable - the controller instance variable. Then on the view, you can use them by referring the controller name:
angular.module('app.home').controller('HomeController', HomeController);

function HomeController() {
    this.name = "tomer";
}

And now on the template html:

{{home.name}}

Makes sense?

Hi @TomerAvni , it makes sense but how do you use $broadcast or $emit in you controller without $scope?

@ TomerAvni ,I used ur approach nd it worked fine,but I have an another issue now,I wand to change my expression {{home.result}} placed inside my form.I will change it after form submission .With $scope it would hv been working without page reload in a simple manner .But as for using this.result it doesnt works,it works only first time when page loads.Any workaround?

@flavienbert and @TomerAvni If I understand correctly, I think Angular 2.0 wants to get rid of the idea of $scope in controllers (no $scope.$emit and $scope.$broadcast anymore) so that Angular can have uni-directional event flow (mimic-ing React), so my interpretation of all this is that we should start using $rootScope.$broadcast for events/state changes? Is this a fair assessment of how to move forward?

Unable to instantiate controller with $scope despite using dist/router.es5.js from the commit fix: allow injecting $scope into controller constructor .
I have also tried router.es5.scope.js from the plnkr by @iPaoo above but it errors out saying
Unknown provider: $componentLoaderProvider
I am stuck in middle of an A1 application + ngNewRouter where in one of the controllers, I need to use $scope.
Please provide me a version of router.es5.js which allows that


First I tried to build it using cmake-gui, the generated .soln file failed to compile because there were syntax errors in the code.
Then I tried to use the direct vc2019 files located in the build folder and the resulting static library when linked gives me tons of undefined references when compiling with even when I set the library directories properly and include the lib in the required libraries.
Next I tried to use https://github.com/qPCR4vir/nana_helloworld because I usually use CLion and CMake for my projects and it keeps asking me for boost and I hate boost, I wanna get as far away as possible from the boost library and use standard c++ filesystem and threads and mutexes but it always gives me compile error because boost is not found.
How am I supposed to use it then? I can't even get it to compile and I've been trying to do it for 3-4 days with no luck.
 Ironically I got wxWidgets working easily but NANA DOESN'T WORK AT ALL.

Dealing with compilation/linker/runtime/human errors is a daily thing for C++ programmers. We ought to pay attention to the cause of errors if we want to solve them. Since you failed to build nana_helloworld application, let me see if I can reproduce some errors you might have seen.
I used Visual Studio 2019 as it's rather a no-brainer to me.

After cloning nana repository from https://github.com/cnjinhao/nana.git , open the file build/vc2019/nana.sln.

Firstly, I noticed that Solution Configuration is Release and Solution Platform is x64 so that I have to use the same setting when I build the sample application later.

Bulding the nana library went smoothly and output library file nana_v142_Release_x64.lib is located in build/bin folder.

Nextly, I created Visual Studio project for the sample application. The source code I used is https://github.com/qPCR4vir/nana_helloworld/blob/9c6651b373f8bf12e097781c683bc8f85c8f30ee/helloworld_demo.cpp Al I need to do now is create VS project and build the application with it.

Choose menu item, File > New > Project

Select Empty Project C++ template and push Next button

Specify Project name and Location as you like then press Create button

New project is created but Solution Configuration is Debug and Solution Platform is x86 so let's change those and add helloworld_demo.cpp file to the project by drag&dropping the file to Solution Explorer view

Then we have to change project properties.

Select VC++ Directories and add new lines to Include Directories and Library Directories.

Select C/C++ > Code Generation and specify Multi-threaded (/MT) to Runtime Library.

Select C/C++ > Language and specify Preview - Features from the Latest C++ Working Draft (/std:c++latest) to C++ Language Standard.

Select Linker > Input and add nana_v142_Release_x64.lib to Additional Dependencies.

And build the project.

How am I supposed to use it then? I can't even get it to compile and I've been trying to do it for 3-4 days with no luck.

Welp, wish you luck then.

 Ironically I got wxWidgets working easily but NANA DOESN'T WORK AT ALL.

Honestly, I think wxWidgets has more history and well matured. Its library design can be somewhat conservative but that won't be an issue with a little bit of patience.

Hello, thanks for helping out but I usually go the CMake route on building projects.
Any tips on how to go about adding the project to my already existing CMake?
My CMakeLists is the bare minimum right now
cmake_minimum_required(VERSION 3.16)
project(Axzil)
include_directories(include)
add_executable(Axzil
src/main.cpp)
Do I compile the libraries separately and use them as .lib and .a or is there a way to simply add the project to cmake and let it compile along with my project and be used as static library.

For reference, I checked with current master commit 554c4fe87fc31b8ee104228e9117d545d34855b5
Here are the steps to build helloworld_demo application with CMake.

Inside nana's build directory, I typed below command.

D:\libs\nana_cnjinhao\build>cmake .. -DCMAKE_GENERATOR_PLATFORM=x64

Now that VS files are created in the directory, I typed below command.

D:\libs\nana_cnjinhao\build>cmake --build . --target ALL_BUILD --config Release

Unexpectedly, I saw some compilation errors listed below.

  split_string.cpp
D:\libs\nana_cnjinhao\source\system\split_string.cpp(21,32): error C2146: 構文エラー: ')' が、識別子 'and' の前に必要です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(21,32): error C2065: 'and': 定義されていない識別子です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(21,36): error C2146: 構文エラー: ';' が、識別子 'sep' の前に必要です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(21,56): error C2059: 構文エラー: ')' [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(22,13): error C2059: 構文エラー: ';' [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(25,32): error C2065: 'start': 定義されていない識別子です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(25,39): error C2065: 'end': 定義されていない識別子です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(25,45): error C2065: 'start': 定義されていない識別子です。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(28,2): error C2059: 構文エラー: 'return' [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(32,1): error C2059: 構文エラー: '}' [D:\libs\nana_cnjinhao\build\nana.vcxproj]
D:\libs\nana_cnjinhao\source\system\split_string.cpp(32,1): error C2143: 構文エラー: ';' が '}' の前にありません。 [D:\libs\nana_cnjinhao\build\nana.vcxproj]

These errors have been fixed by #564 so you can incorporate those changes manually or wait next release by cnjinhao or use develop branch (cnjinhao:develop-1.8) or change VS project property shown in below picture.

After that I typed below command again to build library file.

D:\libs\nana_cnjinhao\build>cmake --build . --target ALL_BUILD --config Release

I see no errors during the build and nana.lib file appeared in build/Release folder.

Then I tried to build helloworld_demo application with CMake. Below is the CMakeLists.txt file I used.

cmake_minimum_required(VERSION 3.16)

project(helloworld_demo)

if(MSVC)
	add_definitions("/MP")
endif()

set_property(DIRECTORY PROPERTY VS_STARTUP_PROJECT helloworld_demo)

set(NANA D:/libs/nana_cnjinhao)

include_directories(${NANA}/include)

add_executable(helloworld_demo
	helloworld_demo.cpp
	)
set_property(TARGET helloworld_demo PROPERTY
             MSVC_RUNTIME_LIBRARY "MultiThreaded$<$<CONFIG:Debug>:Debug>")

find_library(NANA_LIBRARY nana HINTS ${NANA}/build/Release)
target_link_libraries(helloworld_demo PRIVATE ${NANA_LIBRARY})
target_compile_features(helloworld_demo PUBLIC cxx_std_17)

Firstly mkdir build folder and inside the folder I typed below command.

D:\projects\nana_test\build>cmake .. -DCMAKE_GENERATOR_PLATFORM=x64

After that I typed below command.

D:\projects\nana_test\build>cmake --build . --target ALL_BUILD --config Release

Then helloworld_demo.exe was built inside build/Release folder.

Do I compile the libraries separately and use them as .lib and .a or is there a way to simply add the project to cmake and let it compile along with my project and be used as static library.

I suppose it's pretty much doable after you built nana library. I see many *.cmake files inside nana's build/cmake folder.
How to use those files is left to be discovered by you.

Thanks a lot.


Add FFmpeg WASM support

[x] Add FFmpeg.wasm
[ ] Video File-Converter (GIF)
[ ] Video Editing Tools (trim etc)

SharedArrayBuffer is not available by default.
The cost of implementing this library is way higher than the benefits it provides. Maybe a separate web-app?
Shelved for now.


struct MemPool: Rewrite Rav1dMemPool as fully safe and use it for Rav1dContext::picture_pool

Fixes #641.

This rewrites from scratch the unsafe Rav1dMemPool as the safe MemPool.  There are a few important differences here:

Rav1dMemPool used Box<[u8]> essentially.  MemPool uses Vec<T> (T = u8 in this case), which saves on some reallocations, as Rav1dMemPool would only reuse an allocation if it was exactly the same size.
By returning Vecs, the caller can decide how initialization is done, as MemPool only does the capacity allocation.

Since these picture allocations are large, this code is performance-critical.  When running on ./tests/large/chimera_10b_1080p.ivf -l 100, I observed a ~5% performance regression initially (401 ms baseline, 415 ms new).  However, if I do not zero out the data buffer (using Vec::resize_with, which only initializes the new elements from the previous use), then I observe a 5% performance improvement (376 ms).  As I explain in a comment, I do Vec::resize_with in cfg!(debug_assertions), but in release mode, I unsafely call Vec::set_len.  The intermediate slices of the data that are materialized are technically UB, since the data is uninitialized.  However, it should always be written to before read, and since this is so performance-critical, I think this is worth it.
That said, sometimes the benchmarking seemed noisy, so someone else can take a look, too, if they'd like.

Just as a datapoint, in zlib-rs we similarly have big allocations that we don't want to initialize. We use the BorrowedBuf (vendored because it is unstable) https://doc.rust-lang.org/std/io/struct.BorrowedBuf.html
The core idea behind this type is to store data as MaybeUninit<u8>:
pub struct BorrowedBuf<'data> {
    /// The buffer's underlying data.
    buf: &'data mut [MaybeUninit<u8>],
    /// The length of `self.buf` which is known to be filled.
    filled: usize,
    /// The length of `self.buf` which is known to be initialized.
    init: usize,
}

And then pass out (mutable) slices to the filled part of just &[u8].
    /// Returns a shared reference to the filled portion of the buffer.
    #[inline]
    pub fn filled(&self) -> &[u8] {
        // SAFETY: We only slice the filled part of the buffer, which is always valid
        unsafe { MaybeUninit::slice_assume_init_ref(&self.buf[0..self.filled]) }
    }

and there are functions to initialize a certain part of the underlying buffer.
So now the cost of ensuring that there is no UB is just some integer compares on each access rather than initializing all of the memory at the start.
Maybe this idea can be used here too?

Just as a datapoint, in zlib-rs we similarly have big allocations that we don't want to initialize. We use the BorrowedBuf (vendored because it is unstable) https://doc.rust-lang.org/std/io/struct.BorrowedBuf.html
The core idea behind this type is to store data as MaybeUninit<u8>:
pub struct BorrowedBuf<'data> {
    /// The buffer's underlying data.
    buf: &'data mut [MaybeUninit<u8>],
    /// The length of `self.buf` which is known to be filled.
    filled: usize,
    /// The length of `self.buf` which is known to be initialized.
    init: usize,
}

And then pass out (mutable) slices to the filled part of just &[u8].
    /// Returns a shared reference to the filled portion of the buffer.
    #[inline]
    pub fn filled(&self) -> &[u8] {
        // SAFETY: We only slice the filled part of the buffer, which is always valid
        unsafe { MaybeUninit::slice_assume_init_ref(&self.buf[0..self.filled]) }
    }

and there are functions to initialize a certain part of the underlying buffer.
So now the cost of ensuring that there is no UB is just some integer compares on each access rather than initializing all of the memory at the start.
Maybe this idea can be used here too?

Hey @folkertdev, thanks for the suggestion!  In this, what's the difference between self.filled and self.init?  Perhaps that could work later down the road for the picture data, so I'll keep it in mind, but for now I've decided to make things MaybeUninit<u8>s since other pic allocators (like from C) won't initialize the data anyways.

The BorrowedBuf data structure is used e.g. in Tokio (and in the future, also stdlib maybe) where you have some buffer that is reused multiple times (usually multiple reads), and you don't want to initialize the memory up-front.
So init keeps track of how many bytes have been initialized. This is at least filled, but may be higher if a previous iteration initialized more bytes and was then cleared. So a slice buf[..filled] is data that is currently valid/meaningful, buf[..init] is initialized data (so technically that can be converted to a &[u8] safely) and the remainder of buf is - as far as rust knows - uninitialized data.
Often it may be sufficient to have filled and init be the same. Then you have something similar to a Cursor where everything to the left is valid and guaranteed initialized, and to the right unused and potentially uninitialized.

The BorrowedBuf data structure is used e.g. in Tokio (and in the future, also stdlib maybe) where you have some buffer that is reused multiple times (usually multiple reads), and you don't want to initialize the memory up-front.
So init keeps track of how many bytes have been initialized. This is at least filled, but may be higher if a previous iteration initialized more bytes and was then cleared. So a slice buf[..filled] is data that is currently valid/meaningful, buf[..init] is initialized data (so technically that can be converted to a &[u8] safely) and the remainder of buf is - as far as rust knows - uninitialized data.
Often it may be sufficient to have filled and init be the same. Then you have something similar to a Cursor where everything to the left is valid and guaranteed initialized, and to the right unused and potentially uninitialized.

Oh, I didn't know it might be added to std at some point.  That's promising.  We'll keep this in mind; it definitely may prove very useful elsewhere.  We currently do something a little bit similar just with a Vec by calling .resize() without first .clear()ing it, since we just need initialized bytes and not necessarily correctly filled bytes.


⚠ Bofrost.lu has degraded performance

In b7056b3, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 970 ms

Resolved: Bofrost.lu performance has improved in 8610c61.


Squeeze error

Trying to run main.lua, I'm getting the following error:
Word vocab size: 599589, Char vocab size: 42, Max word length (incl. padding):  65
creating an LSTM-CNN with 2 layers
/home/robin/dev/torch/install/bin/luajit: ./model/TDNN.lua:27: attempt to call field 'Squeeze' (a nil value)
stack traceback:
        ./model/TDNN.lua:27: in function 'tdnn'
        ./model/LSTMTDNN.lua:53: in function 'lstmtdnn'
        main.lua:150: in main chunk
        [C]: in function 'dofile'
        .../dev/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
        [C]: at 0x00406670

I see some recent commits about Squeeze so I presume this is related; any tips?
Thank you!

Never mind! Updated packages as recommended in other issues. Sorry about that.


Add Python 3.13 support

@dmayo3 can you show me path to repo and files

Hi @Brijeshthummar02
Thanks for reaching out.
I'm not sure what you mean about the path to the repo or which files?
If you're looking for files related to Python version compatibility, you'll want to check the following:
pyproject.toml: This file includes details on dependencies and compatible Python versions.
tox.ini: This is used for running tests across different Python environments.
.python-version: Specifies the Python versions for local development.
CI workflows:

.github/workflows/mocksafe.yml
.github/workflows/release.yaml

These control our CI pipelines, including which Python versions are used for testing and releases.
Let me know if you need any more help navigating these files or making changes to add Python 3.13 support.


Feature/xref standardisation

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
if you change the schema you must patch the test databases as well, see Updating the schema
the PR must not fail unit testing

Description
The list of sources used for gene naming and gene description has been consolidated in a single place and used for most vertebrate species. This removes the need to maintain multiple lists for different species with the risk of them being out of sync.
Additionally, when multiple choices are available, a re-ordering has been applied to increase the chances of making the same choice when the pipeline is re-run
Use case
Describe the problem. Please provide an example representing the motivation behind the need for having these changes in place.
We have genes where the name and the description are different, because different sources were used for each. This is confusing users. We also have cases where the same gene has a different name from one release to another, because there are two choices and one is arbitrarily chosen each time
Benefits
If applicable, describe the advantages the changes will have.
More consistency in the xref assignment, less queries from users, easier explanation of the name assignment strategy
Possible Drawbacks
If applicable, describe any possible undesirable consequence of the changes.
None that I can see
Testing
Have you added/modified unit tests to test the changes?
no test cases, but the xref pipeline has been run on a large number of species to check the results
If so, do the tests pass/fail?
the pipeline runs fine and produces consistent results on the species it was tested on
Have you run the entire test suite and no regression was detected?
NA

Coverage remained the same at 78.333% when pulling 54690723edfe753820075a826f534ceb1601d362 on feature/xref_standardisation into b82332c075009fb255ef4f4b18cf6c338afd1b48 on master.

the gene_display_xref_sources method returns a reference array and a hash
the gene_description_sources method used to return a reference array and now returns a reference array and a hash
the hash is the second argument and is therefore ignored as not expected
this also means that if we want to add an ignore_list (badly named list as it is in fact a hash) for the gene_display_xref_sources, the argument is already provided

It looks to me like gene_description_sources used to return an actual array, not a reference array, which is what @precedence is expecting - there's no dereferencing.

That's a fair point, let me have a look

That was a good spot, I have changed the return type and was able to test it on our favourite xenopus example, ENSXETG00000034365
it now gets both display_xref and description from the same NCBIGene xref


Simple seed project

Adds a project that looks like this when running:

Omg. I need to stop doing this.


fix(fonts): Remove FontAwesome, add SourceSansPro

Resolves #237
Impact: minor
Type: bugfix
Component

Removes FontAwesome all.css file
Add SourceSansPro

Screenshots

Testing

Make sure to turn off Source Sans Pro font from your Font Book
http://localhost:4040/#!/Typography

🚀 Preview deployed
Built with commit bda4480898de6f19410153a512f237600346f078
https://deploy-preview-238--stoic-hodgkin-c0179e.netlify.com

:tada: This PR is included in version 0.23.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add permission checks to Help command

Right now if the does not have enough permissions the whole bot just crashes.
I encountered this in my fork and implemented this fix.

Hey^^
Yeah, I will fix that soon. Thanks for your issue.

Alright. This is something with your fork since my version doesn't event handle errors in this case to prevent a bot crash. Because of that I don't think that would be necessary.


[#5072]Improvement(docs): Add docs for Trino cascading query

What changes were proposed in this pull request?
Add docs for Trino cascading query
Why are the changes needed?
Fix: #5072
Does this PR introduce any user-facing change?
Update docs
How was this patch tested?
NO

@mchades @FANNG1
Would you like to take a look?


Extract animation root motion with .vmdl decompile

Fixes #665

So is this movement something we can show/implement in our model viewer? If so, can you make a separate issue for that please.

So is this movement something we can show/implement in our model viewer? If so, can you make a separate issue for that please.

I have another branch that modifies the model's transforms using the delta information from the movement data. There's a few more things to figure out there though, but I think this part of it was already very useful for anim exports, so I've made this PR.


Problem with Input fields in IE

Hello everyone!
Cyrillic characters in input fields are displayed incorrectly when I use Internet Explorer (I tried with IE-11). Field displays something like "?@825B" instead of "привет". Does anybody know how to fix it?
FF and Google chrome displays characters correctly.
Try at http://semantic-ui.com/elements/input.html

Sorry that commit was tagged incorrectly and is not related to this issue.

Closing until there's a valid test case.


Add fsspec support to open_dataset

Codecov Report

Merging #45 (94d87aa) into main (afcb81f) will decrease coverage by 0.68%.
 Consider uploading reports for the commit 878133e to get more accurate results

@@            Coverage Diff             @@
##             main      #45      +/-   ##
==========================================
- Coverage   93.12%   92.43%   -0.69%
==========================================
  Files           7        7
  Lines         538      542       +4
  Branches       62       64       +2
==========================================
  Hits          501      501
- Misses         20       22       +2
- Partials       17       19       +2

Impacted Files
Coverage Δ

xarray_sentinel/sentinel1.py
92.96% <71.42%> (-1.86%)
:arrow_down:

tests/test_esa_safe.py
100.00% <100.00%> (ø)

xarray_sentinel/esa_safe.py
81.88% <100.00%> (-0.26%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update afcb81f...878133e. Read the comment docs.

At the moment all datasets except the bursts ones work with fsspec as well:
>>> from xarray_sentinel import sentinel1
>>> gs_urlpath = "gs://mybucket/S1B_IW_SLC__1SDV_20210401T052622_20210401T052650_026269_032297_EFA4.SAFE"
>>> sentinel1.open_dataset(gs_urlpath, group="IW1/orbit")
<xarray.Dataset>
Dimensions:       (azimuth_time: 17)
Coordinates:
  * azimuth_time  (azimuth_time) datetime64[ns] 2021-04-01T05:25:19 ... 2021-...
Data variables:
    x             (azimuth_time) float64 4.3e+06 4.359e+06 ... 5.187e+06
    y             (azimuth_time) float64 1.454e+06 1.453e+06 ... 1.408e+06
    z             (azimuth_time) float64 5.419e+06 5.372e+06 ... 4.593e+06
    vx            (azimuth_time) float64 5.963e+03 5.914e+03 ... 5.103e+03
    vy            (azimuth_time) float64 -91.12 -116.1 -140.9 ... -454.7 -478.0
    vz            (azimuth_time) float64 -4.695e+03 -4.756e+03 ... -5.602e+03
Attributes:
    reference_system:  Earth Fixed
    Conventions:       CF-1.7
    title:             Orbit information used by the IPF during processing
    comment:           The dataset contains a sets of orbit state vectors tha...
    history:           created by xarray_sentinel-0.2...

I'll add a test and consider merging this before attempting to get bursts to work as well.

I added a test. I also found that an effort is underway to have rasterio / GDAL play well with fsspec, but it is far from operational, see: https://github.com/mapbox/rasterio/pull/2141
Please review


Snapshot store screenshots on failure

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Description
The flag store_screenshots_on_failure was added to allow screenshots to be collected even if tests failed. Instead of assuming the error code 65 means that the build failed, we now assume that the tests failed and therefore allow to get the screenshots that were made.
If the flag store_screenshots_on_failure is not set, the behavior of snapshot is the same as before.
If the flag store_screenshots_on_failure is set, we collect and generate the report before raising an error.
Due to complexity issues rubocop reported (after my additions), the "launch" methods was split into several smaller ones.
Motivation and Context
We wanted to make screenshots of failing tests. We added snapshot to a XCTest teardown and even if a test failed we now have a screenshot of the screen that the error occurs in.
Our test was adding a snapshot call to tearDown of a XCTestCase and then calling XCTFail in a test method. With the flag on, it behaved as before and no screenshots were collected. With the flag on, screenshots will get collected and a report generated and still the snapshot step will raise an exception.

Hey @mbalex! Thank you so much for the PR! I am a bit hesitant to approve this PR because it seems like a lot of code change to add functionality that is a bit out of scope of snapshot given that the tool's intent is to generate screenshots for distribution to the AppStore rather than capturing failed test output. I think we can work towards this if we take smaller, more simple steps. I am going to close this for now, but we are definitely open to discussion about this so please do not hesitate to keep discussion flowing! 🚀 🐙


Module 7: Design a Report in Power BI Desktop, Part 1

Module: 07
Lab/Demo: 1, Exercise 1
Task: 06
Step: 10
Description of issue
Instructions need to be updated...
Currently it states:
Step 10: Add the following fields to the visual wells/areas:
Axis: Date | Month
Value: Sales | Sales and Targets | Target
It should be corrected to:
Y-Axis: Date | Month
X-Axis: Sales | Sales and Targets | Target
Cheers,
Jeff Smalley MCT
Little Chicago

This one seems to be related to https://github.com/MicrosoftLearning/PL-300-Microsoft-Power-BI-Data-Analyst/issues/96.
It would be beneficial to check other steps / visualizations for the same UI renaming while resolving this issue.

This has been updated.


Not able to user Wiremock as a HTTPS Proxy server

I am able to generate the wiremock.jks in windows and using the following command to start the Wiremock from Command prompt
Command: C:\wiremock>C:\jdk-22.0.1\bin\java -cp "wiremock-standalone-3.9.1.jar;wiremock-graphql-extension-0.9.0-jar-with-dependencies.jar" wiremock.Run --extensions io.github.nilwurtz.GraphqlBodyMatcher --port 9999 --https-port 2929 --https-keystore wiremock.jks --enable-browser-proxying --trust-all-proxy-targets --verbose
After that I am using https:localhost:9091 as my proxy to fetch the Wiremock Response in chrome browser, buy getting following Error in chrome browser:
net::ERR_PROXY_CONNECTION_FAILED
Can someone please help me resolve this issue?

Hi @AnishVohra I suggest you head over to the Slack community to ask about this as it seems like a support issue rather than a bug.


Peer Dependencies npm install --save @Sentry/capacitor Fails

Environment
Angular 16: I can successfully download @sentry/angular-ivy
Capacitor 5:
Iconic 7:
npm 9.5.1
node 18.16
Steps to Reproduce

Following the installation guide for angular/capacitor set up
npm install --save @sentry/capacitor @sentry/angular-ivy or npm install --save @sentry/capacitor or npm install --save @sentry/capacitor --force
installation fails

Expected Result
The npm install would be successful
Actual Result
install fails (see screenshots)
Current Dependencies:
"dependencies": {
"@angular/common": "^16.0.3",
"@angular/core": "^16.0.3",
"@angular/forms": "^16.0.3",
"@angular/platform-browser": "^16.0.3",
"@angular/platform-browser-dynamic": "^16.0.3",
"@angular/router": "^16.0.3",
"@awesome-cordova-plugins/app-version": "^6.3.0",
"@capacitor-community/firebase-analytics": "^5.0.1",
"@capacitor/android": "^5.0.4",
"@capacitor/app": "^5.0.2",
"@capacitor/camera": "^5.0.2",
"@capacitor/core": "^5.0.5",
"@capacitor/haptics": "^5.0.2",
"@capacitor/ios": "^5.0.4",
"@capacitor/keyboard": "^5.0.2",
"@capacitor/status-bar": "^5.0.2",
"@ionic/angular": "^7.0.9",
"@ionic/pwa-elements": "^3.1.1",
"@ionic/storage-angular": "^4.0.0",
"@trapezedev/configure": "^7.0.10",
"capacitor-native-settings": "^5.0.0",
"cordova-plugin-app-version": "^0.1.14",
"envsub": "^4.1.0",
"ionicons": "^7.1.1",
"ngx-mask": "^16.0.8",
"rxjs": "^7.8.1",
"swiper": "^8.4.6",
"tslib": "^2.5.2",
"zone.js": "^0.13.0"
},
"devDependencies": {
"@angular-devkit/build-angular": "^16.0.3",
"@angular-eslint/builder": "^16.0.3",
"@angular-eslint/eslint-plugin": "^16.0.3",
"@angular-eslint/eslint-plugin-template": "^16.0.3",
"@angular-eslint/template-parser": "^16.0.3",
"@angular/cli": "^16.0.3",
"@angular/compiler": "*",
"@angular/compiler-cli": "^16.0.3",
"@angular/language-service": "^16.0.3",
"@capacitor/cli": "^5.0.6-nightly-20230627T150701.0",
"@ionic/angular-toolkit": "^9.0.0",
"@ionic/cli": "^7.1.1",
"@types/jasmine": "^4.3.2",
"@types/jasminewd2": "^2.0.10",
"@types/node": "^20.2.5",
"@typescript-eslint/eslint-plugin": "^5.59.8",
"@typescript-eslint/parser": "^5.59.8",
"eslint": "^8.41.0",
"eslint-plugin-import": "^2.27.5",
"eslint-plugin-jsdoc": "^45.0.0",
"eslint-plugin-prefer-arrow": "^1.2.3",
"jasmine-core": "^4.6.0",
"jasmine-spec-reporter": "^7.0.0",
"karma": "^6.4.2",
"karma-chrome-launcher": "^3.2.0",
"karma-coverage": "^2.2.0",
"karma-coverage-istanbul-reporter": "^3.0.3",
"karma-jasmine": "^5.1.0",
"karma-jasmine-html-reporter": "^2.0.0",
"prettier": "^2.8.8",
"protractor": "^7.0.0",
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
},

Does it install with the following command?
npm install --save-exact<EMAIL_ADDRESS><EMAIL_ADDRESS>--update-sentry-capacitor

@lucas-zimerman That results in the same error.
It looks like the @sentry/angular peer dependency of @sentry/capacitor conflicts with any Angular 16 project, even though @sentry/angular is marked as optional. Maybe related to a fix done for https://github.com/npm/cli/issues/4859, but I am not sure.

I think the simplest fix would be to replace the peer dependency @sentry/angular with @sentry/angular-ivy in the @sentry/capacitor package.
I tried adding overrides to my package.json, but that did not work in this case.

Any plans on getting this fixed?

Any update on this?

Made a pr for this: https://github.com/getsentry/sentry-capacitor/pull/458

@lucas-zimerman Can you please approve and merge the changes so that we can start using the fix soon (with Angular 16)? Thanks a lot!

I was able to install this package by placing overrides in my package.json
  "overrides": {
    "@sentry/angular": {
      "@angular/common": "*",
      "@angular/core": "*",
      "@angular/router": "*"
    }
  }

I was able to install this package by placing overrides in my package.json
  "overrides": {
    "@sentry/angular": {
      "@angular/common": "*",
      "@angular/core": "*",
      "@angular/router": "*"
    }
  }

Did it build with angular 16 using sentry-angular instead angular-ivy?

Did it build with angular 16 using sentry/angular instead of /angular-ivy?

It builds with @sentry/angular-ivy. With those lines added to your package.json, the command npm install --save @sentry/capacitor @sentry/angular-ivy should succeed.

Hello everyone I've tried to reproduce the issue with a new project and I was unsuccessful.
https://github.com/krystofwoldrich/sentry-capacitor-angular-ivy/blob/64024cf710c47b523bed4a9dd5e7cc26c965c62a/package.json
Could anyone share a minimal reproducible example?
Based on the peerDependencies of sentry-capacitor the @sentry/angular package should not install automatically in projects with Angular 12 and newer.
I used this command to add Sentry npm install --save --exact<EMAIL_ADDRESS><EMAIL_ADDRESS>Without specifying the version of @sentry/angular-ivy the latest gets installed which can cause issues with mismatched types with sentry-capacitor.

Hello everyone I've tried to reproduce the issue with a new project and I was unsuccessful.
https://github.com/krystofwoldrich/sentry-capacitor-angular-ivy/blob/64024cf710c47b523bed4a9dd5e7cc26c965c62a/package.json
Could anyone share a minimal reproducible example?
Based on the peerDependencies of sentry-capacitor the @sentry/angular package should not install automatically in projects with Angular 12 and newer.
I used this command to add Sentry npm install --save --exact<EMAIL_ADDRESS><EMAIL_ADDRESS>Without specifying the version of @sentry/angular-ivy the latest gets installed which can cause issues with mismatched types with sentry-capacitor.

add to package.json
"overrides": {
"@sentry/angular": {
"@angular/common": "",
"@angular/core": "",
"@angular/router": "*"
}
}
npm install --save<EMAIL_ADDRESS>(latest atm)
npm install<EMAIL_ADDRESS>the angular-ivy should match the javascript version of the sentry/capacitor

I'm also no longer able to reproduce this issue, even with the original reporter's command. Maybe something was fixed on the npm side? I have since upgraded npm to v10.1.0 where before I was running version 9, I think.

@lonaru Thanks for the message. I've tried with npm 8.19.4 and that worked.

Since we can't reproduce the issue I'm closing this. In case there is a new context or reproducible example, please, comment here and we can reopen it.


[installation errors] fatal error: nccl.h: No such file or directory

Hello, thanks for providing such a great work. However, I cannot install tutel successfully during
the compilation. I have exported the lib path of nccl_2.7.8-1-cuda10.1/include/nccl.h into LD_LIBRARY_PATH. But the error logs seem that it still cannot find the NCCL path.
Do you have any idea on how to solve this error? Thanks!

running install_lib
running build_py
running build_ext
building 'tutel_custom_kernel' extension
Emitting ninja build file /mnt/lustre/chengguangliang/zhouqianyu/segdgformer/cvpr_2023/tutel-main/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] /mnt/cache/share/gcc/gcc-7.3.0/bin/g++ -MMD -MF /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/build/temp.linux-x86_64-3.7/
tutel/custom/custom_kernel.o.d -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/mnt/lustre/chengguangliang/miniconda3/lib/python3
.7/site-packages/torch/include -I/mnt/lustre/chengguangliang/miniconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/mnt/lustre/che
ngguangliang/miniconda3/lib/python3.7/site-packages/torch/include/TH -I/mnt/lustre/chengguangliang/miniconda3/lib/python3.7/site-packages/torch/include/TH
C -I/mnt/lustre/share/cuda-10.1/include -I/mnt/lustre/chengguangliang/miniconda3/include/python3.7m -c -c /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/tutel/custom/custom_kernel.cpp -o /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/build/temp.linux-x86_64-
3.7/tutel/custom/custom_kernel.o -Wno-sign-compare -Wno-unused-but-set-variable -Wno-terminate -Wno-unused-function -Wno-strict-aliasing -DUSE_GPU -DUSE_N
CCL -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENS
ION_NAME=tutel_custom_kernel -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
FAILED: /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/build/temp.linux-x86_64-3.7/tutel/custom/custom_kernel.o
/mnt/cache/share/gcc/gcc-7.3.0/bin/g++ -MMD -MF /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/build/temp.linux-x86_64-3.7/tutel/
custom/custom_kernel.o.d -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/mnt/lustre/chengguangliang/miniconda3/lib/python3.7/sit
e-packages/torch/include -I/mnt/lustre/chengguangliang/miniconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/mnt/lustre/chengguan
gliang/miniconda3/lib/python3.7/site-packages/torch/include/TH -I/mnt/lustre/chengguangliang/miniconda3/lib/python3.7/site-packages/torch/include/THC -I/m
nt/lustre/share/cuda-10.1/include -I/mnt/lustre/chengguangliang/miniconda3/include/python3.7m -c -c /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/tutel/custom/custom_kernel.cpp -o /mnt/lustre/chengguangliang/zhouqianyu/tutel-main/build/temp.linux-x86_64-3.7/tu
tel/custom/custom_kernel.o -Wno-sign-compare -Wno-unused-but-set-variable -Wno-terminate -Wno-unused-function -Wno-strict-aliasing -DUSE_GPU -DUSE_NCCL -D
TORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NA
ME=tutel_custom_kernel -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/mnt/lustre/chengguangliang/zhouqianyu/tutel-main/tutel/custom/custom_kernel.cpp:19:10: fatal error: nccl.h: No such file or directory
#include <nccl.h>
^~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Try installing without NCCL extension..

My machine details are as follows:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Apr_24_19:10:27_PDT_2019
Cuda compilation tools, release 10.1, V10.1.168

I used pytorch1.8.1 with cuda 10.1. I wonder whether tutal can be installed with cuda10.1?
I used the following commands for the installation:

export PATH=/mnt/lustre/share/gcc/gcc-5.4/bin/:$PATH
export LD_LIBRARY_PATH=/mnt/lustre/share/polaris/dep/nccl_2.7.8-1-cuda10.1/include/nccl.h:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/mnt/lustre/share/gcc/gmp-4.3.2/lib:/mnt/lustre/share/gcc/mpfr-2.4.2/lib:/mnt/lustre/share/gcc/mpc-0.8.1/lib:$LD_LIBRARY_PATH
export TORCH_CUDA_ARCH_LIST='3.5;5.0+PTX;6.0;7.0'

python setup.py install --user

This is an improper environment configuration not recognized by pytorch. Can you make a copy of nccl.h to /usr/include, and a copy of libnccl.so to /usr/lib/x86_64-linux-gnu? (Softlink is also fine.)


Create factory for creating Feign clients manually exactly like they would have been created by the @FeignClient annotation

Enhancement
We generate our Feign client interfaces with swagger-codegen. OpenAPI (formerly known as Swagger) is widely used to describe schemas for (RESTful) interfaces.
So we have our defined OpenAPI schema and out of this schema the Feign client interface classes are generated.
Now we have those interfaces but we can't add the @FeignClient annotation because the classes are generated.
Today we are creating the Feign clients manually like it's described here.
But that leaves a bad vibe to us. We don't know for sure if we did it exactly like the FeignClientFactoryBean does. And we have to keep track on further development in later versions of Spring Cloud.
So a factory method would be nice that creates a Feign client exactly like it's been done by using the @FeignClient annotation.
This factory method shall get the target interface as input parameter as well as all the parameters from the annotation.
The factory will then be injected into a @Bean definition in a @Configuration class like this:
@Configuration
public class FeignConfig {
    @Bean
    public FeignClientInterface feignClient(final FeignClientFactory feignClientFactory) {
        return feignClientFactory.create(FeignClientInterface.class, "feign-client-name");
    }

Or perhaps as builder:
@Configuration
public class FeignConfig {
    @Bean
    public FeignClientInterface feignClient(final FeignClient.Builder feignClientBuilder) {
        return feignClientBuilder
                .targetClass(FeignClientInterface.class)
                .name("feign-client-name")
                .decode404(true)
                .primary(false)
                .build();
    }
}

I would be happy to create a patch.
Just point me into the right direction and show me a good starting point.

Patches are welcome, take a look at our contribution guidelines to get started
https://github.com/spring-cloud/spring-cloud-openfeign/blob/master/.github/CONTRIBUTING.md

Closed via  750cb35


Mapper training during policy training

Hi there,
I have been working on a variation to the projection unit to add a different type of "fog of war" approach for the sensors. However I do not fully understand the code implementation because the mapper training and policy training seem to happen simultaneously. Wouldn't that lead to sorted, correlated data/label pairs in the supervised part? Is there some shuffling happening that I am missing?
Would really appreciate an answer to how this was approached.

We have a large replay buffer that stores data over multiple episodes. We randomly sample data from this buffer to break the correlation in data.

Thanks a lot. I might have to train them seperately since I have not enough memory for such a large replay buffer.


Obj loader

We need to add something like Martin's obj loader for 0.5. Probably don't need all the rotation and scaling stuff - we can defer to Vector for that. Rather it should be something brutally simple: load the obj and present it as a dict mapping vectors to blocks. For the material mapping we might to provide something a little extra - maybe a function to query all the materials used by the file, or something?
Also need to provide recipes showing how to scale/translate/rotate the resulting dict of vectors (although that should be pretty obvious from the vector tutorial, but you can bet people who want obj loading will jump past that stuff).

I'm declaring this done - it's working nicely enough for me now. The only thing to add is tests; that can wait for 0.6 and I want to get this out before the weekend!


[red-knot] Add support for editable installs to the module resolver

Summary
This PR adds support for editable installations in site-packages to the module resolver. Still a few details to be checked vis-à-vis Python's runtime semantics.
Work towards https://github.com/astral-sh/ruff/issues/11653.
Test Plan
cargo test -p red_knot_module_resolver

Dude, stop working 😂

Dude, stop working 😂 I thought you needed some rest

I have an addiction :(

An unrelated comment for resolver (and some of them might be because how I structured the code). I find it difficult to read the code because I always have to jump up and down. The ModuleResolutionSettings aren't next to the ValidatedSearchPathSettings, instead, the Pth code becomes in between. We shouldn't refactor the code as part of this PR but maybe you can come up with an ordering that brings the code closer together.

Yeah, I have the same opinion. I'm planning on doing a followup where I move some more code out into a separate submodule just for finding the search paths -- but although this PR makes the problem worse, I didn't want to do it in this PR, as it would have made the diff harder to read and review.

to use IndexMap over OrderedSet.

I already changed the PR so that it uses indexmap::IndexSet internally rather than ordermap::OrderSet. But I think I have a confusingly named type alias in there. I'll change the name of that.


Create kurdish.ts

Description
Add Kurdish (کوردی) layout.
Checks

[x] Tests ( npm run test ) are passing

Thanks for adding this layout - make sure all changes (from other PRs) are in the same branch so the test can run properly. Regards

Hello @GoRaN909, I have compared your layout to other Kurdish keyboards online and sadly they are not similar. For your layout to be accepted it needs to be standardized with the proper sources. I will decline on this basis but you may reapply once it fulfills the requirements. Thanks!

Hey @GoRaN909 if you can provide layouts that you're basing yours from I would be glad to see them. Those I looked at appeared after a "Kurdish virtual keyboard" search on Google. One tip is to test your layout in the sandbox per the instructions here, making sure that keys don't look weird or text overflows from their buttons. Thanks!


Screenshots are outdated

Please see https://developercommunity.visualstudio.com/content/problem/733283/cannot-disable-preview-features.html
The screenshots are out of date which led me to believe I was unable to disable "Preview Features" because it wasn't showing up in the menu.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b4f95345-6686-8e29-ff3f-cbea4a6eb24d
Version Independent ID: 695d975e-ea9b-139b-4f98-f9655a751f59
Content: Turn preview features on or off - Azure DevOps Services
Content Source: docs/project/navigation/preview-features.md
Product: devops
Technology: devops-collab
GitHub Login: @KathrynEE
Microsoft Alias: kaelli

Closing this issue. Question has been addressed in Developer Community link.


fix(CSS) remove position: relative

It is my understanding that position: relative is only needed for special cases. If people need it, they can add it to the element in question. Testing this change in my own app resulted in zero visual change, and a slightly smaller vendor.css.
This also facilitates the usage of ember-popper with flexi elements. As is, my tooltip addon is borked on the relatively positioned elements.
Eager to get this into 2.0.0 since it is a breaking change.

I will need confirmation on more browsers.

Seems fine to me, but agree with @runspired that we need to check all browsers. Definitely seen some things break in Safari for no reason with flexbox.

I don't think this is entirely worth the headache of breaking changes it may cause for people, and I'm happy to report that Popper.js now works fine with relatively positioned elements.
Closing this out for now


CAMEL-18452: Integrate azure-sdk-bom in a non-conflicting manner

Accidentally closed my original PR which is: https://github.com/apache/camel/pull/8266


Blog only shows the 4.0.0 release on new.nodejs.org

https://new.nodejs.org/en/blog/ shows the new release while https://nodejs.org/en/blog/ does not show it.

LGTM on both, maybe a caching issue on your side?

likely the forced 4h cache timeout we're pushing with cloudflare on html assets, something we need to tune, perhaps by sending better headers from the server

^ to clarify, the 4h is to both edge caches and the server but whenever there's a new website deploy the edge caches get purged so it'd be the browser side that's caching and a force reload should take care of it

Ah that makes sense. I see it fine now. Thanks!


You are running a SCAM site and Hosting code on Github

You aren't very good at obfuscating your code are you ? All I want is my money back and I will go away, simple.

Hello @thomascrha,
if you got scammed you should contact the police instead of harassing someone that has absolutely nothing to do with whatever happened to you.
Since you spammed on the issue tracker of several of my open source projects you should have noticed that I'm the author of a number of open source projects, some of which also have demo sites hosted on GitHub pages.
It is entirely possible - though not a good idea - that a developer includes resources from some of my demo pages in their web page, which can easily explain those domain requests you pointed out in your screenshot.
It also seems the author of the page from your screenshot used my photo (https://blueimp.net/static/profile-74fe7ba7.jpg) without my permission.
Again, I have nothing to do with this site and certainly don't approve of their usage of my picture.
The owner of the domain apparently registered it via GoDaddy, which you can find out via their WHOIS database, or simply by using the whois command-line program.
So I suggest you contact them about the abuse and again, should contact the police if you got scammed.
Kind regards,
Sebastian


Raspi arm build ?

Anyone build this docker, for arm ?

would love to see an arm build as well.  wasn’t obvious to me how to build an arm version either. :(

would love to see an arm build as well.  wasn’t obvious to me how to build an arm version either. :(

Plus one for an arm64 docker!

Plus one for an arm64 docker!

Isn't that the same thing?

Sorry, "plus one" as in "another vote for"...

Hah gotcha

Hi, please try this one docker pull timeseriesadmin/timeseriesadmin:0.1.13-multiplatform-dev.0 https://hub.docker.com/r/timeseriesadmin/timeseriesadmin/tags?page=1&ordering=last_updated and let me know if it works as I don't have any testing environment for that.
I've never publish any multi-platform Docker images before, so I'm counting on your help and feedback.
If it will work I can extend my release pipeline with multi-platform builds as it seems to be pretty straight forward.
For my own reference here are the commands which I used to build this one:
docker buildx create --use
docker buildx build  --platform linux/amd64,linux/arm64,linux/arm/v7 -t timeseriesadmin/timeseriesadmin:0.1.13-multiplatform-dev.0 --push .

@jmarceli the arm build works perfectly.  was able to pull that docker and publish no issue.  Works great on my raspberry pi4.  Thanks!

It runs, have not tested yet but it works!

I'm glad to hear that. I will include multi-platform builds to the next releases.


Mocha 4.0.0 support?

Would be good to have. Not sure if any changes are necessary beyond the peer dep bump.

mocha 4 drops support for older node versions - versions we need to keep supporting.
It's probably really easy to add mocha 4 support here, and we should do that, but all of Airbnb's packages should remain on mocha 3 indefinitely.


bump version to 0.10.0

bower.json in the repo root only has 0.9 specified. This bumps to 0.10.0 to match the current relase

Thanks, appreciated. I'm making lot of changes and trying to automatize the releasing process with semantic-release, but some parts are manual yet.

thanks for all of your work. this directive has been indespensible to my app!

@tombatossals Current git tag for 0.10.0 has version 0.9.0 in bower.json. Could you move the tag maybe?


Inconsistent import ordering via ruff check --fix --select I?

Thanks for your wonderful work!
Situation
We do see some unexplained behaviour of ruff check --fix --select I.
Given I have two different macOS devices, with this spec

macOS 14.6.1 (23G93)
Python 3.11.9 via Homebrew
equally created venv
ruff 0.6.3 via poetry install or uv pip install, no difference

The reduced execution looks like this:
$ .venv/bin/python -V
Python 3.11.9
$ .venv/bin/ruff -V
ruff 0.6.3
$ .venv/bin/python -m poetry run ruff check --config ruff.toml --fix --select I cluster_api/alembic_/env.py
All checks passed!

Issue
On one of both devices we see the file to stay like this:

on the other one it will be changed like so:

This applies to other files as well, but lets focus on one example. Do you have any suggestion what may be wrong?
Config
Ruff configuration in place for both of them
$ cat ruff.toml
# https://docs.astral.sh/ruff/configuration/

# Exclude a variety of commonly ignored directories.
exclude = [
    ".bzr",
    ".direnv",
    ".eggs",
    ".git",
    ".git-rewrite",
    ".hg",
    ".ipynb_checkpoints",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".pyenv",
    ".pytest_cache",
    ".pytype",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    ".vscode",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "site-packages",
    "venv",
]

# The GitHub editor is 127 chars wide
line-length = 120
indent-width = 4

target-version = "py311"

[lint]
# 1. Enable flake8-bugbear (`B`) rules, in addition to the defaults.
# select = ["E4", "E7", "E9", "F", "B"]

# Disable fix for unused imports (`F401`).
unfixable = ["F401"]

[lint.per-file-ignores]
"tests/*" = ["F811"]
"smoke_tests/*" = ["F811"]

[format]
# Like Black, respect magic trailing commas.
skip-magic-trailing-comma = false

Turned out that we had an alembic package top-level, which was refactored to be moved. That top-level alembic still existed on some devices, which caused ruff to detect it as local package.
Thanks @hossainemruz for figuring!


Fix Float#truncate signature.

Float#truncate can get an Integer as an argument and return Integer or Float
depending on the arg value.

Amended the previous commit to include the original signature.

Thank you! Looks great!
I've kicked off the CI, we'll merge when it passes!