[SUPPORT] File is deleted during inline compaction on MOR table causing subsequent FileNotFoundException on a reader

Describe the problem you faced
When inline compaction is turned on and when the actual compaction plan is completed, the commit file is referencing the file which has been deleted during the compaction process. Later, this is causing the reader to fail with FileNotFoudException
To Reproduce
I managed to reproduce the issue on a constant basis. After the first compaction action is completed it causes all subsequent reads to fail, because the commit file is referencing the already deleted parquet file on the system. Please see Additional Context session for more details. The issues can only be reproduced when multiple tables are used within the same SparkSession.
Expected behavior
After inline/async compaction the commit files in .hoodie folder are in sync with the files in the file system. Also there are no files deleted during the compaction.
Main observations so far

The issue is only reproducible with multiple StreamingQuery. The issue is not reproducible with a single Hudi table
The issue is happening right after the first round of compaction happens (compaction happens for each StreamingQuery separately after configured 3600 seconds)
The issue is reproducible both for inline compaction and async compaction scenarios
The issue is reproducible with the code written without using foreachBatch and also with using new SparkSession and dedicated thread for each StreamingQuery
The root cause of the issue is that during the compaction the parquet file is being deleted (most probably by deleteInvalidFilesByPartitions in HoodieTable class). But the commit file is still referencing that file in section fileIdAndRelativePaths. It might be the concurrency issue during the compaction phase
changing 'hoodie.write.markers.type' did not remove the issue

Environment Description

EMR version: 6.5.0

Hudi version : 0.9.0-amzn-1

Spark version : 3.1.2

Hadoop version : 3.2.1

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : no

Additional context
We are using Spark streaming with Kafka topics as a source. Topic -> foreachBatch -> StreamingQuery per table -> DataFrame write -> Hudi MOR table. For each table we are using the following related configuration options
        "hoodie.datasource.write.table.type" = "MERGE_ON_READ"
        "hoodie.datasource.write.precombine.field" = "value.source.lsn"
        "hoodie.datasource.write.recordkey.field" = "cluster,shard,key"
        "hoodie.datasource.write.partitionpath.field" = "table,cluster,shard"
        "hoodie.datasource.write.keygenerator.class" = "org.apache.hudi.keygen.ComplexKeyGenerator"
        "hoodie.datasource.write.hive_style_partitioning" = "true"
        "hoodie.finalize.write.parallelism" = "12"
        "hoodie.upsert.shuffle.parallelism" = "12"

        "hoodie.compact.inline" = "true"

        "hoodie.compact.inline.max.delta.seconds" = "720"

        "hoodie.compact.inline.trigger.strategy" = "TIME_ELAPSED"

        "hoodie.clean.automatic" = "true"

        "hoodie.cleaner.policy" = "KEEP_LATEST_COMMITS"

        "hoodie.cleaner.commits.retained" = "18"

        "hoodie.metadata.cleaner.commits.retained" = "18"

        "hoodie.keep.min.commits" = "36"

        "hoodie.keep.max.commits" = "72"

        "hoodie.clustering.inline" = "false"
        "hoodie.clustering.inline.max.commits" = "4"
        "hoodie.clustering.plan.strategy.target.file.max.bytes" = "1073741824"
        "hoodie.clustering.plan.strategy.small.file.limit" = "629145600"

        "hoodie.metadata.enable" = "false"
        "hoodie.metadata.keep.min.commits" = "36"
        "hoodie.metadata.keep.max.commits" = "72"

        "hoodie.datasource.compaction.async.enable" = "false"

        "hoodie.write.markers.type" = "DIRECT"

        "hoodie.embed.timeline.server" = "false"

Course of Events
Let us take the file which the reader tries to find 4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet and show how this is changed

Compaction completed and and there is also no cleans yet to delete the old files
╔═════════════════════════╤═══════════╤═══════════════════════════════╗
║ Compaction Instant Time │ State     │ Total FileIds to be Compacted ║
╠═════════════════════════╪═══════════╪═══════════════════════════════╣
║<PHONE_NUMBER>2305          │ COMPLETED │ 3                             ║
╚═════════════════════════╧═══════════╧═══════════════════════════════╝

═══════════╤═════════════════════════╤═════════════════════╤══════════════════╗
║ CleanTime │ EarliestCommandRetained │ Total Files Deleted │ Total Time Taken ║
╠═══════════╧═════════════════════════╧═════════════════════╧══════════════════╣
║ (empty)                                                                      ║
╚══════════════════════════════════════════════════════════════════════════════╝

On s3 we can see the following timeline for the compaction process. Please mark the modification time
20220411202305.commit			         commit		April 11, 2022, 22:23:55 (UTC+02:00)
20220411202305.compaction.inflight	         inflight	        April 11, 2022, 22:23:08 (UTC+02:00)
20220411202305.compaction.requested	 requested	April 11, 2022, 22:23:07 (UTC+02:00)

On S3 we can see the following
4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet	Delete marker		April 11, 2022, 22:23:55 (UTC+02:00)
4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet	parquet			April 11, 2022, 22:23:28 (UTC+02:00)

4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1198-14280_20220411202305.parquet	parquet			April 11, 2022, 22:23:54 (UTC+02:00)
4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_2-75-1434_20220411191603.parquet		parquet			April 11, 2022, 21:19:15 (UTC+02:00)
Please pay attention to the fact that the file under consideration has been deleted with the delete marker at the same time the compaction commit happened which is 22:23:55. Also please pay attention that the only thing that changed is the writeToken. After that moment there is a new file 4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1198-14280_20220411202305.parquet. However, this file is not reflected in<PHONE_NUMBER>2305.commit which can be seen below
"fileId" : "4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0",
"path" : "cluster=96/shard=14377/4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet",
"prevCommit" : "20220411191603",
"numWrites" : 122486,
"numDeletes" : 0,
"numUpdateWrites" : 122457,
"numInserts" : 0,
"totalWriteBytes" : 7528604,
"totalWriteErrors" : 0,
"tempPath" : null,
"partitionPath" : "cluster=96/shard=14377",
"totalLogRecords" : 846489,
"totalLogFilesCompacted" : 7,
"totalLogSizeCompacted" : 325539587,
"totalUpdatedRecordsCompacted" : 122457,
"totalLogBlocks" : 7,
"totalCorruptLogBlock" : 0,
"totalRollbackBlocks" : 0,
"fileSizeInBytes" : 7528604,
"minEventTime" : null,
"maxEventTime" : null

"fileIdAndRelativePaths" : {
    "4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0" : "cluster=96/shard=14377/4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet",
    "0139f10d-7a88-481b-b5df-6516500076b0-0" : "cluster=96/shard=14377/0139f10d-7a88-481b-b5df-6516500076b0-0_0-1177-14258_20220411202305.parquet",
    "21000940-a573-4c46-8ad5-79003ac9daf5-0" : "cluster=96/shard=14377/21000940-a573-4c46-8ad5-79003ac9daf5-0_2-1177-14260_20220411202305.parquet"
  },
"totalRecordsDeleted" : 0,
"totalLogRecordsCompacted" : 2543875,
"totalLogFilesCompacted" : 21,
"totalCompactedRecordsUpdated" : 368137,
"totalLogFilesSize" : 978467842,
"totalScanTime" : 45421,

Now when I check the file system view with the command show fsview latest Hudi shows the new file but not deleted

║ Partition               │ FileId                                 │ Base-Instant   │ Data-File                                               │ Data-File Size │ Num Delta Files │ Total Delta Size │ Delta Size - compaction scheduled │ Delta Size - compaction unscheduled │ Delta To Base Ratio - compaction scheduled │ Delta To Base Ratio - compaction unscheduled │ Delta Files - compaction scheduled                                                                           │ Delta Files - compaction unscheduled ║

║ cluster=96/shard=14377/ │ 4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0 │<PHONE_NUMBER>2305 │ s3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1198-14280_20220411202305.parquet │ 7.2 MB         │ 4               │ 178.2 MB         │ 178.2 MB                          │ 0.0 B                               │ 24.814273985207016         │ 0.0                                          │ [HoodieLogFile{pathStr='s3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/.4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_20220411202305.log.4_0-1758-20808', fileLen=46785516}, HoodieLogFile{pathStr='s3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/.4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_20220411202305.log.3_1-1610-19092', fileLen=46440363}, HoodieLogFile{pathStr='s3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/.4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_20220411202305.log.2_0-1450-17202', fileLen=47068201}, HoodieLogFile{pathStr='s3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/.4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_20220411202305.log.1_0-1297-15405', fileLen=46545393}] │ []

Tried options

Turn off File Sizing by setting hoodie.parquet.small.file.limit to 0 to make sure the file is not deleted
Setting hoodie.datasource.compaction.async.enable to false
With one table the inline compaction is working as expected. Cleaning is also triggered and does what it should
Code has been rewritten to not use foreachBatch
Even though we are using Single Writer (single Spark application) we tried also optimistic_concurrency_control. The concurrent access is not detected and the issue still persists

Possibly Related Issues

https://issues.apache.org/jira/browse/HUDI-3370. This one describes the similar symptoms but we do not use metadata table service
Comment https://github.com/apache/hudi/issues/2151#issuecomment-706029681 that implies that when using spark.write() the inline compaction should be off. Not yet found any restrictions in the source code for that matter
https://github.com/apache/hudi/issues/2043#issuecomment-682100271. Seems like using foreachBatch might not be a good approach to begin with
https://github.com/apache/hudi/issues/2771#issuecomment-814682374 Not sure if having inline compaction and async compaction might be the root cause
Another issue with compaction when using foreachBatch approach https://github.com/apache/hudi/issues/4839#issuecomment-1046839752
Time line server might be a root cause as well https://issues.apache.org/jira/browse/HUDI-2860 and https://issues.apache.org/jira/browse/HUDI-1138

Stacktrace
Lost task 0.0 in stage 1.0 (TID 1) (ip.ec2.internal executor 2): java.io.FileNotFoundException: No such file or directory 's3://some-bucket/landing-zone/some_table/cluster=96/shard=14377/4ce009e6-5622-4874-bb5d-a11e3bb9eaa3-0_1-1177-14259_20220411202305.parquet'
	at com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem.getFileStatus(S3NativeFileSystem.java:521)
	at com.amazon.ws.emr.hadoop.fs.EmrFileSystem.getFileStatus(EmrFileSystem.java:694)
	at org.apache.parquet.hadoop.util.HadoopInputFile.fromPath(HadoopInputFile.java:61)
	at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:456)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.footerFileMetaData$lzycompute$1(ParquetFileFormat.scala:318)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.footerFileMetaData$1(ParquetFileFormat.scala:317)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat.$anonfun$buildReaderWithPartitionValues$2(ParquetFileFormat.scala:319)
	at org.apache.hudi.HoodieMergeOnReadRDD.read(HoodieMergeOnReadRDD.scala:105)
	at org.apache.hudi.HoodieMergeOnReadRDD.compute(HoodieMergeOnReadRDD.scala:77)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

After changing the code and removing foreachBatch we were able to fix the issue https://github.com/apache/hudi/issues/2043#issuecomment-682100271. However, now the issue is reproducible for both inline and async variants of compaction.
I have updated the section Main observations so far.

Upon further investigation and after enabling additional logs on EMR, the deletion of the file during compaction is happening in the class org.apache.hudi.table.HoodieTable#reconcileAgainstMarkers
if (!invalidDataPaths.isEmpty()) {
        LOG.info("Removing duplicate data files created due to spark retries before committing. Paths=" + invalidDataPaths);`

However, later in the logs this file is written and commited in the instant
INFO SparkRDDWriteClient: Committing Compaction<PHONE_NUMBER>2316. Finished with result HoodieCommitMetadata{partitionToWriteStats={cluster=96/shard=14377=[HoodieWriteStat{fileId='9d9f72e9-9381-40d0-af0c-cb48c25bd78d-0', path='cluster=96/shard=14377/9d9f72e9-9381-40d0-af0c-cb48c25bd78d-0_0-617-7132_20220414232316.parquet', prevCommit='20220414225217', numWrites=122886, numDeletes=0, numUpdateWrites=121939, totalWriteBytes=23331178, totalWriteErrors=0, tempPath='null', partitionPath='cluster=96/shard=14377', totalLogRecords=341027, totalLogFilesCompacted=3, totalLogSizeCompacted=285373803, totalUpdatedRecordsCompacted=121939, totalLogBlocks=9, totalCorruptLogBlock=0, totalRollbackBlocks=0}]}, compacted=true,

So it leaves the system in an inconsistent state. It looks like some concurrency issues to me
I will try to submit multiple StreamingQuery in different threads by leveraging spark scheduling pool. Will update about the status

@kasured : before I dive in, few pointers on the write configs used.

I see you have enabled both inline and async compaction. Guess w/ streaming sink to hudi, only async compaction is possible and for MOR table, hudi automatically does async compaction. So, probably you can remove these configs.

"hoodie.compact.inline" = "true"
 "hoodie.datasource.compaction.async.enable" = "true"

and I also see you have enabled clustering. can we disable clustering and see if the issue is still reproducible.

with these changes, can you let us know if the problem still persists?

@nsivabalan Thank you for looking into that. I have updated the configuration in the description as it was a little out of date. Since the creation of the ticket you can see that I have tried multiple options.

At first iteration I had foreachBatch which was not causing async compaction to happen (please see the linked issues). After the code was rewritten to use just structured streaming constructs async compaction started to be scheduled and executed. So I have tried both inline enabled with async disabled, and vice versa and the issue that I describe is reproduced in both cases
Not sure what you mean as I have cluster disabled explicitly with "hoodie.clustering.inline" = "false". And also I have not seen any clustering actions neither in the .hoodie nor in the logs

All in all please check these two sections Main observations so far and Tried Options. They are up to date and have the summary of all that I have tried so far

yes, I really appreciate your digging in deeper.
let me try to understand the concurrency here.
what do you mean by multiple concurrent streaming writes? there are 3 streams reading from diff upstream sources and writing to 1 hudi table? or one streaming pipeline which writes to 3 different hudi tables? or 3 different streaming pipeline writing to 3 diff hudi table but using same spark session ?

btw, we did fix an issue wrt how spark lazy initialization and caching of results could result in wrong files in commit metadata https://github.com/apache/hudi/pull/4753. looks like exactly matching what you are reporting.
Can you try applying the patch and let us know if you still see the issue.

@nsivabalan Sure, let me provide more details. There is a StreamingQuery entity which s started by Spark to consume the stream. This is basically what we use and described here https://hudi.apache.org/docs/compaction#spark-structured-streaming
So what we do is we create multiple StreamingQuery streams and start them. Each of them though consumes from single kafka topic and writes to single Hudi table. So it is 3 different streaming pipeline writing to 3 diff hudi table but using same spark session with the only exception that we use 3 different SparkSession objects. Each of them are reusing single sparkContext which is okay as there should be only one spark context per jvm.
As to 4753 I have already specified it in the section Possibly Related Issues HUDI-3370. However, from what I checked it is related to metadata service which we do not use "hoodie.metadata.enable" = "false". May it also be relevant even if we do not use metadata table? I am asking cause we are using 0.9.0 from Amazon and I will need to replace it with the one with patch

its not related to metadata table as such. essentially, the actual data files as part of the compaction commit could be different from what is found in compaction commit metadata. So, when reconciling markers, we may delete unintended files.
yes, it is applicable even if you don't enable metadata.
and thanks for clarifying your use-case. I get it now.

I can see that the fix version is 0.11.0. Can this patch be safely backported to 0.9.0 though?

@kasured I am from AWS EMR team. I hope you have opened a ticket with AWS support and we can work through that channel to backport this fix and provide you patched jars.

@kasured is it possible for you to have 3 separete spark applications that do not share spark context ?

@nsivabalan We were able to reproduce the similar scenario locally. Please, use the following repository to check and confirm on your end https://github.com/kasured/hudi-compaction-5298

@kasured Im currently looking into this and was able to reproduce the issue after running your https://github.com/kasured/hudi-compaction-5298.
When I built hudi 0.11.0 (from master) with profile spark-3.1.2 and provided my 0.11.0 spark bundle jar to your sample repro I noticed that I did not get the FileNotFoundException. Im currently looking to see if the fix https://github.com/apache/hudi/pull/4753 can be backported to 0.9.0

@kasured If you have opened a case with AWS EMR support, we have a backport of the fix for hudi 0.9.0 we can provide you. Let us know so we can close this thread out for now.

Closing this issue as its already been fixed. thanks for raising the issue.

Closing this issue as its already been fixed. thanks for raising the issue.

Hi @nsivabalan can you please help to solve the above issue?
I am facing the same problem.


Add service and service template models for AnsibleTower

Added ServiceTemplateAnsibleTower and ServiceAnsibleTower and their automate models.
automate namespace and state machine will be in another PR.

@gmcculloug @mkanoor please review

@miq-bot remove_label wip

@miq-bot rm_label wip

@gmcculloug @mkanoor please review

@gmcculloug I updated the service provisioning entry point. Please review.


Validation issue with +48 967 753 358 (GB)

Please read the "guidelines for contributing" (linked above) and fill in the template below.
Country/region affected (e.g., "US"): GB
Example number(s) affected ("+1<PHONE_NUMBER>"): +48 967 753 358
The phone number range(s) to which the issue applies ("+1 555 555-XXXX"):
The type of the number(s) ("fixed-line", "mobile", "short code", etc.):
The cost, if applicable ("toll-free", "premium rate", "shared cost"):
Supporting evidence (for example, national numbering plan, announcement from mobile carrier, news article): IMPORTANT - anything posted here is made public. Read the guidelines first!
link to demo

Duplicate of https://github.com/googlei18n/libphonenumber/issues/1314


use jsDelivr CDN, update jQuery & fancybox versions

I'm a bit biased since I volunteer for jsDelivr :)  Here are the advantages vs ajax.googleapis.com

can collate all JS or CSS into single file & HTTP request (slightly better net compression also).  The combined file is cached in the POP after 5+ calls.
much better performance in China

The chance of a browser's cache has //ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js is very slim.
Works with my site, but I don't use jQuery of anything else by FancyBox.
jsDelivr also allows version auto-updating, but I personally don't recommend that past minor updates.  (eg jquery<EMAIL_ADDRESS>will serve jQuery 2.1.5 & FancyBox v2.1.9 if/when available.)
According to my brief tests here in Denver, speed is about the same for a website I'm working on, even though Google has a POP inside my city, & jsDelivr does not yet (will sometime).  Could be a more improvement gap if I maxed out origin HTTP requests (eg had lots of pictures).
Thank you for all your hard work!

I just noticed Grunt downloads FancyBox.  Let me know if you want me to amend this PR to eliminate that.

I merge the patch for font loading, and other patch will be better to be included in a jsDelivr landscape.
Maybe you can fork this repo or provide an option for user, and if option given, users may have more choice for CDN, such as cdnjs and so on.
BTW, jsDelivr look good, I do like your work.

fork this repo or provide an option for user

I prefer the idea of jsDeliver CDN or origin/local hosting if you want me to add a switch.  Though I think the switch should exist in {project root}, so I will want to add a PR in hexojs/hexo.
cdnjs is good, but jsDelivr better since both use CloudFlare, but jsDelivr uses 3 other CDNs that are multiplexed into a mega-CDN.  jsDelivr is in countries cdnjs is not, like Russia, Turkey, Israel...  Plus has redundant POPs in many cities.
So I don't know an advantage of using another CDN

I'm beginning to think that external tools would be the most flexible & powerful solution.
https://www.npmjs.com/package/gulp-cdnizer


migrate style check to precommit

Issue #, if available: #2637 partially
Description of changes: moving some style checks to common pre-commit, which can be efficiently run by the developer locally and also for PR runs only on the diff, not the full project, so the iteration shall be much faster... In addition, a bot can be installed to this repository that may send some automated fixes related to the open PR so a developer will not think about styles too much :flamingo:
BTW, it would be good to migrate also the just license so the whole workflow can be dropped
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
cc: @jaheba @kashif @lostella kind ling as it seems there are no revivers by default :chipmunk:
Please tag this pr with at least one of these labels to make our release process faster: BREAKING, new feature, bug fix, other change, dev setup

@Borda the style check workflow is pretty quick compared to tests, I think?

the style check workflow is pretty quick compared to tests, I think?

That is true, but with pre-commit, you can run it locally and only on changed files compared to relying on validation after pushing or calling several commands as it is in the job on whole codebase

@Borda  lets move to ruff instead of black?

@Borda lets move to ruff instead of black? for fix and format

Sure, happy to do it.
Just thought to make this 1:1 chance and additional improvement in next PR, what do you think @kashif?

sounds good!

@jaheba @lostella is there anything else I could do for this PR? :chipmunk:

@Borda I think it's fine to have pre-commit hooks, but maybe we leave the github workflows untouched for now? We can always remove them later if we believe it's the way to go. But I think having the workflow active in the repo is a clear reference on what checks the code is expected to pass.

maybe we leave the github workflows untouched for now?

Sure, we'll put them back, just if I may suggest, lest mark them as not required and make the pre-commit bot required; what do you think?

@lostella noticed that probably lints did not have a fixed version, so it could run anytime in the future with different changes... so the precommit config fixes that and guarantees reproducibility...
anyway is it fine if I run the formating and fix issues as part of this PR?

@lostella, this is one now ready for review too :flags:

@lostella friendly ping :flamingo:

@lostella friendly ping :chipmunk:

@lostella seems all is :green_circle: now :tada:

@lostella seems all is 🟢 now 🎉

mind have a look, when you have time 🦩

@lostella could you pls have a look? 🐿

@lostella seems GH changed roles for CI; would you mind approving this PR or adding me there...

@lostella could you pls have a look? 🐿

seems GH changed roles for CI; would you mind approving this PR or adding me there...

@lostella could you pls have a look? 🐿

seems GH changed roles for CI; would you mind approving this PR or adding me there...

@lostella let me know if I should resolve the conflicts for merging...


Simplify TestStep API

 Issue by marcoguerri
Wednesday Feb 05, 2020 at 12:51 GMT
Originally opened as https://github.com/facebookincubator/contest/issues/23

I have been thinking of how to implement single target cancellation for two reasons:

It's a feature we might want to support in the future
I believed it would be necessary when losing locks on the targets. However, after reading again the implementation based on Zookeeper, if things go wrong, we'll lose the connection pool to the whole ensemble. So we won't lose per-target-locks, but we'll lose all locks: we'll need to cancel the whole job (which we can do now).

Even though number 2) doesn't really apply, I had some ideas on how to simplify the API for TestStep. There is now a good amount of duplication. Taking a sample TestStep as example:
func (ts *SSHCmd) Run(cancel, pause <-chan struct{}, ch teststep.TestStepChannels, params teststep.TestStepParameters, ev eventmanager.Emitter) error {
        if err := ts.validateAndPopulate(params); err != nil {
                return err
        }

        f := func(cancel, pause <-chan struct{}, target *target.Target) error {
              // DO WORK
        }
        return teststeps.ForEachTarget(Name, cancel, pause, ch, f)

Validate Params
Define the function object
Call ForEachTarget

I would divide the TestStep API into high level API (supporting ForEachTarget and ForAllTarget semantics), and low level API, which uses directly the channels and can be used if more complex Target aggregations are necessary. Most of the use cases should be covered by the high level API.
With the high level API, the plugin would not implement the TestStep interface, but would define function objects and parameters that would be used to instantiate a generic SingleTargetTestStep struct. For example, myplugin.go would do something as follows:
var pluginName = "myplugin"

func run(cancel, pause <-chan struct{}, target *target.Target) error {
    // Do work
}

func resume(cancel, pause <-chan struct{}, target *target.Target) error {
    // Do work
}

func NewMyPlugin() TestStep {
    return teststep.NewSingleTargetTestStep{
         Run: run,
         Resume: resume,
         Name: pluginName
   }
}

How does this help?

It makes it easier to implement the API, as it removes the duplicated code coming from having to define the TestStep struct, call validateAndPopulate, call ForEachTarget, etc. A possible "regression" is that validateAndPopulate will change semantics, it will only be validate, there won't be anything to populate. I think it's fine, and it's actually in line with the direction we have taken to offload parameter expansion to the Params object.
It will move the responsibility to schedule each Target into a TestStep instance to the TestRunner: this will allow per-Target control.
It allows to implement the ForAllTargets semantics in the same way, also reducing at minimum the duplication
It will still allow to use the "low level API", i.e. implementing directly the TestStep interface and use the channels directly.

can be included in #135
will review.


keepalived-vip using alpine package

The keepalived-vip is building keepalived from source. Since some days keepalived is available in apline testing (https://pkgs.alpinelinux.org/package/testing/x86_64/keepalived). I suppose it has not been availablel when keealived-vip was built. Are there any reasons not to use the repository package? If there are please document them ;)

I suppose it has not been availablel when keealived-vip was built

Exactly.


Specialized C containers for known constraints

Traditionally containers are designed to be good at typical usage patterns. The Python dictionary and the Python list are well optimized for what code typically needs. However, in order to shave of the maximum performance, this involves trade offs, that are not ideal for all the cases and we might be better off with other approaches in some cases.
Consider this code:
def f():
   result = []

   for value in unknown_iterator_size():
         result.append(something_unknown(value))

   return sum(result)

This is a very common usage pattern in Python code. The list is purely local. It doesn't escape. Unfortunately we cannot know the size of the list we are talking here. We could achieve in this particular instance, and earlier sum, but lets assume that is not done, and it's very high levels optimization needed to get us there, and sum is not the issue here.
The point is rather, that PyList_New() Python C/API is going to have to be called, to create a list right now. That does malloc an object, and inside of it, a storage space, both of which will later have to be released with free. For every append we have to consider that the list storage space might have to be expanded, which is not every time, but leads to a couple more malloc during PyList_Append Python C/API.
Imagine, we control the sum well enough to know it is going to make only start to finish value iteration, and nothing more besides this. Imagine we create a container, that is only good for append, in fact need not provide any other operation at all, or at least doesn't need those to be fast, as e.g. they are done only once, e.g. len(), the equivalent of PyList_Size would be allowed to make an actual calculation, and not be a stored value in the structure, it would e.g. only contain a pointer to the end.
Imagine further that we keep storage spaces pre-allocated for non-overlapping lists, that are deemed sufficiently large to hold the list information and relatively many elements, and the container would have bounds checking only, but normally no growing needed.
CPython, not knowing the usage of the list, has to cover the case of many small lists with its implementation, cannot allocate too much storage in the beginning, and wants to limit the over allocation of lists, but we can start out relatively big for this local list, no worry.
Avoiding the malloc entirely, will do us good already.
Obviously, we also want the container to also be more flexible, and store other values than PyObject *, suppose (not the case right now), we knew that it was a list of float values. We would want to store C double instead of the PyObject * for as long as possible, and let sum know, that is it, what it's working on, and be more direct.
Avoiding these PyObject * boxes for the values, would be a even larger memory saving measure, and one that is not currently possible without specialized containers.
So what this would need is a implementation of Nuitka_LocalListFlavorXYZ_New() (the flavor indicates the constraints accepted, e.g. AppendValueIterationOnly) with variants of _Append and _Iter (iterator creation), _Next (next value), that work as described.

For the start, this is pure C task, where you would create small C program that does similar to what the Python function above is doing, using the Python C/API. And then an alternative code, using your flavored API container, that during new, only sets the append pointer to the beginning, in append then writes through it, and checks for overflow.
For sum() you would use the Python C/API manually in the original version to create an iterator over the PyList with PyList_Iter and go over it, not even doing the actual sum, just to see the performance of that. And the flavored API would allow for the same, but e.g. likely has a built-in iteration pointer, that makes this fast.
Again, the iterator will be a doing malloc/free in Python C/API (do not forget to PyObject_Del the things). For the values put into the container, feel free to use a constant value like Py_None, we do not want to test any operation outside of this.
Then measure the performance difference and report.

Knowing the size ahead of time should allow a flavor without even bound checks.
In case of overflow, the container should indicate that with a return value, and Nuitka would treat this in the using code, by having a alternative type, that is either the flavored list, or the PyList pointer, doing what it is going to do with int in what is going to be experimental Nuitka code by the summer.

This means, giving up, and switching to a fallback implementation, is a task that we would deal on the outside with. Nuitka could then switch the flavor of the list on the fly, or go back to the most general form, whatever.

@girijamanoj this is something we discussed, give it a try?

Sure @kayhayen, Thanks a lot :) for giving detailed information.

Hello @kayhayen , i am Dharmraj, i would like to contribute  to Nuitka for GSOC 2019. Thank you for detailed description for this issue. can i go with this issue?

@kayhayen rightnow, i am getting familiar with nuitka, after successfully  build a small program, i am trying to run the .exe file from .dist file but it gives me a error that application was unable to start correctly(0xc000007b)

@DMRathod Did you try the develop branch of Nuitka too. But it seems you have an issue to create potentially, go ahead and do that, filling out the issue template will help me solve it, or help me help you solve it. :)

@kayhayen what if we create Linkedlist?

@sannanansari these are very hard to get thread safe, which is a downside should be ever start to use that, and they involve memory allocation, don't they? And that is precisely the main overhead we aim to avoid.
The main advantage of linked lists would be that insertion at not the end and removal can both be quicker and releases memory sooner, than the vector that PyList_Type holds. But that would only show for large lists, but then, instead of 3 values, you store 3 nodes, which contain 2 pointers and 1 value, i.e. use up 3 times as much memory.
Should your allocation be dumb, i.e. from the end of big memory only, getting to maintain these pointers seems hard, and lets not forget, in Python, all removals will be centered around the value, or the index, and insertions too, around the index. Insertion at element 7 then will require you to search it first, and I am doubting if an memcpy of the remaining elements after a fixed offset, may not be on average faster.
And as I said, the majority of lists is append only, in which case a linked list will give clearly no benefit at all.

What if we take input from user at terminal. Just in case.

@sannanansari I prefer static examples for reproducible results

@kayhayen there is something that i don't understand here will we use a preallocated big space in the heap that we will distribute them on our lists in a smart way using our container or we will build our local lists space in global memory or not that or that

@omarahmed1111 The idea is that compile time analysis puts boundaries on list size, and life time. Lists that cannot be used at the same time, will using the same memory. Lists that are bounded in size, but live at the same time, can use different portions. This is an analysis that Nuitka will have in the future.
I do not understand what you mean with global memory vs. heap memory. We are definitely not talking about stack memory though.

@kayhayen i mean that we want to avoid malloc and we want to assign a space for our lists so will we use malloc with a big size one time or as little times as we could and don't free it and we will use this memory in our lists as they don't live at the same time and free this memory at the end


Fix to prevent 'strict mode does not allow assignment to unassigned variables' error

I ran into this issue the other day when setting up a new bundling / minifying system for our project:

Strict-mode does not allow assignment to undefined variables: $

Found someone with a similar issue here:
https://github.com/SortableJS/Sortable/issues/1004
And the resulting fix in Sortable.js was like so:
https://github.com/SortableJS/Sortable/blob/master/Sortable.js#L19
Tried the same in Intercooler and voila - it worked! Thought I'd share so this doesn't trip anyone else up in a similar situation. Thanks!

Thanks for the fix. Could you rebase this single commit onto development instead of master? I tried just changing the target of the pull request, but it sucked a bunch of unrelated commits into the pull request when I did.

@jsampson sure thing! thought rebasing was the way to do that, but that appears to have messed things up - I'll close this and create a new pull request based off of dev per your request.


Use 'Headers' in recorded response.

Please review closely. The tests should be verifying behavior but we want to make sure semantics don't change as this no longer does any regex-based validation. I'm confident it doesn't.
A guide:
assertTrue/assertContains --> assertEquals
assertFalse/assertNotContains --> assertNull


There shall be only one active running harvesting task per csw

Environment

Python version: 3.7
MrMap version: pre 1.0.0

Proposed Functionality

There shall be only one active running harvesting task per csw.
IF the async celery task is canceled, the running task which fills the HarvestResult is newer running again and the HarvestResult shall also be marked as canceled.

Use Case

Harvesting jobs are very long running jobs. So if the user starts two harvesting tasks at the same time, the server is busy for while for a non sense job.

Database Changes

add a unique_together constraint for field metadata and timestamp_end. So it is not possible to add two HarvestResult objects for the same metadata without setting the timestamp end explicit.
check on HarvestResult.save() if there is an orphaned HarvestResult without running celery task. --> delete them

Unique together constraints raises form errors, which is the best practice in django, cause we doesn't need to implement own code logic in our forms/views.

External Dependencies
None

done with  #404


twig does not recognize vite directives

I have a problem, the twig view does not recognize the vite directives
{% block stylesheets %}
{{ vite_entry_link_tags('app') }}
{% endblock %}

{% block javascripts %}
{{ vite_entry_script_tags('app') }}
{% endblock %}

When I inspect in the browser, the app.js and app.css files are not found, either when I do the build or in development.
I attach my vite configuration file
I attach my vite configuration file
import { defineConfig } from "vite";
import { viteStaticCopy } from "vite-plugin-static-copy";
import path from "path";
import symfonyPlugin from "vite-plugin-symfony";

export default defineConfig({
  plugins: [
    symfonyPlugin(),
    viteStaticCopy({
      targets: [
        {
          src: 'assets/images',
          dest: 'bundles/'
        },
        {
          src: 'assets/fonts',
          dest: 'bundles/'
        },
        {
          src: 'assets/webfonts',
          dest: 'bundles/'
        },
        {
            src: 'assets/index.php',
            dest: ''
        }
      ]
  ]},
  build: {
    outDir: "public",
    assetsDir: "bundles",
    rollupOptions: {
      input: {
        app: "./assets/app.js",
      }
    },
  },
});

hi @dj-Andres,
can you please open your issue in the principal repository : https://github.com/lhapaipai/symfony-vite-dev/issues like mentionned in the template : https://github.com/lhapaipai/vite-bundle/issues/new
The issues that are still present here were opened before I implemented the template, that's why they were not closed.
Thanks


Eception in published NET6-app

Unfortunately I still have a problem very similar to issue #12. This time it works fine in Visual Studio using SharpHook 3.0.1, but when we deploy the app on a client it crashes I'm a afraid. I attached some screen captures from Windows Event Viewer. We publish as a self contained .NET6 application.
I tested to use the test project I sent you in #12 and published it as a self contained app and could reproduce it. So if you use that testproject and Publish using the settings from the attached VS screen capture I hope you can reproduce it. Otherwise just let me know if there's anything else you need from me!
Cheers!

.

Hi! Thanks for posting this issue! I will look into it shortly.

Yup, just published your test app myself. It doesn't work for x86 (with the same error as you described), but it does work for x64. So I have a question - do you really need x86 support? If not, then publish the app for x64, and there'll be no issue.
I'm closing this issue as a duplicate of #10. If you want to follow up on this problem, then let's move the discussion to that issue.

Version 3.0.2 is released and this issue is fixed there.

Awesome. Thanks!


M5157: Incomplete sentence: "missed help for access show. adding it."

@routlaw,
Could you review this and reply, please? Many thanks in advance.

@carolinacmoravia,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Acrolinx Scorecards
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard

docs-ref-conceptual/release-notes-azure-cli.md
72
34
link

More info about Acrolinx

Install Acrolinx locally for VSCode

The scorecard links are good for one hour. If you're prompted to authenticate, you can close and reopen the PR to get new links. Or, use these credentials:
Account: report-viewer
Password: kermit$

False positives or issues

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/release-notes-azure-cli.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@routlaw,
Could you review this and reply, please? Many thanks in advance.

@rloutlaw - Required localization fix. Should create a VSTS bug for it to track if necessary.

@routlaw,
Would you kindly help to review this, please?
Many thanks in advance.

@routlaw,
Would you kindly help to review this, please?
Many thanks in advance.

@routlaw,
Verified that reported issues still reproduces on current version of this file. Would you kindly help to review this, please?
Thank you.
Cc: @SunnyDeng

@carolinacmoravia - Please file a VSTS issue internally to resolve this. Release notes for previous versions are normally not updated. Closing this PR as it is a bug report.


Update documentation for the annotation tools

cc @paulhfu : this updates some of the function names for things you have added, so that they are excluded from the documentation.
I don't think that interferes with any of your changes in #91, but still wanted to ping you here as a heads up.

Thanks for the heads up @constantinpape. #91 is almost finished and there are no conflicts. Good to see this refactoring in any case. From now on I pay more attention to privacy markers and type hints.


Event handlers not cleared up when clearing a layergroup

http://playground-leaflet.rhcloud.com/gog/1/edit?html,console,output
Drag a map few times.
Open console.
Execution time increases almost each time you drag a map.

Reproducible in rc3.
Appears after this commit: https://github.com/Leaflet/Leaflet/commit/8047b0b7a99ab1bed75bab7cca9c1c3a0601da87

@yohanboniface Some event listeners are not being cleared up properly after that commit. Under the circumstances of the reproducible example provided by @LuSilf, they create both a memory leak and a slowdown (by being fired again?) somehow.

So, the culprit is this line: https://github.com/Leaflet/Leaflet/commit/8047b0b7a99ab1bed75bab7cca9c1c3a0601da87#diff-0e79846ee2c407aa1ac0ccbfd8b15a44R168
We copy the listeners to a new array, but never actually assign that copy to the _events array. Instead, we never clear up listeners, but it still works since all the listeners are assigned to the falseFn. It only shows up if you have a lot of listeners. So... good catch, @LuSilf!
I'll try to make a fix shortly.

And good investigation @perliedman :)

@yohanboniface thanks :smile: Trying to figure out how to test this without digging into the internals...


Added RunwayML API

I have added RunwayML Api to the Readme Section for the issue #89

RunwayML & Runway.team are two different companies offering very different services.


feat(kit): Multiselect add ability to use objects in native select

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Refactoring
[ ] Build or CI related changes
[ ] Tests related changes
[ ] Documentation content changes

What is the current behavior?
Closes #4010
What is the new behavior?

argus bot didn't attach screenshots


I can't get it to work with Neovim

This happens with diff files too.
Here and example:

checkhealth returns this for gitcommit:
nvim-treesitter: require("nvim-treesitter.health").check()

Installation ~
- OK `tree-sitter` found 0.20.7 (parser generator, only needed for :TSInstallFromGrammar)
- OK `node` found v19.3.0 (only needed for :TSInstallFromGrammar)
- OK `git` executable found.
- OK `cc` executable found. Selected from { vim.NIL, "cc", "gcc", "clang", "cl", "zig" }
  Version: Apple clang version 13.1.6 (clang-13<IP_ADDRESS>.5)
- OK Neovim was compiled with tree-sitter runtime ABI version 14 (required >=13). Parsers must be compatible with runtime ABI.

OS Info:
{
  machine = "arm64",
  release = "21.5.0",
  sysname = "Darwin",
  version = "Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000"
} ~

Parser/Features         H L F I J
  - gitcommit           ✓ . . . ✓

What are you using for queries?
I don't use neovim so you may have better luck opening this against the nvim-treesitter repo

True, I will add the issue there, thanks!

Hey @marcelarie I have had the same symptom for months on my M1 laptop. Did you ever find a solution, or open an issue in the nvim-treesitter repo, that I could follow?

you have more info here https://github.com/nvim-treesitter/nvim-treesitter/issues/1854 :)


Add tech preview shortcode And make the matching body payload a tech preview

And make the matching body payload a tech preview. Explains why the behaviour may be erratics on the matching body payload with the gitops comments.

Changes

Submitter Checklist

[ ] 📝 A good commit message is important for other reviewers to understand the context of your change. Please refer to How to Write a Git Commit Message for more details how to write beautiful commit messages. We rather have the commit message in the PR body and the commit message instead of an external website.
[ ] ♽ Run make test before submitting a PR (ie: with pre-commit, no need to waste CPU cycle on CI. (or even better install pre-commit and do pre-commit install in the root of this repo).
[ ] ✨ We heavily rely on linters to get our code clean and consistent, please ensure that you have run make lint before submitting a PR. The markdownlint error can get usually fixed by running make fix-markdownlint (make sure it's installed first)
[ ] 📖 If you are adding a user facing feature or make a change of the behavior, please verify that you have documented it
[ ] 🧪 100% coverage is not a target but most of the time we would rather have a unit test if you make a code change.
[ ] 🎁 If that's something that is possible to do please ensure to check if we can add a e2e test.
[ ] 🔎 If there is a flakiness in the CI tests then don't necessary ignore it, better get the flakyness fixed before merging or if that's not possible there is a good reason to bypass it. (token rate limitation may be a good reason to skip).

Can we add annotation support feature as well beta ?

Merging this now if require you can send another PR with doc changes


Optional + Nullable cannot be explicitly set to null

Serialization writes this
            if (OptionalProperty.IsDefined(BestOf))
            {
                if (BestOf != null)
                {
                    writer.WritePropertyName("best_of"u8);
                    writer.WriteNumberValue(BestOf.Value);
                }
                else
                {
                    writer.WriteNull("best_of");
                }
            }

Basically the WriteNull is unreachable code.

Yeah - this is a shortage of our current design.
We could never distinguish whether a property is explicitly set to null by our customer, or it is null when constructed and no one has ever touched it.
In the partial solution of patch operation, we could do that by adding something in the set method. If we would like to solve this, we could apply that to all the models.


Resize listeners in iframe causing DOM mutations

Hi,
I am wondering if your library can handle the following problem:
A website in the iframe may set a resize listener that triggers a DOM mutation. For example on resize set the height of the html element to 100% or auto depending on the height of the window.
In my current project I am currently keeping an iframe sized to its contents in a similar, but probably less sophisticated, way as this library, but I am running into the above problem and I am curious if using your library can solve it.
My current solution watches the dom in the iframe with a mutation observer. The iframe's height is then set via a double resize, setting height first to auto, then calculating the height of the iframe contents, then setting the iframe's height to that calculated height. This leads to an infinite loop if the website in the iframe has a resize listener that causes a DOM mutation. The dom mutation will trigger the calculation of the iframe height, which resizes the iframe, which triggers the resize event, which causes a DOM mutation etc etc.

Interesting question, it is possible to create a loop by setting the html
element to 101% of the size of the window. However, what you’re talking
about is a different problem.
The short answer is I’m not sure, as no one has asked this before. The
longer answer is look at all the different options for working out the
height, by using taggedElement you might workaround this by working out the
height on an element un effected by the element who’s CSS you are playing
with. Also the tolerance option might help as well.
The lib also has some limited debouncing of events that might help you.
So the best can say is have a play and that I would be interested to know
if it does solve you’re problem.
On Tue, Nov 1, 2016 at 3:16 PM, Joeri de Gooijer<EMAIL_ADDRESS>wrote:

Hi,
I am wondering if your library can handle the following problem:
A website in the iframe may set a resize listener that triggers a DOM
mutation. For example on resize set the height of the html element to 100%
or auto depending on the height of the window.
In my current project I am currently keeping an iframe sized to its
contents in a similar, but probably less sophisticated, way as this
library, but I am running into the above problem and I am curious if using
your library can solve it.
My current solution watches the dom in the iframe with a mutation
observer. The iframe's height is then set via a double resize, setting
height first to auto, then calculating the height of the iframe contents,
then setting the iframe's height to that calculated height. This leads to
an infinite loop if the website in the iframe has a resize listener that
causes a DOM mutation. The dom mutation will trigger the calculation of the
iframe height, which resizes the iframe, which triggers the resize event,
which causes a DOM mutation etc etc.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/davidjbradshaw/iframe-resizer/issues/422, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAkHEs2w4mkK8v_5R99wFeTYObJ7nmMmks5q51fJgaJpZM4KmNA2
.

 I'll let you know what I find out.

So yea, I can prevent that loop by using the lowestElement height calculation method.
With this method you wont resize the iframe every time a dom mutation hits, only if the height is actually different from before the dom mutation, which is totally fine of course. So I gues I'll be implementing your library now! :) thanks for all the work


Update to Puppet 5 and Hiera 5

Opening PR to update this module to Puppet 5 and Hiera 5 specs. This branch is a work-in-progress and not ready merge at this time.

For now, you will need to modify the Rakefile so that
exclude_paths = [
  "pkg/**/*",
  "vendor/**/*",
  "spec/**/*",
]

also contains "plans/**/*", from what I have gathered.

@thespain is there more to do here or is it ready?


Making kpcovr test deterministic by using  svd_solver

In response to #100, removing non-deterministic behavior from the kpcovr_test

What issues do you see with self.Y?

On Apr 20, 2021, at 4:08 PM, hurricane642 @.***> wrote:
@hurricane642 commented on this pull request.
In tests/test_kpcovr.py https://github.com/cosmo-epfl/scikit-cosmo/pull/104#discussion_r616718619:

@@ -25,7 +25,9 @@ def init(self, *args, **kwargs):
self.X = SFS().fit_transform(self.X)
self.Y = SFS(column_wise=True).fit_transform(self.Y)

   self.model = lambda mixing=0.5, **kwargs: KPCovR(mixing, alpha=1e-8, **kwargs)

   self.model = lambda mixing=0.5, **kwargs: KPCovR(

       mixing, alpha=1e-8, svd_solver=kwargs.pop("svd_solver", "full"), **kwargs

   )

Is this really enough to fix the problem? I'm just afraid that there are difficulties because of the way self.Y is created.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/cosmo-epfl/scikit-cosmo/pull/104#pullrequestreview-639998120, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALKVP3WDGZ2MH4IWUY7ALL3TJWDEPANCNFSM43IFQGSQ.

What issues do you see with self.Y?
…

    ```

self.Y = np.array(
[self.Y, self.X @ np.random.randint(-2, 2, (self.X.shape[-1],))]
).T

What issues do you see with self.Y?
…

self.Y = np.array(
[self.Y, self.X @ np.random.randint(-2, 2, (self.X.shape[-1],))]
).T
The randomness of the data

I totally missed this! Taking another look over

What issues do you see with self.Y?
…

self.Y = np.array(
[self.Y, self.X @ np.random.randint(-2, 2, (self.X.shape[-1],))]
).T
The randomness of the data

I totally missed this! Taking another look over

That shouldn't cause an issue, as both KPCovR and PCovR are operating on the same dataset each time, and so randomness in the DS won't cause randomness in the results.

What issues do you see with self.Y?
…

self.Y = np.array(
[self.Y, self.X @ np.random.randint(-2, 2, (self.X.shape[-1],))]
).T
The randomness of the data

I totally missed this! Taking another look over

That shouldn't cause an issue, as both KPCovR and PCovR are operating on the same dataset each time, and so randomness in the DS won't cause randomness in the results.

That's the thing, the results are really the same. There is a difference in the third digit, and it seems to me that this is not so much an error of the algorithm, as the difference due to rounding (that is due to different values there in 4-5 digits after the decimal point).

What do you mean? We are addressing the source of the difference, not the difference itself. the switching of SVD solver should fix this.

That's what I'm talking about. I mean, we have two answers, 0.751 and 0.75. Should this be considered the same response (according to the idea of the algorithm)? Or if for the same dataset two algorithms give a difference of 0.001, then one of them doesn't work? I just don't know what the accuracy is here in general.

What do you mean? We are addressing the source of the difference, not the difference itself. the switching of SVD solver should fix this.

That's what I'm talking about. I mean, we have two answers, 0.751 and 0.75. Should this be considered the same response (according to the idea of the algorithm)? Or if for the same dataset two algorithms give a difference of 0.001, then one of them doesn't work? I just don't know what the accuracy is here in general.

It should not be considered the same response, as we discussed in dev

What do you mean? We are addressing the source of the difference, not the difference itself. the switching of SVD solver should fix this.

That's what I'm talking about. I mean, we have two answers, 0.751 and 0.75. Should this be considered the same response (according to the idea of the algorithm)? Or if for the same dataset two algorithms give a difference of 0.001, then one of them doesn't work? I just don't know what the accuracy is here in general.

It should not be considered the same response, as we discussed in dev

Ok, that's it, then no questions asked.

Randomness in the dataset CAN cause randomness in the result - different
datasets might be harder or easier to fit.
Also why randint? I can see a risk with randint getting a vector where all
the Y are the same, no?
On Tue, 20 Apr 2021 at 16:21, rosecers @.***> wrote:

What issues do you see with self.Y?
… <#m_-6898360145820963130_>
self.Y = np.array(
[self.Y, self.X @ np.random.randint(-2, 2, (self.X.shape[-1],))]
).T
The randomness of the data
I totally missed this! Taking another look over
That shouldn't cause an issue, as both KPCovR and PCovR are operating on
the same dataset each time, and so randomness in the DS won't cause
randomness in the results.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cosmo-epfl/scikit-cosmo/pull/104#issuecomment-823314522,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIREZ4YGARSWEI6UCCKAR3TJWEVLANCNFSM43IFQGSQ
.


The LLM assumes files are empty when changing a function

Which API Provider are you using?
OpenRouter
Which Model are you using?
anthropic/claude-3.5-sonnet:beta
What happened?
When working on an existing project were a function needs to be updated, the diff will show most of the code being removed (error), when respond with "only modify the function you need to, you are attempting to remove most of the code" it will apologize, review the file and try again (and do the same thing again), I then say "you're doing it again" and the finally it gets it right.where a function needs to be updated, the diff will show most of the code being removed (error). When responding with "only modify the function you need to, you are attempting to remove most of the code," it will apologize, review the file, and try again (and do the same thing again). I then say, "You're doing it again," and finally,

Steps to reproduce

Use an existing semi-complex project
Use claude-dev to diagnose a complex bug (for example parsing data from a JSON file that you need to take another action on based on the meta data in the JSON file)
work on a bug in a file with many functions (5 or more)
Work with claude-dev on the bug
Claude-dev will eventually try to replace all of the code in a file (error) rather than just the function that needs to change (you can see it in the diff)
prompt back - "you are trying to delete all of the code, please only edit the function you need to update)
Claude-dev will often repeat the same issue again and you prompt - "you're doing it again"
Claude-dev via the LLM will finally do it right

see screen cap for example
Relevant API REQUEST output
No response
Additional context
It gets worse as the context window overflows and the LLM starts to forget things, at around 12M tokens up or more..

this has got fairly bad on a large project, it deleted all of the functions in a file and I had to revert

Please see this comment


Various Updates

Some updates to make metamaska easier to understand and use from the command line, as well as sample data, CLI script, and Flask API.

Add requirements.txt
Add run.py to easily run from the command line
Add flask/app.py sample Flask API
Add dataset/dataset_sample.json
Help output clarity
README updates

We also need to

[ ] https://github.com/dogancanbakir/metamaska/blob/main/pyproject.toml#L49 - update from "^1.1.2" to "1.1.2"
[ ] add https://github.com/dogancanbakir/metamaska/blob/main/CHANGELOG.md
[ ] bump version


docs(UI): Add DevTools RxJS Welcome Back

Description:
This brings back the custom ASCII art and sample code into the console of the dev tools as used in the older docs. I think it is pretty neat as a way to get a quick sample running.

Coverage increased (+0.2%) to 96.96% when pulling 624e0140ba3a4f3bcfb77266779b3dfbbd110427 on pertrai1:rxjs-devtools-welcome into e3911bc081999c3a8b39011831bdd6e527e62f40 on ReactiveX:master.


fix: Don't crash when receiving non-string, non-array headers

I recently encountered an obscure bug, value.join is not a function, that was hard to track down. While one probably shouldn't be sending weird data to headers, in this case it was a boolean which would stringify okay. This change should keep everything functionally the same but be slightly more defensive and avoid such a crash.

@AaronMoat: Thank you for submitting a pull request! Before we can merge it, you'll need to sign the Apollo Contributor License Agreement here: https://contribute.apollographql.com/

TypeScript prevents this error, since header values are expected to be string | string[] | undefined.
That said, the use case is reasonable and so is the change. Would you mind adding some tests to RESTDataSource.test.ts which demonstrate the expected outcome is working? You'll probably have to use a // @ts-expect-error where you pass non-string non-array header values in, that's fine with me.
Thanks for taking care of the CLA / changeset / linting, etc.
Everything else looks good!

@trevor-scheer absolutely agree that typescript should usually prevent this. I got caught out by typescript letting you shoot yourself in the foot like the below:

https://www.typescriptlang.org/play?ssl=17&ssc=1&pln=17&pc=2#code/C4TwDgpgBA8g9mAzlAvFAShYBXATgOwBVwIAeUSOAMygHMsAJCAQwBMJdEA+AbgCgA9AKgiAegH4+fKtnwBjYAEs4+OoxbtOACgCUALgwQ5cXK1KJguRfloAaKBas2uUAN58RI3Fjyr3ngKgAOhD6YCY2DkQASXxgDnxmABtdWw9AkTZWa1oAQXw4YAALDgjNAwByAA8KtICAX356qRl5JRU1cI0o2PiCZN03dJFjfAsHOABbLCKcg0xjU3NLHPtHHKgAHygAIzg4JJZ8FzR-DInpso5KxFrhwMQpiBhi0u7cA0tsCHvG++8cAQLjMck0gA
I'm happy to add a couple of tests. The changes are in HTTPCache.ts, would it make more sense for tests to be in the corresponding test file or are you hoping to "integration" test it via the data source?

@AaronMoat thanks for sharing the playground, really surprising to see TS lets you spread getHeadersInternal() into the object. Adding an intermediate variable shows it just ignores its type altogether:

Yeah, I think just a couple integration level tests would be preferred. Thanks for checking.

Yeah, it was quite surprising to me!
Added a test that fails in master for cachePolicyHeadersToFetcherHeadersInit.
The change in cachePolicyHeadersToNodeFetchHeadersInit probably isn't necessary (but won't do any harm either), turns out, because it's only called with response headers, so I can't add a test that realistically would cause it to fail in master.


Add `please refer to specific README files below for installation and…

… setup instructions`

closed in favor of #104


Zarf silently fails when init package architecture doesn't match cluster architecture

Environment
Device and OS: M1 Mac
App version: v0.25.0
Kubernetes distro being used: k3s
Other: Assumption is that this is agnostic to device/OS and k8s distro. This would likely occur on any mismatch between zarf init package architecture and target cluster architecture.
Steps to reproduce

From an arm64 machine, run zarf init
If you don't already have an init package in your local ~/.zarf-cache directory, zarf will prompt you to download an init package with the same architecture of the machine you're on (which may or may not be the same architecture of the target cluster). If it finds one already in the local cache, it's likely that most of the time this package is the same architecture as the machine you're on.
Initialize your cluster with zarf init that uses a different architecture for the init package than the target cluster. In this case use an arm64 init package to initialize an amd64 target cluster.
zarf hangs on opening a port-forwarding tunnel for the zarf injector service

Expected result
Zarf performs pre-flight checks and lets me know that I can't initialize a cluster with an init package that has a different architecture
Actual Result
Zarf hangs on the init stage, and doesn't produce any information that lets you know why
Visual Proof (screenshots, videos, text, etc)

Severity/Priority
This isn't happening in a production environment, but arm64 machines for local dev are becoming more commonly used (thanks Apple), so I would consider this a medium priority.
Additional Context

Could possibly use the kubernetes.io/arch label on nodes to query for the cluster architecture. It is technically possible to have nodes of different architectures in the same cluster, so I am not sure how or if we need to be able to handle that scenario.


Bump Terraform provider version to v5.45.0

Description of your changes
This PR bumps Terraform provider version to v5.45.0
I have:

[x] Read and followed Crossplane's contribution process.
[x] Run make reviewable to ensure this PR is ready for review.
[ ] ~Added backport release-x.y labels to auto-backport this PR if necessary.~

How has this code been tested
Will update

/test-examples="examples/spanner/v1beta1/instance.yaml"

/test-examples="examples/storage/v1beta2/bucket.yaml"


Automation and setup handling of Evals

This commit adds automation scripts via go and make for generating and running evals using promptfoo.

added eval gen script
added eval provider gen script
added eval run and view script
added make file
added basic build eval

This is looking great!!! Thank you 🙏 I'll try running it soon.
It looks so far there are basic evals for the build, verify, and fix prompts, and they're using Go for examples... is that right?
Does that mean the next step is adding more languages, or is more groundwork needed first?

This is looking great!!! Thank you 🙏 I'll try running it soon.
It looks like so far there are basic evals for the build, verify, and fix prompts, and they're using Go for examples... is that right?
Does that mean the next step is adding more languages, or is more groundwork needed first?

Next step is adding more languages, which ive started to work on, and making the tests more robust :)


Finished with nonzero exit code.

I have created a simple Hello-World app:
import com.imgui.*

fun main() {
    with(ImGui) {
        text("Hello, world!")
   }
}

Yet, it doesn't show anything on screen and exits with the following message:
Execution failed for task ':runDebugExecutableNative'.
> Process 'command 'D:\hello-imgui\build\bin\native\debugExecutable\hello-imgui.exe'' finished with non-zero exit value -1073741819

check this out https://github.com/Dominaezzz/kotlin-imgui/issues/22#issuecomment-797750989

That's a surprisingly huge amount of configuration required, for even a small program.

The cost of flexibility I guess.
It shouldn't be that much code. At least in the context of graphics programming, it's pretty small I'd say.
Can I ask what you're using imgui for?

I am writing a small download manager with file-tree view.
I agree that flexibility is important. But it would be nice to have at least 2 APIs: full-fledged & simple - which would hide this complexity until you actually need it.

Looks like you got past this.


Cannot compile an app with @types/node v8 installed

Although Opentelemetry supports  >=node8 at runtime, Typescript users may run into issues whenever breaking changes are introduced into the node typings. This repo uses the latest node14 types, which will break users whenever new types are introduced, like NodeJS.Timeout.
For Azure, we have backwards compatability tests for node types, which just run tsc on a sample app with different versions of @types/node installed. I could port these over if this would be helpful.
What version of OpenTelemetry are you using?
Latest: ^0.9.0
What version of Node are you using?
node8 + Typescript
What did you do?
package.json
...
"@opentelemetry/node": "^0.9.0",
"@types/node": "^8.0.0",
...

$ tsc

node_modules/@opentelemetry/core/build/src/platform/node/timer-util.d.ts:2:50 - error TS2694: Namespace 'NodeJS' has no exported member 'Timeout'.

2 export declare function unrefTimer(timer: NodeJS.Timeout): void;

Found 1 error.

error Command failed with exit code 2.

I could port these over if this would be helpful.

I think it's a good idea to run the compile step with multiple tsc versions, but to run a full bootstrap and test might be too expensive.

I could port these over if this would be helpful.

I think it's a good idea to run the compile step with multiple tsc versions, but to run a full bootstrap and test might be too expensive.

Yep this is all it would take. The test app would just be a single file importing in @opentelemetry/node (or @opentelemetry/node-sdk!), and the test passes/fails based on the result of tsc

In that case I think this is a great idea.


See node information/output in modal overlay?

I see that it is possible to show node configuration in the Info tab and some node output in the debugging tab or even in a separate browser tab. In some cases, for example a dynamic chart, it would be nice to quickly inspect the node state through a graphical widget that would appear near the node.
Are there any examples of viewing the state or output of a node in a modal overlay?

Hi, thanks for raising an issue.
If you check out our contribution guidelines you'll see we prefer to handle discussions on possible features on the mailing list or slack team.
That way the whole Node-RED user community can help, rather than rely on the core development team.
Thanks!


Right-clicking doesn't work inside cells

Environment data

VS Code version: c3f12631

Jupyter Extension version (available under the Extensions sidebar):  v2021.8.1013163132

Python Extension version (available under the Extensions sidebar):  v2021.6.944021595

OS (Windows | Mac | Linux distro) and version: Pop!_OS 21.04 x86_64

Python and/or Anaconda version:  Python 3.8.5

Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda

Jupyter server running: Local | Remote | N/A

Expected behaviour
pop-up context meny on right-click
Actual behaviour
Have tried clicking using both laptop's embedded mouse buttons and the wired mouse's right button. also tried the laptop's mouse button, after detaching the wired mouse. However, the keyboard's context menu button works (usually, it also acted up on and off sometimes). I am using a wireless keyboard from micropack.

I wonder if it's something particular to your OS, which window manager do you use with Pop!_OS? Do context menus work elsewhere in vscode?

@roblourens  DE is GNOME and Window manager is Mutter

So you right click and simply nothing happens? Do you see any errors in the devtools (Help > Toggle developer tools)


feat: show jwt link directly for moderators and copy it

Allows joining as moderator via the mobile and electron apps
Rationale for the textarea instead of input field: the guest and the moderation link
should also be visually different, despite the long room name.

Looks like this:

I think we need the extra link, not replace it.
The coating link brings you to the page in the screenshot, and the token is refreshed on every visit, so sharing the link with the token in it is not a good idea h less is for immediate use.

I did not see any expiry date in the token creation, thus I tried to keep it simple. Is there an expiry that I just overlooked or is there something else that invalidates the token after some time?

I'll double check, maybe I am mistaken.

Any update on this one? Otherwise I would like to get this merged so that the electron app (and probably the mobile apps) finally work as moderator on moderated-meetings as well


LinkedHashMap not on classpath

Is the LinkedHashMap not loaded on the classpath?

And map.update doesn't seem to exist either, so which version of Dart is being used? Could the Dart version be listed on the footer?

It's based on 1.25 (with plans to update). If you click on the DatPad title in the page, it'll show the about box w/ the current version:

see also https://github.com/dart-lang/dart-services/issues/334

ah ha, so you can click on the DartPad title to get that. Thanks that helps me understand what the options are. Looks like this is a duplicate.


DNS records audit

From the discussion on Slack, audit and cleanup the existing mediacloud.org DNS records on Cloudflare, AWS Route 53

Updated the DMARC policy to v=DMARC1; p=quarantine; pct=50<EMAIL_ADDRESS>The policy is applied to only 50% of messages received by mail servers. Messages that don’t pass DMARC are delivered to recipients’ spam folders. Only a small percent of messages are impacted, and recipients can review messages that are sent to spam
We can monitor impact of the policy & thereafter upgrade to a policy that rejects 100% of mails that to don meet DMARC authentication, or relax the policy

Cleaned up records on AWS's Route 53.
The srv.mediacloud.org record/hosted zone still exists since this is also present in Cloudflare **


feat: Support team level buckets by using the direct node role ARN

This pull request enables a customer to directly grant access to the EKS node role are removes the need for /root at the account level.

This PR is included in version 1.16.0 :tada:


Support embeddings stored in step results

Summary

Support embeddings listed in step results.
Motivation and Context

I'm looking into cucumber-cpp for my company.
We need to be able to embed text or images into the reports, but that feature is not available when using the wire protocol.
This patch is the 1st of a series a 4, respectively impacting cucumber-ruby, cucumber-ruby-core, cucumber-ruby-wire and cucumber-cpp.
Basically, the idea is to allow embeddings to be specified in the success or fail responses sent through the wire protocol. E.g.:
["success",{"embeddings":[{"label":"Embedded text","mime_type":"text/plain","src":"Some text"}]}]
["fail",{"embeddings":[{"label":"Embedded image","mime_type":"image/png;base64","src":"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="}],"message":"There was no  banana"}]

The embeddings will then be stored in the step Result in order to be accessible by the formatter.
The patch on cucumber-ruby (cucumber/cucumber-ruby#1354) will let formatters access embeddings stored in a step result.
The patch on cucumber-ruby-core (cucumber/cucumber-ruby-core#170) will store in the step result the embeddings returned by the step invocation.
The patch on cucumber-ruby-wire (cucumber/cucumber-ruby-wire#20) will allow the embeddings stored in the wire response to be returned by the wire-based step invocation process.
The patch on cucumber-cpp (cucumber/cucumber-cpp#223) will allow C++ steps to send embeddings through wire responses.
Note: embeddings were not considered to be useful for pending steps, but support should be possible if required.
Details

In the current implementation, it's not possible to access data from the wire response as nothing is returned from the wire invocation when a step succeed (a step is considered successful by default, an exception needs to be raised in case of failing or pending step).
Successful step
Cucumber::Core::Test::Runner::RunningTestCase::Status::Base.execute
  Cucumber::Core::Test::Step.execute
    Cucumber::Core::Test::Action.execute
      block -> Cucumber::StepMatch.invoke
        Cucumber::Wire::StepDefinition.invoke
          Cucumber::Wire::Protocol.invoke
            Cucumber::Wire::Protocol::Requests::Invoke.execute
              Cucumber::Wire::RequestHandler.execute
                Cucumber::Wire::Connection.call_remote
                  Cucumber::Wire::DataPacket reponse = // parsed packet
                  Cucumber::Wire::DataPacket.handle_with
                    Cucumber::Wire::DataPacket.handle_success // noop
                  // response discarded
      Cucumber::Core::Test::Action.passed -> return Cucumber::Core::Test::Result::Passed
  Cucumber::Core::Test::Result::Passed.describe_to(Cucumber::Core::Test::Runner::RunningTestCase)
    Cucumber::Core::Test::Runner::RunningTestCase.passed
      Cucumber::Core::Test::Runner::RunningTestCase@status = Cucumber::Core::Test::Runner::RunningTestCase::Status::Passing

Failing step
Cucumber::Core::Test::Runner::RunningTestCase::Status::Base.execute
  Cucumber::Core::Test::Step.execute
    Cucumber::Core::Test::Action.execute
      block -> Cucumber::StepMatch.invoke
        Cucumber::Wire::StepDefinition.invoke
          Cucumber::Wire::Protocol.invoke
            Cucumber::Wire::Protocol::Requests::Invoke.execute
              Cucumber::Wire::RequestHandler.execute
                Cucumber::Wire::Connection.call_remote
                  Cucumber::Wire::DataPacket reponse = // parsed packet
                  Cucumber::Wire::DataPacket.handle_with
                    Cucumber::Wire::RequestHandler.handle_fail -> throw Cucumber::Wire::Exception // error message (from response) embedded in exception
                  // response discarded
      Rescue in Cucumber::Core::Test::Action.execute
        Cucumber::Core::Test::Action.failed -> return Cucumber::Core::Test::Result::Failed
  Cucumber::Core::Test::Result::Failed.describe_to(Cucumber::Core::Test::Runner::RunningTestCase)
    Cucumber::Core::Test::Runner::RunningTestCase.failed
      Cucumber::Core::Test::Runner::RunningTestCase@status = Cucumber::Core::Test::Runner::RunningTestCase::Status::Failing

This patch allow two things:

Visitors of Result to be notified of embeddings (if any).
The json formatter (who does not use the visitor pattern) to read embeddings (if any) in Result.

Those changes should be backward compatible.
How Has This Been Tested?

No test in that repository, but I just discovered there is a FakeWireServer, I need to see if I can do something with it. Suggestions are welcomed.
Types of changes

[ ] Refactor (code change that does not change external functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] I've added tests for my code
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@Sigill in what order do we need to deal with these things. Is this something you would like merged in now? Or does it rely on the other items.

@luke-hill The patches must be integrated in the following order:

cucumber-ruby (cucumber/cucumber-ruby#1354)
This patch allow the json formatter to use embeddings stored in Passed/Failed step results. Nothing happen if the embeddings field is not present (it is added by the 2nd patch).
cucumber-ruby-core (cucumber/cucumber-ruby-core#170)
This patch allow whatever is invoking a step to return an object (PassedInvokeResult, FailedInvokeResult) describing the result of the invocation (status, embeddings), and Action to create the resulting Passed/Failed object. The previous logic is still supported.
cucumber-ruby-wire (cucumber/cucumber-ruby-wire#20)
This patch make the wire protocol request handler responsible of step invocations return a PassedInvokeResult/FailedInvokeResult. This change support Wire json responses not containing an embeddings field (added by the 4th patch).
cucumber-cpp (cucumber/cucumber-cpp#223)
This patch adds embeddings to the Wire json response.

Each patch is supposed to be backward compatible with respect to the dependencies, we should therefore be able to integrate them independently (but in that order).
The feature is complete with those patches. I just don't have any test for this patch (I havn't yet looked at FakeWireServer to see if it could help me about that). Suggestions are welcome.
Otherwise, they are ready to be merged.

Perfect answer. I'll get on with getting the first patch in in the next few days.

Merged this in as I can follow the flow as described and as you've explained.

Hi @Sigill,
Thanks for your making your first contribution to Cucumber, and welcome to the Cucumber committers team! You can now push directly to this repo and all other repos under the cucumber organization! 🍾
In return for this generous offer we hope you will:

✅ Continue to use branches and pull requests. When someone on the core team approves a pull request (yours or someone else's), you're welcome to merge it yourself.
💚 Commit to setting a good example by following and upholding our code of conduct in your interactions with other collaborators and users.
💬 Join the community Slack channel to meet the rest of the team and make yourself at home.
ℹ Don't feel obliged to help, just do what you can if you have the time and the energy.
🙋 Ask if you need anything. We're looking for feedback about how to make the project more welcoming, so please tell us!

On behalf of the Cucumber core team,
Aslak Hellesøy
Creator of Cucumber


Not able to see the "View in sanddance"

Hi @Bhupendrasinha , I've looked at the available places to add menus for VSCode and it seems that "Open Editors" is not a place where context menus can be added. I'm not sure why.
I can't find any other extensions in the marketplace that are able to add a context menu here. If you find one, please let me know and we can hopefully use it as an example. Otherwise, I am going to close this issue for now. Thanks!

Oh, BTW, you can see get "View in SandDance" when you right click in the file tree, or if you are viewing the file, by right-clicking the tab at the top.


Stucked on Testnet Rescanning - switching back to Mainnet without ending Testnet Setup?

I switched to the Testnet like in #108 - so I got a full synced Mainnet Node before.
Now I stucked on the Testnet Rescanning

The "?" in the progress seems to show a deeper problem with the Testnet.
I rebooted the Raspiblitz with a terminal comand, but it didn't change the situation.
Now I want to switch back to the Mainnet but I don't know how this works and which terminal commands I have to use - can someone help me?

I just switched to the mainnet again by using the terminal command: "sudo /home/admin/95switchMainTest.sh"
I'm on ther Mainnet again wohoo


LESQ-1096: fix confirmation email sent twice

Description
Music year: 1988
rendering two sign-up/sign-in forms was causing the email confirmation
to be sent twice. I believe this was intended to make it easier to
present an alternative mobile view of the sign-up UI.
it appears to ultimately be an implementation bug in Clerk, because this
only happens with the confirmation link flow and not with the confirmation
code flow. Also if you refresh the page it will send another link which
is not great.
How to test

Go to https://deploy-preview-2834--oak-web-application.netlify.thenational.academy/sign-up
Sign up with an email address
You should see the "Verify your email" screen
Check your email
You should have received one confirmation email

🚨 I had to make some tweaks to the way the responsive layout was implemented to fix this. Definitely worth checking that I haven't caused a regression in the page layout (I tested and it all looked correct)
🤦🏻♂ 😮💨  Reloading the "Verify your email" screen will also cause another email to be sent. This appears to be a bug in Clerk because these emails should be throttled to the countdown you can see on screen.
Also, it will continue to send 2 emails in development owing to strict mode which causes the component to double render. In fact I was getting 4 emails in development because of strict mode and the double form bug combined. Fun.
Checklist

[ ] Added or updated tests where appropriate
[ ] Manually tested across browsers / devices
[ ] Considered impact on accessibility
[ ] Design sign-off
[ ] Approved by product owner
[ ] Does this PR update a package with a breaking change

:tada: This PR is included in version 1.572.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


haxelib submit broken using latest haxe

$ haxelib submit haxelib.zip
/usr/local/Cellar/haxe/HEAD/lib/haxe/extra/haxelib_src/src/tools/haxelib/Data.hx:239: characters 28-31 : Unknown identifier : doc

@back2dos would you take a look?

Are you saying this broke after updating Haxe?

I meant the error occurs when using the latest development branch of haxe, together with the latest master branch of haxelib.
haxelib is run using the script generated by make install:
#!/bin/sh
exec haxe -cp /usr/local/Cellar/haxe/HEAD/lib/haxe/extra/haxelib_src/src --run tools.haxelib.Main "$@"

Uhm, will do.

Is this still an issue?

Yes. It still behaves the same as I said in the previous comment.

Any clue as to what haxelib_src might be?
On Tue, Mar 10, 2015 at 4:14 AM, Andy Li<EMAIL_ADDRESS>wrote:

Yes. It still behaves the same as I said in the previous comment.
—
Reply to this email directly or view it on GitHub
https://github.com/HaxeFoundation/haxelib/issues/164#issuecomment-77988771
.

"haxelib_src" is the source of haxelib, checked out the master branch with the latest commit 94e233284e1bae7a1324e6ce58b7084465c4f2a3.
In fact, the issue can be reproduced by:
haxe -cp path/to/haxelib_src --run tools.haxelib.Main submit packagedLib.zip

I'm starting to think it was a very bad idea to move haxelib outside of haxe GIT...

Can you elaborate?
On Tue, Mar 10, 2015 at 9:58 AM, Nicolas Cannasse<EMAIL_ADDRESS>wrote:

I'm starting to think it was a very bad idea to move haxelib outside of
haxe GIT...
—
Reply to this email directly or view it on GitHub
https://github.com/HaxeFoundation/haxelib/issues/164#issuecomment-78016293
.

Well, it seems even harder than before to take decisions, nobody seems to feel responsible for fixing things (maybe I should ?), and regressions are introduced without (almost) anyone noticing or reacting.

I'll go on strike if you bring this back into my field of vision!

[edit] (maybe @Simn should ?)  :-)

Yes, this project clearly lacks any kind of leadership. And it's hard to step into the role without the foundation laying out project goals. I would be willing to take on more responsibility, but I'd rather not have to go get feedback, particularly if it's not given or given after several months. Anyway, this is probably something to discuss at the WWX and not here.

This issue is probably related to the use of macros on haxelib. haxe --run is run on the macro context, so it cannot contain any macros ( like https://github.com/HaxeFoundation/haxelib/blob/master/src/tools/haxelib/Validator.hx#L153 ). I think we need to solve this before 3.2-rc

By the way, I can reproduce this as well

The fix for this hasn't been included in the latest client in the server and available to 'selfupdate'.

It should be in the latest release.

@jonasmalacofilho Done.


Support sklearn 0.24.0

#1592 is preventing us from upgrading to sklearn 0.24.0. Once that's fixed, we should be able to make the upgrade.

For whoever picks this up, we also need to refactor test_jupyter_graph_check in model_understanding to support this latest release.
The test checks that no warnings are raised but sklearn now raises a warning about a future breaking api change.


use the new app addon for deployment instruction

Signed-off-by: Mike Ng<EMAIL_ADDRESS>

/hold
https://github.com/qiujian16/multicloud-operators-subscription/pull/2
https://github.com/open-cluster-management-io/multicloud-operators-subscription/pull/15
needs to be merged first then this can be merged.

/unhold
/assign qiujian16


Add IDE files to gitignore

Add JetBrains, Vim, and Mac DS_Store files to gitignore

Hi
I don't think that we're supposed to expand this for every shingle piece of metadata for every single IDE :) In situations like those please use a global ignore list for your system.
https://docs.github.com/en/github/getting-started-with-github/getting-started-with-git/ignoring-files#configuring-ignored-files-for-all-repositories-on-your-computer


not support tensorflow-gpu version?

Hi, thanks for the repo. I meet an error when runing the example code of ex_cnn_head_pose_estimation_images.py.
It seems the error occurs with tensorflow-gpu, when I change to tensorflow==1.5 with python 2.7 (same as you have mentioned) it works well.
Belows is the record, thanks for help.
Processing image ..... 1.jpg
2019-06-25 17:06:17.044046: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
2019-06-25 17:06:29.808930: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2019-06-25 17:06:29.831308: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Traceback (most recent call last):
File "/home/user/anaconda2/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/user/anaconda2/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/user/anaconda2/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node Conv2D}}]]

Hi @plutoyuxie

It seems the error occurs with tensorflow-gpu, when I change to tensorflow==1.5 with python 2.7 (same as you have mentioned) it works well.

I did not understand when it is working and when it is not working. There are a few possibilities that led to the issue. One can be that the version of cuDNN you have on your system is not compatible with tensorflow 1.5. It is pretty annoying but specific versions of tensorflow require specific versions of cuDNN.

Thanks.


About the data information returned by CDRFLEx.get_rt_control_output_data_list, Doubts about actual_motor_torque, actual_joint_torque, raw_joint_torque, and mass_matrix

English translation：
Hello, I am an engineer from Wuhan Jingtian. I am not very familiar with some user questions. In the returned data information of CDRFLEx.get_rt_control_output_data_list, actual_motor_torque, actual_joint_torque, raw_joint_torque, mass_matrix doubts, I would like to ask you to understand the specific problems, I ask the user of the robot arm to specify:
Chinese translation：
您好，我是武汉京天的工程师，我这边有一些用户的问题不是很了解，关于CDRFLEx.get_rt_control_output_data_list返回的数据信息中，actual_motor_torque、actual_joint_torque、raw_joint_torque、mass_matrix的疑惑，向您这边了解一下，具体问题我请机械臂的使用者具体说明：

Hello, I want to use the real-time control mode of the doosan robotic arm to build a Cartesian impedance controller. However, I found that there is an issue with the mass matrix returned by the rt monitor. For example, the values at positions [5,5] and [6,6] of this matrix are close to 0, such as 0.01. When using this matrix for control, the joint forces obtained for the end effector error are close to 0. I have also considered the issue of setting the end effector tool, but after setting it's mass, the matrix remains essentially unchanged.
你好，我想使用东山机械臂的实时控制模式构建笛卡尔阻抗控制器，但是我发现机械臂返回的质量矩阵是有问题的，例如该矩阵的[5,5]和[6,6]位置的值0.01已经接近0了，使用该矩阵进行控制时末端关节的误差得到的关节力接近0，我也考虑了末端工具质量设置的问题，但是设置之后该矩阵基本没有变化。