Decommission the COESSING hub

Description
In https://github.com/2i2c-org/pilot-hubs/issues/473, the end date for the COESSING hub is stated as August 31st. This date has now passed and so we should confirm the hub is no longer in use and remove it, especially since this particular hub is running in the old Pangeo project that we are planning to migrate away from.
Value / benefit

No longer maintaining unused resources
Saving credits

Implementation details
No response
Tasks to complete
Phase I

[ ] Confirm with Community Representative that the hub is no longer in use and it's safe to decommission
[ ] Confirm if there is any data to migrate from the hub before decommissioning

[ ] If yes, confirm where the data should be migrated to
[ ] Confirm a 2i2c Engineer has access to the destination in order to complete the data migration

Phase II - Hub Removal

[ ] (Optional) Migrate data from the hub
[ ] Remove the hub deployment

helm delete --namespace HUB_NAME HUB_NAME
kubectl delete namespace HUB_NAME

[ ] Remove hub entry from the appropriate *.cluster.yaml file

Phase III - Cluster Removal
This phase is only necessary for single hub clusters.

[ ] Run terraform plan -destroy and terraform apply to destroy the cluster
[ ] Remove the following files from the repository:

The associated *.cluster.yaml file
The associated CI deployer key in secrets/
Remove the name of the cluster from CI

Updates
No response

I believe that the Community Representative (@paigem) had a few workshop attendees that were interested in using the hub infrastructure on an ongoing basis.
Paige - we are in the midst of setting up a new Pangeo hub on 2i2c infrastructure. Would it make sense to just migrate those folks to the Pangeo hub? Or is there a strong reason to keep them on a COESSING-dedicated hub?

Thanks @sgibson91 for starting this issue and @choldgraf for pinging me about this!
I think in an ideal world, it would be great to keep the COESSING hub running. I want to avoid the idea of providing a valuable resource during the school, and then promptly removing access to it, particularly since most attendees don't have regular access to computing resources beyond their laptops.
However, I realize that running and maintaining the COESSING hub comes at a cost (of both time and money). I know very little about the cost breakdowns, but if it were the case that the costs were similar in keeping up the COESSING hub versus migrating a few (probably <20) users to Pangeo hub, then I would definitely push to keep the COESSING hub open. I assume that it would be much more cost-effective to migrate a few users over, in which case it probably makes the most sense to do this. It would be great to have @rabernat's view here.
Note that we plan to host another COESSING workshop at the same time next year, and would likely want to have a COESSING hub again at that time.
If we decide to decommission the COESSING hub, it would be great if I could have a week or two before the hub goes down to contact all attendees and gather a list of folks who want to be migrated to Pangeo hub.

Thanks @paigem. There is another option here providing this is ok by @rabernat. Currently the COESSING hub is running on it's own cluster in the pangeo-181919 project. That whole project is in the processing of being decommissioned as part of the migration to 2i2c. We now have a new cluster up-and-running with a hub that Ryan is using for his classes. We could migrate the COESSING hub to that cluster which would reduce costs since it's sharing a cluster with the main Pangeo project.
If we can't migrate it to the new Pangeo cluster, @choldgraf could we migrate the hub to one of our other dask-enabled clusters and work out a cost breakdown with Paige?

We could migrate the COESSING hub to that cluster which would reduce costs since it's sharing a cluster with the main Pangeo project.

Going forward, "that cluster" (i.e. the one we are using for my class) will become the main, public Pangeo cluster, replacing https://us-central1-b.gcp.pangeo.io/. I propose we leave the COESSING hub up and running on pangeo-181919 until we are ready to announce the general migration to https://pangeo.2i2c.cloud/. At that time the COESSING users can migrate along with everyone else.
👍 ?

@paigem what do you think? :point_up:

I like @rabernat's proposal, provided that @paigem has no objections!

This sounds like a great plan!
I assume that the COESSING users can have their files automatically transferred to the new hub when the time comes? (Just wondering if I need to give much warning to COESSING folks to back anything up, etc.)

@paigem yes we can do that!

I have updated the top comment to reflect the next steps we've agreed upon. We won't action anything on the COESSING hub until Pangeo prod has matured.

@paigem our timeline got pushed back and I now plan to tear down old infrastructure on 22nd November - is that ok for the COESSING users?

Yes @sgibson91 that time frame will work just fine! Sorry for the delayed response.
I have made all participants aware of the switch, and written out instructions for how to join the Pangeo deployment. I have heard very little feedback, so my guess is that most people will not make the switch right now, but at least they now have instructions for how to get access to the Pangeo Hub in the future if they so choose! Thanks so much for keeping me in the loop about all this.

Great, thanks @paigem!

Task List
Phase I

[ ] Confirm with Community Representative that the hub is no longer in use and it's safe to decommission
[ ] Confirm if there is any data to migrate from the hub before decommissioning

[ ] If yes, confirm where the data should be migrated to
[ ] Confirm a 2i2c Engineer has access to the destination in order to complete the data migration

Phase II - Hub Removal

[ ] (Optional) Migrate data from the hub
[ ] Remove hub entry from the appropriate *.cluster.yaml file
[ ] Remove the hub deployment

helm --namespace HUB_NAME delete HUB_NAME
kubectl delete namespace HUB_NAME

[ ] Delete the hub's authentication application on auth0 or GitHub

Phase III - Cluster Removal
This phase is only necessary for single hub clusters.

[ ] Run terraform plan -destroy and terraform apply to destroy the cluster
[ ] Remove the following files from the repository:

The associated *.cluster.yaml file
The associated CI deployer key in secrets/
Remove the name of the cluster from CI

The cluster is now torn down, folk should be using us-central1-b.gcp.pangeo.io instead. I will wait a few days before deleting the NFS server, just in case.

Shall we close this once the NFS server is deleted?

Yeah, that was my plan. I just didn't want to purge the NFS straightaway in case any panicked messages from users came in. If I don't hear anything by Monday, I'll delete the NFS server and close this.

The NFS server has now been deleted! 🎉


Update V_TASK_LOG_MESSAGES_V_EXT.SQL

Closing Quotes in "START_DATE" had a different format and weren't recognized in Data Warehouse Cloud.

Tx!


Please help, I am new here dont know whatit is

I am attempting to join the website, but after i push the submit button of the first exercise the monitor screen of the website goes blank. Please help

What browser, browser version and OS are you using?

Chrome 51.0.2704.103 m
Windows 8

I did th exercise in python

Closing old issue. I'm sorry you got not replies. Please open a new one if this issue is still there.


Too long underline in the Date Picker

Describe the bug
When the Date Picker component is focused its underline becomes too long.
How to reproduce
Steps to reproduce the behavior:

Click in the Date Picker input
The input underline becomes too long

Expected behavior
The underline should be as long as the input itself.
Versions
App

Angular: 10
Clarity: 4

Device:

OS: Windows
Browser: chrome, safari, firefox

Additional Notes
Currently there is a problem with reproducing the issue in Stackblitz, but I use just a simple Date Picker:
<form clrForm>
  <clr-date-container>
    <label for="demo">Test Date</label>
    <input type="date" clrDate name="demo" id="demo" [(ngModel)]="date">
  </clr-date-container>
</form>

And here is a video that shows the issue:

Similar issue is observed with the datalist.
The animation is on a container, which takes 100% of the container, while the non-focused underline is on the input field only


CRUD handler 🤔

🙃

The idea was to add a CrudHandler interface with five methods:
path("/user") {
    crud(UserHandler())
}

It would be the same as adding:
path("user") {
    get(UserController::getAll)
    post(UserController::create)
    path(":user-id") {
        get(UserController::getOne)
        patch(UserController::update)
        delete(UserController::delete)
    }
}


[BugFix]Revert "[Enhancement] optimize iceberg const column (#31632)"

This reverts commit f82b51ca89556ca7c9625ff90f07a2c204ffa821.
This pr encountered some bad case when deploy exec plan, revert first and optimize later
Fixes #issue
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5

[FE Incremental Coverage Report]
:disappointed: fail : 5 / 17 (29.41%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/plan/PlanFragmentBuilder.java
0
1
00.00%
[1156]

:large_blue_circle:
com/starrocks/planner/IcebergScanNode.java
5
16
31.25%
[155, 156, 159, 161, 162, 163, 167, 168, 170, 171, 172]

[BE Incremental Coverage Report]
:disappointed: fail : 3 / 4 (75.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/runtime/descriptors.cpp
0
1
00.00%
[585]

:large_blue_circle:
src/connector/hive_connector.cpp
2
2
100.00%
[]

:large_blue_circle:
src/formats/parquet/file_reader.cpp
1
1
100.00%
[]

[FE Incremental Coverage Report]
:disappointed: fail : 5 / 17 (29.41%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/plan/PlanFragmentBuilder.java
0
1
00.00%
[1156]

:large_blue_circle:
com/starrocks/planner/IcebergScanNode.java
5
16
31.25%
[155, 156, 159, 161, 162, 163, 167, 168, 170, 171, 172]

[BE Incremental Coverage Report]
:disappointed: fail : 3 / 4 (75.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/runtime/descriptors.cpp
0
1
00.00%
[585]

:large_blue_circle:
src/connector/hive_connector.cpp
2
2
100.00%
[]

:large_blue_circle:
src/formats/parquet/file_reader.cpp
1
1
100.00%
[]


cageseq error

Hi
I am getting input tuple error. Could you please help me with this?
WARN: Access to undefined parameter `config_profile_contact` -- Initialise it to a default value eg. `params.config_profile_contact = some_value`
WARN: Access to undefined parameter `config_profile_url` -- Initialise it to a default value eg. `params.config_profile_url = some_value`
Run Name          : big_payne
Input             : sample_1/*R1.fastq.gz
5' artifacts      : software/cageseq/assests/artifacts_5end.fasta
3' artifacts      : software/cageseq/assests/artifacts_3end.fasta
trim_ecop         : true
trim_linker       : true
trim_5g           : true
trim_artifacts    : true
eco_site          : CAGCAG
linker_seq        : TCGTATGCCGTCTTC
Remove rRNA       : true
Aligner           : STAR
STAR Index        : hg38.analysisSet.chroms
Min. cluster      : 30
Cluster Threshold : 0.2
bigwig output     : Yes
Save Reference    : false
Max Resources     : 160 GB memory, 20 cpus, 5d 23h time per job
Output dir        : test_results/
Launch dir        : software
Working dir       : software/work
Script dir        : software/cageseq
User              : KG
Config Profile    : chimera
Config Files      : software/cageseq/nextflow.config
----------------------------------------------------
No signature of method: java.lang.Boolean.getFileSystem() is applicable for argument types: () values: []

WARN: Input tuple does not match input set cardinality declared by process `fastqc` -- offending value: sample_1/sample.R1.fastq.gz
WARN: Input tuple does not match input set cardinality declared by process `trim_adapters` -- offending value: sample_1/sample.R1.fastq.gz
[-        ] process > get_software_versions -
[-        ] process > convert_gtf           -
-[nf-core/cageseq] Pipeline completed with errors-

Hi,
Can you run ls sample_1/*R1.fastq.gz and post the resutl here, to check how the input is formatted.

hmm, worth to give it a try with a non-symlinked file

I just did and same error. Could you please give more suggestions or ideas to debug?
No signature of method: java.lang.Boolean.getFileSystem() is applicable for argument types: () values: []

WARN: Input tuple does not match input set cardinality declared by process `fastqc` -- offending value: sample_1/*R1.fastq.gz
WARN: Input tuple does not match input set cardinality declared by process `trim_adapters` -- offending value: sample_1/*R1.fastq.gz
[-        ] process > get_software_versions -
[-        ] process > convert_gtf           -
[-        ] process > get_chrom_sizes       -
[-        ] process > fastqc                -
[-        ] process > trim_adapters         -
-[nf-core/cageseq] Pipeline completed with errors-

what is the java and nextflow version you are using?

I have create my own  configure profile; here is the copy of it.
`params {
// Genomes reference file paths
genomes {
'GRCh38.Gencode.v30' {
fasta = "/GRCh38/FASTA/GRCh38.primary_assembly.genome.fa"
gtf =   "/GRCh38/Gencode/release_30/gencode.v30.primary_assembly.annotation.gtf"
star     = "/GRCh38/star/2.7.1a/Gencode.v30.overhang100/chr_primary"
reflat   = "/GRCh38/Gencode/release_30/gencode.v30.primary_assembly.annotation.reflat"
rrna     = "/GRCh38/Gencode/release_30/gencode.v30.rRNA.interval.list"
rsem     = "GRCh38/rsem/1.3.0/gencode.v30.bam/GRCh38.primary_assembly"
globin   = "/GRCh38/Gencode/release_30/globin_gene.gtf"
}
}
version {
cutadapt = '4.1'
samtools = '1.17'
star = '2.7.5b'
R = '3.5.3'
fastqc = 'v0.11.9'
paraclu = '021-02-04'
sortmerna = '4.3.6'
py_packages = '3.7'
python = '3.7.3'
bedtools='v2.29.2'
read_distribution.py = '5.0.1'
bowtie = '1.3.0'
multiqc = '1.10.1'
}
}
process {
// default executor values
cpus = 1
memory = 2.GB
time = 1.h
// Environmental settings and computational resources
withName:convert_gtf {
    module = 'pigz'
    cpus = 1
    memory = 1.GB // 1x6
    time = {2.h * task.attempt}
}
withName:get_chrom_sizes {
    module = 'pigz'
    cpus = 1
    memory = {1.GB * task.attempt} // 1x6
    time = {3.h * task.attempt}
}
withName:fastqc {
    module = {"fastqc/${params.version.fastqc}"}
    cpus = 4
    memory = 2.GB  // 2*4
    time = {4.h * task.attempt}
}
withName:make_STAR_index {
    module = {"star/${params.version.star}"}
    cpus = 1
    memory = 5.GB  // 12*5
    time = {4.h * task.attempt}
}
withName:trim_adapters {
    module = {"cutadapt/${params.version.cutadapt}"}
    cpus = 4
    memory = 2.GB  // 2*4
    time = {4.h * task.attempt}
}
withName:trim_5g {
    module = {"cutadapt/${params.version.cutadapt}"}
    cpus = 4
    memory = 2.GB  // 2*4
    time = {4.h * task.attempt}
}
withName:trim_artifacts {
    module = {"cutadapt/${params.version.cutadapt}"}
    cpus = 4
    memory = 2.GB  // 2*4
    time = {4.h * task.attempt}
}
withName:sortmerna {
    module = {"sortmerna/${params.version.sortmerna}"}
    cpus = 5
    memory = 2.GB // 4*2
    time = {10.h * task.attempt}
}
withName:trimmed_fastqc {
    module = {"samtools/${params.version.samtools}"}
    cpus = 1
    memory = 2.GB // 1*2
    time = {4.h * task.attempt}
}
withName:star {
    module = {"star/${params.version.star}"}
    cpus = 12
    memory = 5.GB  // 12*5
    time = {10.h * task.attempt}
}
withName:samtools_stats {
    module = {"samtools/${params.version.samtools}"}
    cpus = 1
    memory = 2.GB // 1*2
    time = {4.h * task.attempt}
}
withName:get_ctss {
    module = {"subread/${params.version.subread}"}
    cpus = 12
    memory = 2.GB // 12*2
    time = {2.h * task.attempt}
}
withName:make_bigwig {
    module = {"bedtools/${params.version.bedtools}:python/3.7.3:py_packages/3.7"}
    cpus = 2
    memory = 6.GB // 2*6
    time = {12.h * task.attempt}
}
withName:cluster_ctss {
    module = {"bedtools/${params.version.bedtools}:paraclu/${params.version.paraclu}:python/3.7.3:py_packages/3.7"}
    cpus = 1
    memory = 24.GB
    time = {6.h * task.attempt}
}
withName:generate_counts {
    module = {"bedtools/${params.version.bedtools}:python/3.7.3:py_packages/3.7"}
    cpus = 3
    memory = 20.GB
    time = {6.h * task.attempt}
}
withName:ctss_qc {
    module = {"bedtools/${params.version.bedtools}:python/3.7.3:py_packages/3.7"}
    cpus = 2
    memory = 16.GB // 2*16
    time = {8.h * task.attempt}
}
withName:multiqc {
    module = {"python/3.7.3:py_packages/3.7:gcc/7.3.0"}
    cpus = 2
    memory = 16.GB // 2*16
    time = {2.h * task.attempt}
}
withName:output_documentation {
    module = 'pigz'
    cpus = 1
    memory = 1.GB // 1x6
    time = {12.h * task.attempt}
}

}
`

nextflow/22.10.6
java/11.0.2/

Do the tests run, i.e. can you run it with -profile test ?

might also be a nextflow dsl1/dsl2 problem. This pipeline is unfortunately still DSL1 (and nextlfow dropped it's support anymore since 22.10.x). I hope I can get out a new version at the end of this summer, but for now could you maybe downgrade your nextflow version?

I am now trying
nextflow/19.10.0
java/1.8.0_211/
`nextflow run cageseq -profile 'test'
I changed projectDir to a specified path everywhere because this nextflow has no space for projectdir argument.
But it gives me this error
Unknown config attribute params.genomes.GRCh37.projectDir` -- check config file:
cageseq/nextflow.config

can you please post the modified cageseq/nextflow.config here

closing because of lack of response


Create an array column in Bigquery table

Hi,
from the docs i see that Bigquery supports array columns (https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#array-type), is there any way to create a column of this type from the ruby client?
Reading the source the only explicit types I can find are:

string
integer
float
numeric
boolean
bytes
timestamp
time
datetime
date
record

Maybe there is a way using the record type?
Thanks in advance!

The Ruby BigQuery gem does support creating array columns. The terminology used in the BigQuery API is a "repeated" column. So as you define your columns you specify the column type and indicate that it is repeated (using the mode optional named argument).
A code example used in the Ruby documentation shows defining a repeated "cities_lived" column.
require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

table.schema do |schema|
  schema.string "first_name", mode: :required
  schema.record "cities_lived", mode: :repeated do |nested_schema|
    nested_schema.string "place", mode: :required
    nested_schema.integer "number_of_years", mode: :required
  end
end

The data returned is a Ruby Array of values, depending on the type of the column.

Hi, sorry for the delay, thanks for the answer, for now I switched to a fixed schema approach (not using the repeated columns), because of the easier integration with a CSV import (from the documentation i see that there isn't support to load repeated data from a CSV file).
Anyway, thank you very much for the answer, I'll keep the example for the next time!


Update to testcontainers released version

Just in time ;)

CI failed -- https://github.com/prestosql/presto/issues/4864


Add option to burn in IMSC catpions

There should be an option to burn-in subtitles in the video frame. This is an extension of #3. The exported PNG could be input for a "burn-in" service.

Resolved in release 0.4.0


onPermissionDenied and onPermissionRationaleShouldBeShown both called

Expected behaviour
What I would expect is that when I show a custom dialog for rationale, and the user still chooses to not allow the permission request, that rationale will not be shown again.
Actual behaviour
What actually happens is the following:

The custom rationale is shown when onPermissionRationaleShouldBeShown is called
The user chooses to deny that permission.
The app calls token.cancelPermissionRequest();
onPermissionDenied is called, as expected.
onPermissionRationaleShouldBeShown is called AGAIN, leaving the user in a never ending loop of dialogs (until they choose to allow the permission).

Can someone explain if this is the intended behavior of the rationale system? Am I just misunderstanding how it is supposed to work?
Happens on:
Genymotion Nexus 5X - 6.0.0
Dexter version 3.0.2

Aha, it seems this was caused by me attempting to check the permissions in the activity onResume(). I moved the call to onCreate and all is well and dandy now.


VideoStreamGDNative: close file in cleanup()

We should close the file handle (created by FileAccess::open()) and remove it when we are done playing the video.
I've had a Windows app playing short videos (~10 sec) in loop, by repeatedly calling set_stream() and play() in a VideoPlayer node with a VideoStreamGDNative instance. A few hours later this has depleted allowed Windows file handle count in the godot process. All subsequent API call related to filesystem open will fail.

Thanks!

Cherry-picked for 3.2.3.


Is there an async version of touch?

Exactly as the question sounds. Is there an async version of touch?

To create an empty file, you can do
await fs._pipe_file(path, b"")

I don't think any async backends support the extended use of touch to update the timestamp or truncate to a specific length; but you can probably update metadata separately (also backend-specific).


maintainers: add alex-fu27

Description of changes
Add me to maintainer-list.nix so I can help weeding out issues. @Atemu
Edit: context

Could you squash your commits?
CI failure seems unrelated, I'll check what's up with that. Best rebase your commit onto master in case it's already fixed.

We prefer to keep the history as is ;)


Options to sort by createdAt & updatedAt dates

Codecov Report

Merging #2114 into master will decrease coverage by 0.05%.
The diff coverage is 36.66%.

@@            Coverage Diff             @@
##           master    #2114      +/-   ##
==========================================
- Coverage   69.04%   68.98%   -0.06%
==========================================
  Files         162      162
  Lines       15113    15136      +23
==========================================
+ Hits        10435    10442       +7
- Misses       3730     3747      +17
+ Partials      948      947       -1

Impacted Files
Coverage Δ

controller/planner_backlog.go
64.13% <100%> (ø)
:arrow_up:

workitem/workitem_repository.go
68.51% <32%> (-1.57%)
:arrow_down:

controller/workitems.go
77.31% <50%> (-0.7%)
:arrow_down:

controller/workitem.go
79.41% <0%> (+0.62%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4fe69ff...059cbf3. Read the comment docs.

/run test-keeper

Thanks @jarifibrahim & @DhritiShikhar for the review!


Add version: UniversityOfAmsterdam.praat version 6.4.09

Pull request has been created with komac v2.2.1 :rocket:

For moderators
This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！

Hi @Exorcism0666,
The package didn't pass a Defender or similar security scan. This might be a false positive and we can rescan tomorrow.
(Automated response - build 877.)

Automatic Validation ended with:

Installation failed with exit code -1978335187
2024-04-20 18:40:27.860 [FAIL] Installer failed security check. Url: https://github.com/praat/praat/releases/download/v6.4.09/praat6409_win-intel32.zip Result: 0x80004005

(Automated response - build 877.)

@wingetbot run


Remove Bean Validation module

This module has been a maintenance burden over the years and during each major release. Therefore, it will not be released starting from v6.
If someone is interested in maintaining it as an extension, please show interest by adding a comment here and I will mention it in the extensions section of the README.
Otherwise, the code of the module can be copied in projects that are interested in using it. After all, that module can be used as an extension by just adding the BeanValidationRandomizerRegistry (+ the required annotation handlers) to the classpath and it will be automatically discovered and used by ER.

Hi, sorry, first time here.
We have been using easy-random 5 with bean validation.
Since we upgraded our project nos we need it to work with the "jakarta" annotations.
I read your instructions in the comment above.
So, where is actually the code for the bean validation module ?


Add support for FMC/TAM stencils

Describe the solution you'd like
I'd like drawio to support FMC and TAM stencils - at least block diagrams.
Most of the shapes are already available, so only a few new shapes (such as read/write access, request/response channel, etc) need to be implemented and then grouped together in their own tab in the "Shapes" window
Additional context
Resources:

Visio FMC stencils
Vision TAM stencils
Reference sheets: http://www.fmc-modeling.org/more2read

Have you tried dropping the vss files into the editor?

Yes, but it resulted in the message "Service unavailable or blocked".

Where are you using draw.io, at app.diagrams.net?

Latest desktop app on Mac OS

Desktop doesn't support vss, use the online version. Please in future fill in the details in the template issue report, otherwise it takes time to ask them later in the thread.


[BugFix] fix project prune rule

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #13205
Fixes #13260
Problem Summary(Required) ：

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

@Mergifyio backport branch-2.4

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_fe_unittest

[FE PR Coverage Check]
:heart_eyes: pass : 10 / 10 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/optimizer/rule/transformation/PruneProjectColumnsRule.java
10
10
100.00%
[]

@Mergifyio backport branch-2.5

@Mergifyio backport branch-2.4 branch-2.5

run pr-title-check

run starrocks_admit_test

run starrocks_admit_test


Feat/list components

NOTE: This draft PR is intended to facilitate discussion about the core architecture for our list component use cases. It is not intended to be a polished or even a feature complete implementation. Both in the code comments and in PR comments, there are callouts for points of discussion, tradeoffs, and the like. Any design will need to meet certain A11Y prerequisites. In this implementation, I've gone with a listbox implementation that includes search (per WAI recommendations) and a distinction between "selected" vs. "active", particularly for keyboard navigation, as it would be a bad user experience to e.g. switch rendition qualities, alternative languages, or subtitles, while simply trying to navigate the options.
The intention is to have a reusable and generic component, similar to e.g. MediaChromeButton, but for lists. I've included MediaCaptionsList as an example usage/extension of the generic component. Some of the discussion points will likely be on the complexities specific to our data modeling/updates for captions+subtitles within this architecture.
The list component and its descendants are unlikely to be used directly by (non-advanced) Media Chrome users. Rather, they will be sub-components of, e.g. a Settings Dialog, Flyaway menu, or similar. Plausible use cases for different data categories:

Subtitles/Captions Selection (included in the example)
Rendition "Quality" Selection
Alternate Audio (e.g. "dubs") Selection
Others?

Below is a design example of a plausible use case for these sub-components:

I need...so much more context. Can you writeup an example guide for how a media chrome user would use this feature?

I need...so much more context. Can you writeup an example guide for how a media chrome user would use this feature?

@heff This is unlikely to be used directly by any non-advanced Media Chrome user. Rather, it'll be the core component for things like a settings overlay/dialog and captions flyaway menus (though there's a bit of discovery we'll have to do on whether or not we'll need to instrumentalize two different aria role configurations. It's possible we'd have a settings overlay/dialog implementation with slots that would result in less advanced users using this, but that's out of scope for this discussion imo.

I think I have my head around the context now, thanks.
A major design decision I'm seeing is that lists auto-render themselves, compared to an example like  <select> or <ul> where it's still the responsibility of the application to build the list markup. Are there specific reasons why these lists need to build themselves? With it all in the shadow dom it does then require things like custom renderers, whereas if the elements were in the light dom they could be customized as you would other elements.
I assume these aren't the only menus media chrome users will want to build in the context of the player, and not all of the menus may be driven by media data, so it does feel like we should start with the lower level components (list/menu container, list/menu item) and build up from there.

I think I have my head around the context now, thanks.
A major design decision I'm seeing is that lists auto-render themselves, compared to an example like <select> or <ul> where it's still the responsibility of the application to build the list markup. Are there specific reasons why these lists need to build themselves? With it all in the shadow dom it does then require things like custom renderers, whereas if the elements were in the light dom they could be customized as you would other elements.

The particular cases this is trying to solve generically would require a non-trivial amount of effort on the part of the integrator. Consider the three callout examples provided in the description:

Subtitles/Captions Selection (included in the example)
Rendition "Quality" Selection
Alternate Audio (e.g. "dubs") Selection

When working with HTTP Adaptive Streaming ("HAS") standards like HLS/MMPEG-DASH, these are going to be described in and provided as part of the media source via the manifest/multivariant playlist. This means that, even in the easy case:

They won't be provided in advance of the src, and are asynchronously added to the media
They come in as "data"
"Selecting" between them is tightly coupled to their definition and is also tied to data.

While it's possible this could be managed at the application level,  these feel like good candidates for a data driven approach. If you can provide an example that wouldn't be especially complicated where an implementor can solve these problems reliably and generically, I could certainly be convinced otherwise. I'm just not seeing any myself.

I assume these aren't the only menus media chrome users will want to build in the context of the player, and not all of the menus may be driven by media data, so it does feel like we should start with the lower level components (list/menu container, list/menu item) and build up from there.

Here, I didn't want to "muddy the waters." By far the biggest ask we've gotten related to these things in both mux player and media chrome are a "quality selector" of some sort and a "captions/subtitles" selector. These are also on our player roadmap. So, rather than trying to solve both (1) the declarative/DOM use case (which is how I'd describe your callout here and I agree we definitely will want) and (2) the "data-driven" use case (as described above and what this draft PR spike is exploring), I went with trying to solve the narrower problem of just (2) based on the priorities.
I believe my current spike solution could be updated without too much difficulty so that the it could share a "common core" architecture/component set for both (1) and (2), but I wanted to avoid implementing to that goal (while still anticipating that in how the components are built). This is partly because was less obvious that we'd need custom web components for that just yet (rather than simply using e.g. <ul> directly, partly because it isn't obvious to me that we'd want them to be built on the same components, and partly because that would slow down progress on the goals here.
That said, this PR is intentionally a draft for discussion, and it intentionally hasn't solved most of the "nitty gritty UI" details. I can definitely see the final non-draft PR benefiting from custom components that solve those problems, and in a way that would at least bring this closer to solving (parts of?) (1).

So, rather than trying to solve both (1) the declarative/DOM use case...I went with trying to solve the narrower problem of just (2) based on the priorities

I definitely appreciate the desire to narrow the scope, but inherent in this task is creating the first "menu item" in the world of media chrome. At some point someone is going to want to set a lot of variables like --menu-item-background: #ccc, and have it impact every menu, regardless of how it's generated. And non-data-driven menus are going to need all the same a11y. So while we don't have to jump right to solving every use case with this first project, we should have the understanding that the declarative (or custom menu) use case will ultimately use the same components for building a menu, not be a completely separate thing.
So then I do question why not solve the declarative use case as part of this? How much harder is it to define a "menu"-like component and "menu-item"-like component that can be used independently, and that the data-driven list generation is built on top of?

If you can provide an example that wouldn't be especially complicated where an implementor can solve these problems reliably and generically, I could certainly be convinced otherwise.

I'm not exactly following your concern, so I'm just gonna jump to how I'd expect this to be implemented otherwise, and you can poke holes in it. I'm gonna use the term "menu" because it feels clearest for the example, but I know you have concerns with that term.
Starting with 3 formal components:

media-captions-menu-button
media-chrome-menu
media-chrome-menu-item

With those three components, there's not really a need for a special MediaCaptionsList component, the captions menu button can internally build the menu from the generic menu components. That would at least prevent some duplicate processing of the captions/subtitles attributes by both the button and a special menu. Starting there, what issues do you see?
I think I'm seeing it like we have these needs:

We need the core UI components to build a menu in media chrome (i.e. menu box, menu item)
Depending on how those work, we might need a way to make it easy to generate the items of a menu from an array of data. (It's not particularly hard to generate a list of <option>s for a <select> menu. Can we make our menu components that easy?)

It feels like we're jumping to number 2, but we're not actually avoiding number 1, and if we're intentional about the primitives of number 1 it could change what we need in number 2. Does that makes sense?

@gkatsev closing this since it's only here for reference at this point. Will keep branch alive though. @ me if you think it would be more convenient for it to continue living and I'll re-open.


Added @Jakob3xD as a co-maintainer.

Description
The project maintainers are inviting @Jakob3xD to opensearch-go maintainers, and they kindly accepted.
Jacob just diligently worked through two significantly complex changes in the client, both breaking backwards compatibility. I think his code quality is very high, I really appreciate his patience, too.

https://github.com/opensearch-project/opensearch-go/pull/246
https://github.com/opensearch-project/opensearch-go/pull/237
https://github.com/opensearch-project/opensearch-go/pull/253

I'm moving @svencowart to emeritus, have not been able to reach them.
Sorted names alphabetically, formatted markdown.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dblock I merged too quickly. Can you please add @Jakob3xD to the repo permissions? And CODEOWNERS also needs to be updated.


repo/support for Avi-Vantage package

The service doesn't come up... it is forever in "waiting" in marathon

@ksivask thoughts?

@ksivask please rebase when you are ready


[Dev UI] Persist Auth JSON accros restarts / recompilation

For example, when setting up a watcher e.g.) npx genkit start -- tsx --watch src/index.ts developer should be able to iterate on flows without resetting the data in the UI.
See also: https://discord.com/channels/1255578482214305893/1306933277189341184
TBD - Beyond auth JSON, what else should we be doing in terms of persistence of ui inputs.

Duplicate #1324


update postgres and timescale version and set pg_prometheus version in

This PR changes the Dockerfile to use postgres 10.4 and timescale 0.10.0. It also uses PG_PROMETHEUS_VERSION and sets it in the control file

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

:+1:


Feature/browser db

This PR introduces MongoDB to store information about running browser processes.
Adding the DB instead of keeping browsers in memory triggered a few design changes in order to keep the application testable:
Previous (simplified) Design:
classDiagram
WorkspaceEndpoint "1" o-- "0..*" OcrdBrowser
OcrdBrowserFactory --> OcrdBrowser: creates
WorkspaceEndpoint --> OcrdBrowserFactory

class OcrdBrowser {
  start()
  stop()
}

New Design:
classDiagram
WorkspaceEndpoint "1" o-- "0..*" OcrdBrowser
WorkspaceEndpoint -- BrowserProcessRepository
OcrdBrowserFactory --> OcrdBrowser: creates and *starts*
BrowserProcessRepository --> RestoringOcrdBrowserFactory
RestoringOcrdBrowserFactory --> OcrdBrowser: restores
WorkspaceEndpoint --> OcrdBrowserFactory

class OcrdBrowser {
  void stop()
}

class BrowserProcessRepository {
<<protocol>>
  insert(OcrdBrowser)
  delete(OcrdBrowser)
  find(owner: str, workspace: str): list[OcrdBrowser]
}

There are a few things that still need improvement:

[x] Replace the current solution for assigning ports with a simple try... except that tries port numbers until we find one that is available
[x] Persisting browser processes in a DB means we need to verify that all stored browsers actually still available. This can either happen by running a clean up action when the monitor starts or by checking if a stored browser is still available when deciding whether to use an existing browser or starting a new
[ ] The additional design work that was necessary to keep things testable made the test setup more complicated. I think there is potential to find a simpler solution again.

I noticed a regression when re-opening a browser after closing the GTK window. I'm not sure why the tests didn't catch it, I'll have to investigate that.

I have fixed the issue with restarting browsers. It turned out to be a missing try:...except: in the stop() method.
There is additional work that can be done to simplify the test setup, however, I can address that in a separate PR or just merge it myself later on.
In my opinion, this PR is ready to be reviewed and merged now @markusweigelt @bertsky

I still experience another problem: Browser connections apparently are not independent of each other, neither for different owners (i.e. browser clients) nor for different workspaces. Here's my db:

_id
address
owner
process_id
workspace

64918e50956bd16e73cbc834
http://localhost:9000
edf609de-c148-4078-9be5-d8074138c014
23
/data/ocr-d/3

64918e5f956bd16e73cbc835
http://localhost:9000
edf609de-c148-4078-9be5-d8074138c014
68
/data/ocr-d/data/3

64918e82956bd16e73cbc836
http://localhost:9000
e455a65e-d4f4-4086-8d3e-442c99198984
111
/data/ocr-d/data/3

(wow, that worked via d&d out of the box!)
So, whenever I switch to another tab/window, all existing tabs/windows say connection lost.
On the logs, it always just says
INFO:     ('<IP_ADDRESS>', 46748) - "WebSocket /workspaces/view/3/socket" [accepted]
INFO:     connection open
INFO:     connection closed
ERROR:root:
                An exception occurred during communication with the browser <ocrdbrowser._subprocess.SubProcessOcrdBrowser object at 0x7f4100d3ffd0>.
                The exception was WebSocketDisconnect(1001)

Wow, that's quite a serious bug, I'll come up with a test for that, so it doesn't happen again

When doing make test, initially (in the first run), the native browser deployer had two failures:
FAILED tests/ocrdbrowser/test_browser_launch.py::test__launching_on_an_allocated_port__raises_unavailable_port_error[SubProcessOcrdBrowserFactory]
FAILED tests/ocrdbrowser/test_browser_launch.py::test__one_port_allocated__launches_on_next_available[SubProcessOcrdBrowserFactory]

Just tried again (after a docker compose build) as pytest -vv tests in a native Py3.11 venv, with both a fresh ocrd-browser:latest Docker image and native browse-ocrd in the active venv. I now see 4 failures:
FAILED tests/ocrdbrowser/test_browser_launch.py::test__launching_on_an_allocated_port__raises_unavailable_port_error[create_browser_factory0] - ocrdbrowser._port.NoPortsAvailableError
FAILED tests/ocrdbrowser/test_browser_launch.py::test__launching_on_an_allocated_port__raises_unavailable_port_error[SubProcessOcrdBrowserFactory] - Failed: DID NOT RAISE <class 'ocrdbrowser._port.NoPortsAvailableError'>
FAILED tests/ocrdbrowser/test_browser_launch.py::test__one_port_allocated__launches_on_next_available[create_browser_factory0] - ocrdbrowser._port.NoPortsAvailableError
FAILED tests/ocrdbrowser/test_browser_launch.py::test__one_port_allocated__launches_on_next_available[SubProcessOcrdBrowserFactory] - AssertionError: assert 'http://localhost:9000' == 'http://localhost:9001'

They are all related to the subprocess mode.  – I guess somewhere we forgot to close/kill the browser as part of the teardown.

Also, why is there no stop_browsers for native mode?

I noticed the same thing, but it does not seem to be the source of the errors.
I added some more logging and during the tests I'm getting this output from browse-ocrd
17:16:16.406 INFO ocrd_browser.util.config.SettingsFactory.build_from_files - Read 0 config file(s):
tried /etc/xdg/ocrd-browser.conf
/root/.config/ocrd-browser.conf
/workspaces/ocrd_monitor/ocrd-browser.conf
Bus error

Do you know what could cause that @bertsky ?

I'm getting this output from browse-ocrd
17:16:16.406 INFO ocrd_browser.util.config.SettingsFactory.build_from_files - Read 0 config file(s):
tried /etc/xdg/ocrd-browser.conf
/root/.config/ocrd-browser.conf
/workspaces/ocrd_monitor/ocrd-browser.conf

This is from here, it just means that no configuration file could be found (not a problem, this is optional and can be used to set default fileGrps etc).

Bus error

I guess this must be a problem with DBus communication.

I added some more logging and during the tests

Could you please post or push these?

I changed some stuff in the SubProcessOcrdBrowserFactory and the launch tests for the native browsers are passing now. Sometimes I get a warning that the event loop is closed. I think it may be related to this: https://bugs.python.org/issue41320
However, the tests are passing despite that.

Unfortunately, I wasn't able to fix the issues with launching browsers so far. Since I'll be gone for the next 2+ weeks here is everything I know and have tried to fix the issues, in case anybody would like to try their hand.
Initial work
First off, since both docker and native browsers used a similar for loop to iterate over possible ports, I have introduced the try_bind() function (in ocrdbrowser/_port.py) that takes care of the looping part. It accepts a function as a parameter that tries to launch some application on a given port. If this function returns the launched application we return, otherwise we move on to the next port.
class PortBindingError(RuntimeError):
    pass

PortBindingResult = Union[T, PortBindingError]
PortBinding = Callable[[str, int], Awaitable[PortBindingResult[T]]]

class BoundPort(NamedTuple, Generic[T]):
    bound_app: T
    port: int

async def try_bind(
    binding: PortBinding[T], host: str, ports: Iterable[int]
) -> BoundPort[T]:
    for port in ports:
        result = await binding(host, port)
        if isinstance(result, PortBindingError):
            logging.info(f"Port {port} already in use, continuing to next port")
            continue

        return BoundPort(result, port)

    raise NoPortsAvailableError()

Issues with the native browser
The broadway server will launch a process that runs continuously and will only print something to stderr if the port is already in use.
My approach is to read from stderr after launching the browser with a timeout of 5 seconds. If this timeout expires, I assume that broadway has been launched successfully, otherwise we return a PortBindingError. This does seem to work in detecting when broadway exits and correctly launches new browsers on new ports (so it fixes the problem @bertsky experienced), but it turns out that the newly launched window will also appear on all existing browsers as well.
process = await asyncio.create_subprocess_shell(
    # broadway cmd here, shortened for readability,
    "broadway ...",
    stderr=asyncio.subprocess.PIPE,
)

try:
    stderr = cast(asyncio.StreamReader, process.stderr)
    err_output = await asyncio.wait_for(stderr.readline(), 5)
    if b"Address already in use" in err_output:
        return PortBindingError()
except asyncio.TimeoutError:
    logging.info(
        f"The process didn't exit within the given timeout. Assuming browser on port {port} launched successfully"
    )

return process

Duplicated windows
On the left we have the browser on port 9000, the right one runs on 9001. The second window on the left appeared when the browser on the right was launched.

Failing tests in GitHub Actions
I am not sure why the tests are failing in GitHub Actions. They have been consistently working from inside my development Docker container. Maybe this is the failure @bertsky experienced that couldn't be reproduced?

Since PR #30 is based on this one and has already many improvements on top of that, I'm closing this PR. We'll get everything in here when we merge #30


add model assertion that two models are related

solves #1
\Astrotomic\PhpunitAssertions\Laravel\ModelAssertions::assertRelated($post, $comment, 'comments');
\Astrotomic\PhpunitAssertions\Laravel\ModelAssertions::assertRelationship(
    $post,
    \App\Models\Comment::class,
    'comments',
    \Illuminate\Database\Eloquent\Relations\HasMany::class
);

@aidan-casey sorry for the long delay.
I've updated the assertion to be able to handle all of our use-cases.
Do you want to check it if it's now doing what you wanted?^^


Which filter method used in k4a_calibration_2d_to_3d when unproject a depth image to point cloud

The k4a kinect does a very good point cloud. At first, I think it is due to the good depth sensor. However, when I do unprojection by myself after getting the depth map, the result has lots of noise. I read the sdk source code and find that kinect use a filter method to do that. But the source code also does an undistortion at the same time. I am confused that I can't recognize the filter method. Therefore I wonder which filter method is used in kinect sdk.
Thanks.

@YuXinFan please check out this page: https://docs.microsoft.com/en-us/azure/Kinect-dk/use-image-transformation, recommend to use (k4a_transformation_depth_image_to_point_cloud) https://microsoft.github.io/Azure-Kinect-Sensor-SDK/master/group___functions_ga7385eb4beb9d8892e8a88cf4feb3be70.html#ga7385eb4beb9d8892e8a88cf4feb3be70 to generate point cloud. The function implementation is in the SDK source code. You can also look at the transformation example: https://github.com/microsoft/Azure-Kinect-Sensor-SDK/tree/develop/examples/transformation if you are interested in color point cloud.
The Azure Kinect viewer's source code: https://github.com/microsoft/Azure-Kinect-Sensor-SDK/tree/develop/tools/k4aviewer is also a good reference of how the out of box visualization tool generate the point cloud (no post processing from viewer, straight render from the SDK)

@rabbitdaxi Thanks for your replay.
Use k4a_transformation_depth_image_to_point_cloud is a good idea. But I have to do unproject out of the real-time k4a program. This function needs a parameter k4a_calibration_t. If I use k4a_transformation_depth_image_to_point_cloud function to do this, I need the struct k4a_calibration_t. But I have no idea about how to store k4a_calibration_t in a file so that another program can load the calibrations from the file.

@YuXinFan if you do not need live streaming, the API should be able to enable you to do playback as well. For example, you can use k4arecorder to record mkv and playback it. In the transformation example, you still can see there is a playback option that you can get k4a_calibration_t from the mkv file.
If you have your customized reason that need to store k4a_calibration_t as binary file and load the binary buffer somewhere else, that should also work.
Please let us know whether we answered your question. If not, please provide details of what you are trying to do and what the current issue you are having (e.g. sample images).
As a side note, if you are asking whether we have post processing from the 2d depth to 3d point cloud to filter noise, the answer is not really, you can look at the implementation of the k4a_transformation_depth_image_to_point_cloud, we basically just unproject the 2d depth to 3d with the brownconrady intrinsics model

@rabbitdaxi Thanks for your answer. But it does not really solve my problem. The problem I met is that after I get depth images and intrinsics and distortion parameters from the kinect dk and try to get the point cloud of the depth image, I apply undistortation on depth image using cv::initDistortationMap and cv::remap with parameters k1,k2,k3,k4. Then calculate 3d position as *x=(u-cx)/fx * d, y = (v-cy)/fy * d, z = d, also filter the points with zero positions. However, the result point cloud has lots of noise on the lines from the edge of objects to the camera focus, unlike the point cloud of sdk viewer tool.
To solve this problem, I find the source code of sdk does an iterative unproject operation for each point when doing unproject,  which does re-projection and compares some error many times. It seems that this function is a filter to improve the result of unprojection. It likes compares re-projection and raw point errors, but I am not sure.
The source code of kinect viewer also uses this method to convert depth image to point cloud.
I think to apply this filter in the source code on my result may be much better. Therefore, I wonder what the function transformation_iterative_unproject really does. And dose my simply undistortation operation matter?

@YuXinFan Thank you for explaining what you are trying to do. I think I am able to provide some pointers to help you understand the calibration and geometry.

k4a_transformation_depth_image_to_point_cloud is still my recommended method for generating point cloud from the depth image, especially we have LuT+SSE implementation of this function which has fast real-time performance. It does not contain any post filter processing. The *transformation_iterative_unproject that compute the 2d to 3d ray LuT implements a common approach that finding correspondence from 2d to 3d with optimization on the re-projection error term (internally compute a Jacobian). This method is operating on the depth image straight from the SDK (distorted), and the project/unproject function internally take into account of the radial and tangential distortion parameters.
Regarding your question about all the intrinsics parameters meaning, please read API Doc, you also can find an simple example of opencv compatibility code snippet that shows how k4a intrinsics and opencv can be connected. The camera model is BrownConrady which is one of supported model by OpenCV (projectpoints can be directly used)
Regarding what your current approach that try to undistort the depth image and then use the simple pinhole camera model to unproject to 3d. The noise could be introduced by your undistort function if you have the cv::remap contains the interpolation method with INTER_LINEAR, is that what you have?

I still recommend you read our documentation. And k4a_transformation_depth_image_to_point_cloud exactly what we provided to user to do such task.

@rabbitdaxi Thanks for your patience.  You are right that the noise is mostly caused by the interpolation method. I change INTER_LINEAR to INTER_NEAREST. Then the noise is disappeared.
However, the INTER_NEAREST method makes objects deformation. The result point cloud will lose shape information. For example, my calibration board after INTER_NEAREST undistortion:

I am confused that why not make undistortion an independent part so that we can freely to choose sdk function method with our own function method in each process. I feel these functions' source code implementation method makes it much coupling with other processes.
The point cloud is not my aim. My existing program interface APIs don't want to be changed. I am almost to change the module of camera deriver from the old camera to Kinect. And I don't want this change influence other modules. But I feel hard to let Kinect fit the existing program.

@YuXinFan It makes sense that when you undistort with LINEAR which introduce the flying pixels on the edge (between depth with 0 value and non-0 value). Interpolation with NN will give you the aliasing artifacts as in your image. One method that can help is using bilinear method with handling the 0 depth pixels during computing weight. There is a sample that you can take a look as a reference. https://github.com/microsoft/Azure-Kinect-Samples/blob/master/opencv-kinfu-samples/main.cpp. You can look at the CUSTOM_UNDISTORTION section. Hope it helps

We will update the undistort example with more details.

@rabbitdaxi Thanks for your answers. Your answers help me a lot.

the implementation of the k4a_transformation_depth_image_to_point_cloud
where could I find the  implementation of  the k4a_transformation_depth_image_to_point_cloud?

@1106204321 , the code is open sourced for this function, you can just search the function in k4a.cpp, which it will call into https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/src/transformation/transformation.c

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/588#issuecomment-528482512
thank you very much!

@1106204321 , the code is open sourced for this function, you can just search the function in k4a.cpp, which it will call into https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/src/transformation/transformation.c

thanks

@rabbitdaxi ，Does the SDK provide source code of 'k4a_calibration_2d_to_3d' ?  I couldn't find it.

How to obtain aligned, undistorted RGBD data, in this case the amount of calculation for projection and unprojection is very small.

@1106204321 Please ref:
/Azure-Kinect-Sensor-SDK/include/k4ainternal/transformation.h
/Azure-Kinect-Sensor-SDK/src/transformation/*


Fix inconsistent use of tabs and spaces in indentation.

Causes TabError:
Sorry: TabError: inconsistent use of tabs and spaces in indentation (iwls.py, line 56)
Sorry: TabError: inconsistent use of tabs and spaces in indentation (iwls.py, line 56)

The issue was reported by Andreas Beckmann in Debian Bug #876700

Coverage remained the same at 83.766% when pulling 46eb0c06c0d1cfb78f2e05548a28e55fc01f12a7 on sebastic:taberror into 8d60c06e5505fdc3017ed5196ae84d91e091e887 on pysal:master.

resolves #981

@TaylorOshan does this look okay?

Yes! Looks great. Sorry for the delay, thought this was already merged.


collection.add_row() fails on new databases

Looks like the latest Notion updates have broken even the newest forks of this project. collection.add_row() works fine on existing databases, but if you create a new database, collection comes back as NoneType.

Unsure what changed in the API, however after inspecting how the module is storing information about collections, I noted the collection_id is available via a different key. The linked patch works for me but I've only done very light testing. Care to give it a try?

@knightjoel Your fix works to solve the issue.

I'm facing the same error. Can anyone share the installation # similar to below
https://github.com/jamalex/notion-py.git@refs/pull/352/merge


Problem set 3 exercise 3 - Class 2

Hello Emil,
I've tried to run problem 3 in the exercises, but I can't load the data, I keep getting an error:
No such file or directory: 'data/NAH1_pivoted.xlsx'
I have tried to use the code in the answers :
nah1 = pd.read_excel('data/NAH1_pivoted.xlsx',skiprows=2)
I use jupyter lab and have also tried to change the "data/" with "numerical/ugesedler" which is where my data is in jupyter lab, but it doesn't help.
I hope you can help.

Hi Frederikke,
The error you receive tells you that python cannot find the file you are trying to load.
A simple solution could be to define a new variable containing the full path to the file. For example something like this:
folder_path = 'C:\\Users\\pdj165\\Documents\\TA\\NumEconCopenhagen\\'
file_name   = 'NAH1_pivoted.xlsx'

df_nah1 = pd.read_excel(folder_path + file_name)

Please notice that I add two backslashes instead of one. You can check out why I do this, and other ways of doing it in this stackoverflow issue: https://stackoverflow.com/questions/2953834/windows-path-in-python
Let me know if this solved your problem?

Yes that worked :)
Thank you!


Capture screenshots in web tests

Codecov Report

Merging #10875 (5de9d32) into main (333aa9b) will decrease coverage by 0%.
The diff coverage is n/a.

:exclamation: Current head 5de9d32 differs from pull request most recent head aa56201. Consider uploading reports for the commit aa56201 to get more accurate results

@@          Coverage Diff           @@
##            main   #10875   +/-   ##
======================================
- Coverage     52%      52%   -1%
======================================
  Files        486      486
  Lines      33791    33791
  Branches    5502     5502
======================================
- Hits       17707    17703    -4
- Misses     14385    14388    +3
- Partials    1699     1700    +1

Impacted Files
Coverage Δ

...intellisense/fileBasedCancellationStrategy.node.ts
61% <0%> (-10%)
:arrow_down:

src/platform/msrCrypto/msrCrypto.js
16% <0%> (-1%)
:arrow_down:

src/kernels/execution/helpers.ts
37% <0%> (+<1%)
:arrow_up:

src/standalone/api/kernelApi.ts
73% <0%> (+1%)
:arrow_up:


[Feature] Implement Google search

There are some project which include google search by implementing custom search api and custom search engine id. Is a google search anyhow possible at the moment? I´ven´t found any info on that. Would appreciate your feedback.

Unofficial support: #2146
Official support is in plan, stay tuned: #165


[$500] [MEDIUM] Scan - No thumbnail is shown for the corrupted file in receipt preview

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: v1.4.44-8
Reproducible in staging?: Y
Reproducible in production?: Y
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause-Internal Team
Action Performed:

Go to staging.new.expensify.com.
Go to workspace chat.
Go to + > Request money > Scan.
Upload a corrupted file (attached below).
Proceed to confirmation page.

Expected Result:
The confirmation page will show thumbnail for the corrupted file.
Actual Result:
The confirmation page shows blank area for the corrupted file.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01ddfcfb2a8a486a23
Upwork Job ID:<PHONE_NUMBER>178574336
Last Price Increase: 2024-03-30

@greg-schroeder I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors.

We think that this bug might be related to #wave5-free-submitters
CC @dylanexpensify

The corrupted file used to create this issue seems to be a very small image, only 16 x 16 pixels. A thumbnail of this image might appear blank, possibly. The corrupted file I am using works as expected. I see a thumbnail of it.

proposals are in review

eta Monday

Proposal review

@Expensify/design do we agree to show thumbnail when attached receipt is not previewable (i.e. corrupted image/pdf)?

I think showing a fallback image makes sense—but I'm not sure I would show that type of thumbnail. I feel like it kinda makes more sense to show our little fallback image thingy:

What do the other designers think?

Agree with Shawn who agrees with Danny

Thanks for feedback.
Please update proposals.

@Krishna2323 the expected behavior should be discussed with design team.
Why don't we just use this GH if we don't allow corrupted images? I see the the linked issue created just today with no context.

We're going to prevent uploading corrupted PDF as scan receipt in #34032.
So to be consistent, we should not allow corrupted images.
cc: @Expensify/design to confirm the expected behavior.

^ is only when we're able to detect if file is corrupted or not.
For no-preview files we cannot detect if corrupted or not, I think it still makes sense to show fallback image like this.

We're going to prevent uploading corrupted PDF as scan receipt in https://github.com/Expensify/App/issues/34032.
So to be consistent, we should not allow corrupted images.

@situchan that makes sense to me.

For no-preview files we cannot detect if corrupted or not, I think it still makes sense to show fallback image like https://github.com/Expensify/App/issues/37435#issuecomment-1995579903.

Agree with this as well!

Okay so based on above comments, can you give me a quick update on where this one stands @situchan? Are we still waiting for proposals or something else?

We're still waiting for proposals.

@situchan is out OOO, reassigning a C+ via the auto-assigner for fairness.

Show the fallback thumbnail for files that do not support preview (@dannymcclain do you have any ideas what file types might fit into this category?)

😬 I honestly do not. Going to tag @shawnborton and @trjExpensify to see if they have any ideas or might know who would know.

I think this comment is helpful for that question

Proposal
Please re-state the problem that we are trying to solve in this issue.
In scan request upload corrupted image file, the user is allowed to upload corrupted and no thumbnail preview is displayed on confirmation page.
What is the root cause of that problem?
In the IOURequestStepScan step after user selects file, need to add step to handle image loading errors. The image can be checked using RNImage.prefetch api.
https://github.com/Expensify/App/blob/8300ae40dd1f882167162fa7c86d0f5152000a87/src/pages/iou/request/step/IOURequestStepScan/index.js#L178
What changes do you think we should make in order to solve the problem?
The validateReceipt function needs to be converted to callback. Add useState variable isReceiptValid
const [isReceiptValid, setIsReceiptValid] = useState(false);

The validateReceipt function after conversion to useCallback and using RNImage.prefetch api to determine if the image is valid.
    const validateReceipt = useCallback((file) => {
        const {fileExtension} = FileUtils.splitExtensionFromFileName(lodashGet(file, 'name', ''));

        if (!CONST.API_ATTACHMENT_VALIDATIONS.ALLOWED_RECEIPT_EXTENSIONS.includes(fileExtension.toLowerCase())) {
            setUploadReceiptError(true, 'attachmentPicker.wrongFileType', 'attachmentPicker.notAllowedExtension');
            setIsReceiptValid(false);
        }

        if (lodashGet(file, 'size', 0) > CONST.API_ATTACHMENT_VALIDATIONS.MAX_SIZE) {
            setUploadReceiptError(true, 'attachmentPicker.attachmentTooLarge', 'attachmentPicker.sizeExceeded');
            setIsReceiptValid(false);
        }

        if (lodashGet(file, 'size', 0) < CONST.API_ATTACHMENT_VALIDATIONS.MIN_SIZE) {
            setUploadReceiptError(true, 'attachmentPicker.attachmentTooSmall', 'attachmentPicker.sizeNotMet');
            setIsReceiptValid(false);
        }
        RNImage.prefetch(file.uri).then(()=>{{
            fileRef.current = file;
            setIsReceiptValid(true);
        }}).catch((error) => {
            setUploadReceiptError(true, 'attachmentPicker.errorLoadingAttachment', 'attachmentPicker.invalidAttachment');
            setIsReceiptValid(false);
        })
    },[isReceiptValid])

The setReceiptAndNavigate function is called after the user picks the file and also needs to be converted to useCallback. The callback would only call the validateReceipt and the remaning steps to go to confirmation step can be shifted in useEffect.
https://github.com/Expensify/App/blob/8300ae40dd1f882167162fa7c86d0f5152000a87/src/pages/iou/request/step/IOURequestStepScan/index.js#L232
    const setReceiptAndNavigate = useCallback((file) => {
        validateReceipt(file);
    },[isReceiptValid]);

The useEffect has dependancy on isReceiptValid and proceeds to confirmation step if the image is valid.
    useEffect(()=>{
        if (isReceiptValid) {
            // Store the receipt on the transaction object in Onyx
            const source = fileRef.current.uri;
            IOU.setMoneyRequestReceipt(transactionID, source, fileRef.current.name, action !== CONST.IOU.ACTION.EDIT);

            if (action === CONST.IOU.ACTION.EDIT) {
                updateScanAndNavigate(fileRef.current, source);
                return;
            }

            navigateToConfirmationStep();
        }
    },[isReceiptValid])

What alternative solutions did you explore? (Optional)

Prevent from using corrupted images for Scan requests;
Show the fallback thumbnail for files that do not support preview

@paultsimura I have covered pt 1, for pt 2 if going by https://github.com/Expensify/App/issues/37435#issuecomment-2032468860. I checked on staging and the fallback thumbnail is already displayed.

Do you think we need an explicit icon that looks like a broken image? Like something like this?

Or maybe I am not quite understanding what you are thinking here.

That also works for me. Wasn't sure what you meant by "FYI, we still don't have any fallback thumbnail for corrupted images."

FYI, we still don't have any fallback thumbnail for corrupted images.

@paultsimura @shawnborton Please confirm, the corrupted images would be prevented from uploading in scan requests and the fallback thumbnails would only be for files which do not support previews e.g. doc, html, xml etc. Please confirm if corrupted images are also to show fallback thumbnails or if fallback thumbnails are only for files which do not support previews.

Sorry I haven't been following along too closely to the original issue - but I guess now that you spell it out that way, should we even let you continue with a corrupted image? What even is a "corrupted" image? Basically if we detect upon upload that the image is corrupted, I would think we'd throw an error.

cc @Expensify/design for other thoughts here too

Yeah good point. Can we detect if an image is corrupt? If so, I also think we should throw an error as early as possible instead of getting to a state where it tries to scan.

Prevent from using corrupted images for Scan requests;
Show the fallback thumbnail for files that do not support preview
@shawnborton @dubielzyk-expensify I understood, the user should be prevented from continuing with corrupted images, makes sense to prevent user from continuing when image is corrupted and use fallback thumbnails for files which do not support previews (doc, html , xml etc)
In the proposal, I am preventing the user from continuing if the image is corrupt.

@Expensify/design please just confirm what would we prefer for the corrupted images:
Show an alerting modal that says "The attachment upload failed" (this doesn't look like a common approach to work with errors in the App)
Show the error on the initial page of the receipt upload (if so – please specify the placement of the error)
Proceed to the confirmation screen, but show the thumbnail and error on it
Any other options?

I don't think we should wait until the confirmation screen—then you'd have to basically start the entire flow over to fix it. I think it's best to do it on the actual scan step. Like Jon said, as early as possible. Here are two options for that:

Bottom RBR error that seems fairly in line with other errors in the app
Center modal - this doesn't quite seem like our normal pattern, but it does get the job done

Keen for @Expensify/design thoughts.

[!NOTE]
We'll probably want some help with the actual error text. I just made something up.

Yup, big fan of Danny's suggestions. When you upload an invalid file type, we use the centered modal error pattern so perhaps we could stay consistent and use that here as well.

Yup, big fan of Danny's suggestions. When you upload an invalid file type, we use the centered modal error pattern so perhaps we could stay consistent and use that here as well.

Oh I didn't realize that but you're totally right. Agree let's just stick with that!
So based on the invalid file type flow:

We should probably do something like this:

@paultsimura @shawnborton @dannymcclain The centered modal error pattern is also used when the file size is too small or too large. I have used the centered pattern.

Dope, I like it!

Show the fallback thumbnail for files that do not support preview;

@dannymcclain @shawnborton
Also please confirm if going with https://github.com/Expensify/App/issues/37435#issuecomment-1995579903, the fallback thumbnails for files which do not support previews can either use app theme background or use color codes based on file extension (https://github.com/Expensify/App/issues/37435#issuecomment-2032468860)
I have poc to demonstrate use of app theme bg and color coded file extension.
fallback thumbnails using color codes
poc01.webm
fallback thumbnails without color codes
poc02.webm

Hmm, I'm torn a bit. Curious what Shawn thinks. I kinda like the color codes? But I'm wondering if it makes sense to use them or not. Using the color codes makes it feel more "filled" which is kinda nice, but it also makes it feel a lot more like an eReceipt (for card transactions) and I don't know if that's a good thing or not.

I'd say no colors but I don't think this issue is about fallback icons anymore is it? That is being handled in a different issue. It seems like now we agree that the crux of this issue is just making sure the user knows the attachment is corrupt and they need to choose a different one.

@shawnborton Thanks.

Prevent from using corrupted images for Scan requests;

@paultsimura I have proposal for handling corrupted images for scan requests. The fallback thumbnails are to be handled in different issue.

This is great. Definitely like the modal as an error and I also prefer no color codes.

Also, it doesn't fix the issue on native platforms

@paultsimura @shawnborton I understood the current issue is only to prevent the user from uploading corrupted attachment and not resolve any other issues. Please elaborate which other issue is supposed to be addressed in the current issue.

and not resolve any other issues

What other issues are you referring to?

@shawnborton I am also trying to find out which other issue @paultsimura is referring to.

I saw your suggestion for the copy:
However, we already have a copy for the corrupted attachment modal (not sure how to trigger the current one though). Would you be ok with it, or should we alter the copy?

If we already have something let's absolutely stick with what's already there!

@paultsimura RNImage.getSize should not be used on android native. The android native uses expo-image library and the RNImage is used for react web component.

@paultsimura The concerned members need to tagged and @shawnborton explained in the https://github.com/Expensify/App/issues/37435#issuecomment-2034986390 that the only issue to be addressed was to prevent the user from uploading corrupt image.

@paultsimura The ImageSize.getsize would only be for android native and not the web. Please check. The expo-image for android exposes prefetch and the react image component for web also has prefetch. I was also first to use the centered error model which fits with the requirements.

@paultsimura The proposal for getSize is incomplete the pseudo-code " /// Block users and show message." is already done in the validateReceipt function call of validateReceipt. I have provide much more complete solution and does not leave any room for pseudo-code expanding to lot more extra effort.

@paultsimura please do point out the overhead, the state variable for isAttachmentInvalid can be utilized but the error message handling is already done in the IOURequestStepScan validateReceipt function call doing the validation in attachment picker would actually adding extra overhead. Please review final time.

@paultsimura Please review, something is not correct with the implementation of ImageSize.getsize as suggested in the accepted proposal.
According to AvatarCropModel.tsx
https://github.com/Expensify/App/blob/8300ae40dd1f882167162fa7c86d0f5152000a87/src/components/AvatarCropModal/AvatarCropModal.tsx#L122
The ImageSize.getSize the success callback is handled in then()
   ImageSize.getSize(imageUri).then(({width, height}) => {
            // We need to have image sizes in shared values to properly calculate position/size/animation
            originalImageHeight.value = height;
            originalImageWidth.value = width;
            setIsImageInitialized(true);

            // Because the reanimated library has some internal optimizations,
            // sometimes when the modal is hidden styles of the image and slider might not be updated.
            // To trigger the update we need to slightly change the following values:
            translateSlider.value += 0.01;
            rotation.value += 360;
        });

but the accepted proposal has suggested without using then.
ImageSize.getSize(
   fileData.fileCopyUri || fileData.uri,
   (width, height) => {
       fileData.width = width;
       fileData.height = height;
       return validateAndCompleteAttachmentSelection(fileData);
   },
   (error) => {
       if (error.message === 'unknown image format') {
           showImageCorruptionAlert();
       } else {
           showGeneralAlert();
       }
   },
);

I tried using ImageSize.getSize without 'then' and the method is not returning success or fail callback
    ImageSize.getSize(file.uri, function (width, height) {
        setFile(file);
        isAttachmentInvalid(true);
      }, function (error) {
        setUploadReceiptError(true, 'attachmentPicker.errorLoadingAttachment', 'attachmentPicker.invalidAttachment');
        isAttachmentInvalid(false);
      });

Ideally the call to getSize should have returned success for valid image and failed for corrupt image.
Also please reconsider the accepted proposal. The react-native-image-size designed only for (ios+android) according to npm package. Even if the avatarcropmodel is using react-native-image-size in the web, does not mean the approach is correct.
I suggest to go with prefetch because the android + ios is using expo-image and the web is using react image component and both have prefetch method.

@paultsimura besides the concern of ios+android, I am finding difficulty using ImageSize.getsize to actually catch corrupt images. The AvatarCropModal is only passing valid images to ImageSize.getSize, the invalid or corrupt images are caught in the call to getImageResolution.

@jsdev2547 Did you try using .catch?
ImageSize.getSize(file.uri)
            .then(({width, height}) => {
                ...
            })
            .catch(() => {
                setUploadReceiptError(true, 'attachmentPicker.attachmentError', 'attachmentPicker.errorWhileSelectingCorruptedImage');
            });

@paultsimura I literally pasted the code block and for valid files, the getsize returns success but then also calls the catch block. Try uploading corrupt image in profile avatar and notice the console error, the ImageSize.getsize atleast for the web currently is only used for valid images and not corrupt.

@paultsimura I do not mean to cause delay but the ImageSize.GetSize seems to lack catch and also is designed for ios+android. Also considering the AvatarCropModal.tsx is only getting valid images, the corrupt images are getting handledin call to getImageResolution. I urge to reconsider and use prefetch api for web and android+ios(expo-image).

@paultsimura The internal engineers can also make suggestions on proposals if needed so as final call, no harm in getting final go ahead from the internal engineer.

Agree with the above statement by @paultsimura - I think we should go with his choice and have @arosiclair validate that choice and then just proceed getting this fixed.

Hold removed, moved back to Daily

Hmm. Sorry, the automation seems to have failed this one. We generally search for payments by issue title + label. I'll get this taken care of now.

Done, apologies.


Update polybase-configure-teradata.md

@pabechevb : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

@pabechevb - thank you for the submission. We are reviewing.

@pabechevb : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Moving to private repo for staging.


Norminal cis-windows for mQTL using TensorQTL

As noted in point 1 here: https://github.com/broadinstitute/tensorqtl/issues/68

there will be >400K CpG sites in the data compared to >20K genes. What would you suggest be a good cis_window for a CpG site, from a computational capacity perspective, for TensorQTL to analyze the data within reasonable CPU hours? 1Mbp radius may be too many tests?

Before this question got answered, this ticket will document some of the cis_windows other people use, although not necessarily using TensorQTL.
Cis_window is the number of bp up/downstream of the CpG  site to determine what SNPs are to be included in the nominal association estimation.

One of the later mQTL paper used 50kb up/down of CpG site as the cis-windows for the nominal association pvalue (NOT the cis_window referred in the title). https://academic.oup.com/hmg/article/29/7/1154/5803132#203353336
This info will be organized into the analysis notebook were we actually use this window.

Another paper, https://www.nature.com/articles/s41598-018-35871-w#Sec7, used a way larger window.

Firstly the cis region to the methylation probe – defined as a window 2 Mbp each side of the target CpG site location – was investigated.

Another earlier paper(2016), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4913906/#sec016,suggests using 500kb:

To distinguish between local (cis) and distant (trans) mQTLs a distance less or equal to 500 kb between a SNP and CpG site was used to define cis-mQTLs. All remaining SNP-CpG pairs were considered trans-mQTLs. In total we found 283,290,917,454 CpG–SNP pairs in the dataset, where 112,842,462 pairs were defined to be located in cis and 283,178,074,992 in trans.

@yuqimiao please post the windows u ultimately decided to use here.

whatever we do, it should not be smaller than the previous ROSMAP xQTL release ...


Add basic profiling support, take 2

Here's a new version of the profiling patch.  It's based on master and separates the branch simplification code into a separate commit.   Please consider applying this.  Notes from old PR follow....
This patch adds basic profiling support. The simulator will record call arc and basic block counts in gprof compatible files.
The basic block counts are imperfect because they only count branch/jmp/jsra/jsra targets as basic blocks at runtime, and will omit the first entry into a basic block through straightline code. gprof basic block counts are usually handled through code instrumentation, which I didn't want to have to require.
The new code increased moxie.cc to over 1k LOC so I simplified the branch code to shrink it back. :)

Hmm... looks like travis is using some ancient g++.  I think the --std=c++11 was introduced in GCC 4.7.  It might work with -std=gnu++0x.  Checking...

Yep - that works.

Merged - broke "make distcheck" - manually fixed that up


Observer does not work with 'New boards hub'

The redesign of the board hub does not work with the observer, but the old one does.
TODO:

Check if contribution ids have changed

Been unable to find a solution to this, as the rule-tester-action still loads with the new hub. Reported and awaiting feeback: https://developercommunity.visualstudio.com/t/Extension-contribution-no-longer-loads-w/1631893

With the new boards hub releasing in the official preview I have yet again reported the problem. Latest status is that they are going to look into it.


Multiple errors on initial test run (newbie needs help)

I'm having this same issue but I think the cause may be different.
I tried running npm run stryker but I got this:
env: node\r: No such file or directory
npm ERR! Darwin 15.6.0
npm ERR! argv "/usr/local/Cellar/node/6.4.0/bin/node" "/usr/local/bin/npm" "run" "stryker"
npm ERR! node v6.4.0
npm ERR! npm  v3.10.3
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT

So I tried node node_modules/stryker/bin/stryker -c stryker.conf.js and still no success:
[2016-08-24 11:28:02.855] [WARN] TestSelectorOrchestrator - Could not find test selector `mocha` (based on the configured testFramework). Stryker will continue without the ability to select individual tests, thus running all test for every generated mutant. Set `testSelector` option explicitly to `null` to ignore this warning. Did you forget to load a plugin? Known test selectors: ["jasmine"].
[2016-08-24 11:28:02.867] [WARN] InputFileResolver - Globbing expression "test/**/*.js" did not result in any files.
[2016-08-24 11:28:02.900] [INFO] InputFileResolver - Found 102 file(s) to be mutated.
[2016-08-24 11:28:03.133] [ERROR] Stryker - One or more tests errored in the initial test run:
        [object Object]
[2016-08-24 11:28:03.133] [ERROR] stryker-cli - an error occurred Error: There were failed tests in the initial test run
    at /Users/PATH_TO_CURRENT_FOLDER/node_modules/stryker/src/Stryker.js:55:23
    at process._tickCallback (internal/process/next_tick.js:103:7)

Any idea what I'm doing wrong here? Help is much appreciated, as I'm trying mutation test for the first time and I wanted to show this Stryker tool to my team. Thanks in advance.

Files:
// package.json
{
  "scripts": {
    (...)
    "stryker": "stryker -c stryker.conf.js"
  },
  "devDependencies": {
    (...)
    "chai": "^3.4.1",
    "mocha": "^2.4.5",
    "sinon": "^1.17.3",
    "stryker": "^0.4.2",
    "stryker-api": "^0.2.0",
    "stryker-html-reporter": "^0.2.0",
    "stryker-mocha-runner": "^0.1.0",
}

// stryker.conf.js
module.exports = function (config) {
  config.set({
    files: [
      // Add your files here, this is just an example:
      {pattern: 'src/test/**/*.js', mutated: true, included: false},
      'test/**/*.js'
    ],
    testRunner: 'mocha',
    testFramework: 'mocha',
    reporter: ['html', 'progress']
  });
};

@dialex thanks for your issue!
Yes, it seems that you are suffering from the same issue as #142 . However, you should still be able to see the errors. Did you try to run stryker with logLevel set to trace? Changing the logLevel is explained here: https://github.com/stryker-mutator/stryker#log-level. Please try to post the underlying cause for the error here so we can help you further.
Sorry for the late response, I will be more active now that i'm back after a vacation.

No success with npm run stryker so I used again the node command. The output is below, I tried to separate it into logical blocks. I can't see any relevant info though 😢
$ node node_modules/stryker/bin/stryker --logLevel trace -c stryker.conf.js

[2016-09-16 17:20:15.657] [DEBUG] ConfigReader - Loading config stryker.conf.js
[2016-09-16 17:20:15.660] [DEBUG] PluginLoader - Loading stryker-* from /Users/dnunes/IdeaProjects/XXXXXX/node_modules
[2016-09-16 17:20:15.663] [DEBUG] PluginLoader - Loading plugins stryker-html-reporter (matched with expression stryker-*)
[2016-09-16 17:20:15.663] [DEBUG] PluginLoader - Loading plugins stryker-mocha-runner (matched with expression stryker-*)
[2016-09-16 17:20:15.663] [DEBUG] PluginLoader - Loading plugins stryker-html-reporter
[2016-09-16 17:20:15.742] [DEBUG] PluginLoader - Loading plugins stryker-mocha-runner
[2016-09-16 17:20:15.769] [DEBUG] Stryker - Using config: {"logLevel":"trace","testFramework":"mocha","testRunner":"mocha","timeoutMs":5000,"timeoutFactor":1.5,"plugins":["stryker-*"],"port":9234,"reporter":["html","progress"],"files":[{"pattern":"src/test/**/*.js","mutated":true,"included":false},"test/**/*.js"],"configFile":"stryker.conf.js"}

[2016-09-16 17:20:15.770] [WARN] TestSelectorOrchestrator - Could not find test selector `mocha` (based on the configured testFramework). Stryker will continue without the ability to select individual tests, thus running all test for every generated mutant. Set `testSelector` option explicitly to `null` to ignore this warning. Did you forget to load a plugin? Known test selectors: ["jasmine"].
[2016-09-16 17:20:15.779] [WARN] InputFileResolver - Globbing expression "test/**/*.js" did not result in any files.

[2016-09-16 17:20:15.803] [INFO] InputFileResolver - Found 108 file(s) to be mutated.
[2016-09-16 17:20:15.803] [DEBUG] TestRunnerOrchestrator - Creating test runner 0 using settings {port: 9234, coverageEnabled: true}

[2016-09-16 17:20:15.929] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:15.925] [DEBUG] PluginLoader - Loading stryker-* from /Users/dnunes/IdeaProjects/XXXXXX/node_modules
[2016-09-16 17:20:15.931] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:15.931] [DEBUG] PluginLoader - Loading plugins stryker-html-reporter (matched with expression stryker-*)
[2016-09-16 17:20:15.931] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:15.931] [DEBUG] PluginLoader - Loading plugins stryker-mocha-runner (matched with expression stryker-*)
[2016-09-16 17:20:15.931] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:15.931] [DEBUG] PluginLoader - Loading plugins stryker-html-reporter
[2016-09-16 17:20:16.012] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:16.012] [DEBUG] PluginLoader - Loading plugins stryker-mocha-runner
[2016-09-16 17:20:16.046] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:16.046] [DEBUG] StrykerMochaReporter - Starting Mocha test run
[2016-09-16 17:20:16.046] [TRACE] IsolatedTestRunnerAdapter - [2016-09-16 17:20:16.046] [DEBUG] StrykerMochaReporter - Mocha test run completed: 0/0 passed

[2016-09-16 17:20:16.049] [ERROR] Stryker - One or more tests errored in the initial test run:
	[object Object]
[2016-09-16 17:20:16.049] [ERROR] stryker-cli - an error occurred Error: There were failed tests in the initial test run
    at /Users/dnunes/IdeaProjects/XXXXXX/node_modules/stryker/src/Stryker.js:55:23
    at process._tickCallback (internal/process/next_tick.js:103:7)

Now it's my time to apologise for the delay!

Sorry to keep you waiting.
I think mocha is reporting test failures, because it discovered zero tests. As you can see Globbing expression "test/**/*.js" did not result in any files.. Are you sure the test folder is not empty? And that you are starting the test from the correct directory. The expression test/**/*.js seems correct to me.

Nice, my bad, I changed the configuration for
{pattern: 'src/test/**/*.js', mutated: true, included: false}, 'src/test/**/*.js'
so now I think I got the config right. (check this image to see the folder structure)
However, doing that only muted the warning you mentioned. Is it because it doesn't recognise my js files as tests? I'm using Mocha, Enzyme, Chai, React, Redux. A typical test looks like this:
describe('Module desc, ', () => {

  it("test desc", () => {
    // Given
    const store = prepareStore();

    // When
    const container = mount(<SomeElement/>);

    // Then
    something.text().should.equal("text goes here");
  });

Don't you find that Could not find test selector 'mocha' suspicious?

A test selector for Stryker is a piece of code that is capable of ensuring that only certain tests are ran.
For example: a code coverage result shows that only test 3, 8 and 42 cover mutant A. If a test selector is present, Stryker will tell it that only those tests have to be executed. If no test selector is present,  all of your tests will be executed for every mutant.
We don't have a test selector for Mocha right now, because the stryker-mocha-runner is not capable of generating code coverage.

Thanks for the insight Simon! So that means that even without Mocha, Stryker should be able to execute all tests for all mutants. That would just take more time but should work. Right now no test/mutator is running at all 😕

@dialex since the 0.5 release of Stryker (with stryker-api 0.4) we changed the api. You now don't use a testSelector, instead you can select the coverageAnalysis explicitly. We also updated the error messages and it is now possible to exclude files (using a ! in the files array, see the stryker readme).
Sorry we didn't respond earlier. We will try to do better from now on.

The version
Well I used npm install stryker stryker-api --save-dev to update my version of stryker, and now my package.json looks like
"stryker": "^0.4.5",
"stryker-api": "^0.2.1",

...shouldn't it be using v0.5.4 (currently the latest on github)?

The good
Finally npm run stryker is returning the same as node node_modules/stryker/bin/stryker -c stryker.conf.js

The bad
I added to the stycker.conf.js file these two lines to match your README example.
coverageAnalysis: 'perTest',
plugins: ['stryker-mocha-runner', 'stryker-html-reporter']

This is part of my current package.json file
"test": "mocha --compilers js:babel-core/register --require ./src/test/test_helper.js 'src/test/**/*.@(js)' --recursive",
"test:watch": "mocha --compilers js:babel-core/register --require ./src/test/test_helper.js 'src/test/**/*.@(js)' --recursive --watch",
"stryker": "stryker -c stryker.conf.js",

Then I ran npm run stryker and got this, which is basically the same issue I always had.
[WARN] TestSelectorOrchestrator - Could not find test selector `mocha` (based on the configured testFramework). Stryker will continue without the ability to select individual tests, thus running all test for every generated mutant. Set `testSelector` option explicitly to `null` to ignore this warning. Did you forget to load a plugin? Known test selectors: ["jasmine"].
[INFO] InputFileResolver - Found 135 file(s) to be mutated.
[INFO] TestRunnerOrchestrator - Starting initial test run. This may take a while.
[ERROR] Stryker - One or more tests errored in the initial test run:
  [object Object]
[ERROR] stryker-cli - an error occurred Error: There were failed tests in the initial test run
    at /Users/dnunes/IdeaProjects/Vocalink/app/ops-console-client/node_modules/stryker/src/Stryker.js:58:23
    at process._tickCallback (internal/process/next_tick.js:103:7)

What now?

Strange.. I just tested it and I get the latest versions of Stryker.
Could you change your package.json to:
....
"stryker": "^0.5.4",
"stryker-api": "^0.4.1",
....

Then remove your node_modules folder and do an npm install?

I did the update you requested. Then I had to update package.json to the new syntax:
"stryker": "stryker run --coverageAnalysis all stryker.conf.js"

Then I got an error saying: Configured coverage analysis "perTest" requires there to be a testFramework configured. Either configure a testFramework or set coverageAnalysis to "all" or "off". So I did.
Now I get 143 error messages, for the 143 test files:
[ERROR] CoverageInstrumenterStream - Error while instrumenting file "/.../src/test/.../SomeTestFile.js", error was: Error: Line 1: Unexpected token Failed to parse file.
And then
[ERROR] Stryker - One or more tests errored in the initial test run: TypeError: Cannot read property 'Error' of undefined
I can't see anything wrong with those files. Mocha executes them just fine.

Could you post what's in your stryker.conf.js right now? We had some breaking changes with the 0.5.0 release of Stryker.
What's on line 1 of "/.../src/test/.../SomeTestFile.js"?

This is my current stryker.conf.js:
module.exports = function (config) {
  config.set({
    files: [
      // Add your files here, this is just an example:
      {pattern: 'src/test/**/*.js', mutated: true, included: false},
      'src/test/**/*.js'
    ],
    testRunner: 'mocha',
    testFramework: 'mocha',
    reporter: ['html', 'progress'],
    coverageAnalysis: 'perTest',
    plugins: ['stryker-mocha-runner', 'stryker-html-reporter']
  });
};

And this is an example of line 1 of my *.js tests:
import React from "react";
or
import {expect, assert} from "chai";

You could set coverageAnalysis to 'off' which should fix this for you. We don't properly support mutating files that include import statements yet. We'll fix/discuss that in #258


Upload quota consumed without files uploaded

I uploaded a big file to test the Keybase file storage, but after I noticed there was not a way to abort the upload, I deleted the partially uploaded file from the file explorer. The client continued uploading it, and now I have 7.5 GB of upload quota occupied (I see 242.5 GB of free space instead of 250 GB) without any file to delete (nothing shows up in the private folder where I uploaded it).
my log id: 242b8f6a078707be18c6d51c

There is no way to cancel an upload at the moment.  You can delete the file, and in the background it will still be uploaded, and then deleted once the upload is complete.  Supporting cancellations is complicated due to the way KBFS works, though in the future we might find a way to do it.  If your quota is not recovered after the upload is completely done, let me know.

If your quota is not recovered after the upload is completely done, let me know.

This is the actual problem

You're sure the upload is complete though?  It might still be going. You can try keybase fs uploads to see.

Yes, already tried, nothing shows up

I looked into your logs and I confirmed the upload completely correctly. There was a conflict resolution involved (where another device wrote to the folder while this device was), which makes the bytes tracking a little more complicated.  In any case, yeah this seems like a quota tracking bug but we will likely not treat it with high priority.  If you do end up needing more quota just let us know. Thanks!

Ok, thanks


fix: some problems with using Korean with TSF

This fixes some problems with typing in Korean when TSF is enabled but the composition string is still not being set correctly. It's still not behaving as expected
(Related to https://github.com/ryancheung/ImeSharp/issues/10)

Thanks!


Fix intermittent failure in TestMidas test.

Change the code so that it passes a SettingsChangeMonitor into the call to "ChangeMeasuredResults".
The SettingsChangeMonitor notices when the SrmDocument in the IDocumentContainer changes, and will cancel the operation. This prevents the settings update code from trying to read from a scale .skyd file.

Most elegant! Might there be other applications for this - other background loaders etc?

Most elegant! Might there be other applications for this - other background loaders etc?

Interesting idea. I see that several other BackgroundLoader's do call "ChangeSettings" or "ChangeMeasuredResults", but most of them are not passing in a SettingsChangeMonitor. (ChromatogramManager does pass in one).
This is only an issue if the ChangeSettings call is going to result in all of the results in the TransitionGroupDocNode's being updated. Usually a ChangeSettings is so fast that it does not need a SettingsChangeMonitor.