ipymd functionality: feedback wanted

@jni, @stefanv maybe you can test this out and give some feedback?
I've just pushed a change to master that lets you use markdown files as the on disk notebook storage, editing them with the web notebook. This was inspired by ipymd which lets you do a similar thing.
This isn't documented yet but I've released it with 1.4.4.
To try it out, add the following to ipython_notebook_config.py in your ipython profile (probably ~/.ipython/profile_default/):
c.NotebookApp.contents_manager_class = 'notedown.NotedownContentsManager'

and then try opening up, editing and saving some markdown in the browser.
Unlike ipymd, notedown preserves prompt numbers and outputs (using json code cells) in the markdown. This isn't very pretty but I get around it by collapsing json cells in my editor (and turning wrap off!).

@aaren Cool functionality but not actually something I'm really looking for right now — we're using notedown precisely to avoid editing in the web notebook! I don't have the bandwidth right now to test it out... Best of luck!

Is this likely to break with future releases of IPython, or is
this kind of storage mechanism supported?  If reliable and stable,
that would be a novel alternative!

@stefanv Support was added for this following https://github.com/ipython/ipython/issues/7222, so it has been a conscious decision to allow this to happen. I'm subclassing the API and didn't need to change too much so hopefully can keep up to date. Should be ok for 3.x anyway.
@jni I still work largely in my text editor. I do still find the notebook useful for code interaction and plotting figures though. This allows me to edit a file with both the browser and my editor at once: I can be editing in vim, then drop into the browser and create some figures, then return to vim and keep editing without worrying about merging the figures in.

@aaren I see! That's a pretty nice use-case. I'll let you know when I try it out! =)

hi @aaren , I'm just discovering this, it's a cool project!
Just a side note: ipymd can preserve text output (that's an option you can enable). The input prompts are reset to the normal order however (I have no use for nonlinear input prompts myself). Finally I'd like to implement saving inline plots automatically in external files and include them in Markdown.

@rossant, thanks :) and thanks for solving the markdown storage problem in ipymd!
Yes, storing the outputs in a more markdowny way would be good. I couldn't think of a satisfactory schema for it though and just decided to copy them as is (json).
We could probably solve the limited case of text or image outputs, but I'm not sure about the general case. Storing images in external files and linking to them would be good. Have a look at notedown --render - this converts images to data URIs (but these don't render on github).

Seems to be working fine for me so far.


fix: filter out Nones from guild.role before sorting

Pull Request Type

[ ] Feature addition
[x] Bugfix
[ ] Documentation update
[ ] Code refactor
[ ] Tests improvement
[ ] CI/CD pipeline enhancement
[ ] Other: [Replace with a description]

Description
Yes, this is a bug fix to a bug fix. More specifically #1454.
Basically, if the cache returned None for a role, sorted would still try to sort those Nones as they haven't been filtered out yet... so you can end up in scenarios where it tries to compare a NoneType to a NoneType.
Changes

Use a funny list comprehension to filter out Nones in the generator passed to sorted for guild.roles.

Related Issues

Test Scenarios

Python Compatibility

[ ] I've ensured my code works on Python 3.10.x
[x] I've ensured my code works on Python 3.11.x

Checklist

[x] I've run the pre-commit code linter over all edited files
[x] I've tested my changes on supported Python versions
[ ] I've added tests for my code, if applicable
[ ] I've updated / added  documentation, where applicable

smh this code is totally very clean and the best example of walrus operators

And it's much more performant than the old code


How to handle events and get the user's rate ?

I added code below in my jsx
<Rating onClick={rating => this.handleRate(rate)} />
and browser console says the rate is not defined. How should I define this rate variable? Or how can I interact with this component to get user's rate value?
Sorry for such a dumb question and it will be nice to help me with this problem. Thanks in advance.

Hi @teruun.
The function you pass to the onClick property is not using the rating parameter but an undefined one (rate). Try this:
<Rating onClick={rate => this.handleRate(rate)} />

Hope it helps.

wow @dreyescat
It helps.  Thank you very much!


Add the ability to measure mass within bounding radii (both gas and hydro)

So rather than just giving an outer radius, the user could provide an inner radius as well. That would let one measure a core-excised gas mass for instance.
The implementation will be slightly different for the hydro masses and gas masses. Gas will need to support the passing of a lower bound to the integral that the model instance performs, hydro will need to measure two masses (as that equation is 'mass within R') and subtract the inner from the outer.

Will also have to change how the profiles store existing gas mass and hydro mass values

Did not mean to press close there...

I'm putting this warning in the module as well, but the annular_mass method I added to the HydrostaticMass profile is probably not mathematically valid, as I just subtract one posterior distribution from another.
However this concludes the features I wanted to add for this issue.


Change null checking on json search results

Migrating From: https://github.com/solana-labs/solana/pull/29970
Fix null checking for JSON Search results on NFT metadata to improve NFT Image display
Problem
Some NFT images do not render on Solana Explorer because they do not have .properties.files defined in their metadata. Example from a recent Solana Spaces airdrop (by CrossMint):

Despite the metadata not having .properties.files, it does include a .image with a URI to the image.  Currently, the Explorer JSON processor has a null check on  .properties.files and NOT on .image, even though we process and use .image for rendering the images.
Summary of Changes
Change our null checker to check that no files metadata present AND no image is present (both must be true to return null). This will allow image rendering of an image if no files metadata is present and also would allow video/animation rendering if no image value is present.

Looks good to me. Thanks for the fix!

Thank you! From the Crossmint team :)

Lets go! Great work!


Web - Workspace - App Crashed when deleting the Workspace

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to URL https://staging.new.expensify.com/
Login with any account
Go to Settings>Workspace>Delete Workspace>Delete

Expected Result:
No crash when deleting the Workspace
Actual Result:
App Crashed when deleting the Workspace
Workaround:
Unknown
Platform:

Where is this issue occurring?

Web
iOS
Android
Desktop App
Mobile Web

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: No
Email or phone of affected tester (no customers): any
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/187217983-09bd90cf-5cee-4c3f-93d8-514fb7927acd.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open StagingDeployCash deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

Identify the pull request that introduced this issue and revert it.
Find someone who can quickly fix the issue.
Fix the issue yourself.

https://github.com/Expensify/App/pull/10400/ Causing this most likely

PR to fix this is a review


bowtie -x index option failure

Hi!
I am trying miRge3 on a few human samples (just 2 for testing purposes from a total of 45 fastq.gz files).
When running it returned me the following errors:
miRge3.0 -s ./fastq_cat/165-EHep01-1_S25_R1_001.fastq.gz,./fastq_cat/165-EHep02-1_S26_R1_001.fastq.gz -lib /home/lab/miRge3/miRge3_Lib -on human -db mirgenedb -o output_dir -gff -nmir -trf -ai -cpu 30 -a illumina

bowtie version: 1.3.0
cutadapt version: 3.1
Samtools version: 1.10
RNAfold version: 2.4.14
Collecting and validating input files...

miRge3.0 will process 2 out of 2 input file(s).

Cutadapt finished for file 165-EHep01-1_S25_R1_001 in 21.141 second(s)
Collapsing finished for file 165-EHep01-1_S25_R1_001 in 0.6396 second(s)

Cutadapt finished for file 165-EHep02-1_S26_R1_001 in 23.9095 second(s)
Collapsing finished for file 165-EHep02-1_S26_R1_001 in 4.5516 second(s)

Matrix creation finished in 1.1645 second(s)

Data pre-processing completed in 51.8239 second(s)

Alignment in progress ...
Alignment completed in 53.947 second(s)

Summarizing and tabulating results...
Traceback (most recent call last):
  File "/home/lab/anaconda3/envs/mirge3/bin/miRge3.0", line 10, in <module>
    sys.exit(main())
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/__main__.py", line 103, in main
    summarize(args, workDir, ref_db, base_names, pdMapped, sampleReadCounts, trimmedReadCounts, trimmedReadCountsUnique)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/libs/summary.py", line 1032, in summarize
    a2i_editing(args, cannonical_4ie, isomirs_4ie, base_names, workDir, Filtered_miRNA_Reads, mirMergedNameDic, mirDic, ref_db, seqDic, onlyCanmiRNA)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/libs/mirge2_tRF_a2i.py", line 1070, in a2i_editing
    bowtie = subprocess.run(str(bwtCommand), shell=True, check=True, stdout=subprocess.PIPE, text=True, stderr=subprocess.PIPE, universal_newlines=True)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'bowtie  --threads 30 -x  --phred64-quals /home/lab/miRge3/miRge3_Lib/human/index.Libs/human_genome -n 1 -f -a -3 2 output_dir/miRge.2020-12-16_14-10-50/SeqToMap.fasta' returned non-zero exit status 1.

I realised the bowtie command miRge3 is trying to execute is:
bowtie  --threads 30 -x  --phred64-quals /home/lab/miRge3/miRge3_Lib/human/index.Libs/human_genome -n 1 -f -a -3 2 output_dir/miRge.2020-12-16_14-10-50/SeqToMap.fasta

After checking the sintax of bowtie in their website I saw the "-x" is to address the path to index_folder/index_basename. In the miRge3 just after the -x option appears the --phred64-quals option followed of the path to the index.
Replacing mirge2_tRF_a2i.py lines can "fix" this issue...:
(1058) removing  + "-x"  :
bwtCommand = str(bwtCommand) + " --threads " + str(args.threads)

(1067) adding + "-x" :
 bwtCommand = str(bwtCommand) + " -x " + str(genome_index) + ' -n 1 -f -a -3 2 ' + str(samToMapFasta)

Trying to run it again give me again the same error but on lines:
Traceback (most recent call last):
  File "/home/lab/anaconda3/envs/mirge3/bin/miRge3.0", line 10, in <module>
    sys.exit(main())
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/__main__.py", line 103, in main
    summarize(args, workDir, ref_db, base_names, pdMapped, sampleReadCounts, trimmedReadCounts, trimmedReadCountsUnique)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/libs/summary.py", line 1032, in summarize
    a2i_editing(args, cannonical_4ie, isomirs_4ie, base_names, workDir, Filtered_miRNA_Reads, mirMergedNameDic, mirDic, ref_db, seqDic, onlyCanmiRNA)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/site-packages/mirge/libs/mirge2_tRF_a2i.py", line 1302, in a2i_editing
    bowtie = subprocess.run(str(bwtCommand), shell=True, check=True, stdout=subprocess.PIPE, text=True, stderr=subprocess.PIPE, universal_newlines=True)
  File "/home/lab/anaconda3/envs/mirge3/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'bowtie  --threads 30 -x  --phred64-quals /home/lab/miRge3/miRge3_Lib/human/index.Libs/human_genome -n 0 -f -a -3 2 output_dir/miRge.2020-12-16_15-48-07/SeqToJudge.fasta' returned non-zero exit status 1.

I did the same operation than before but with lines 1293 and 1301.
After this miRge3 was able to finish with the following output:
bowtie version: 1.3.0
cutadapt version: 3.1
Samtools version: 1.10
RNAfold version: 2.4.14
Collecting and validating input files...

miRge3.0 will process 2 out of 2 input file(s).

Cutadapt finished for file 165-EHep01-1_S25_R1_001 in 25.0222 second(s)
Collapsing finished for file 165-EHep01-1_S25_R1_001 in 0.727 second(s)

Cutadapt finished for file 165-EHep02-1_S26_R1_001 in 26.0285 second(s)
Collapsing finished for file 165-EHep02-1_S26_R1_001 in 6.4742 second(s)

Matrix creation finished in 1.1876 second(s)

Data pre-processing completed in 59.8659 second(s)

Alignment in progress ...
Alignment completed in 54.6129 second(s)

Summarizing and tabulating results...
The number of A-to-I editing sites is less than 10 so that no heatmap is drawn.
Summary completed in 10.8531 second(s)

Predicting novel miRNAs

Performing prediction of novel miRNAs...
Start to predict
No cluster sequences are generated and prediction is aborted.
Prediction of novel miRNAs Completed (39.17 sec)

The analysis completed in 166.7825 second(s)

I don't know if this way is a proper way to fixed. I would feel more confident if you can double-check this and let me know if that is correct or this implies other changes in the code. Also, if the original code is ok, could you help me to solve why it give me these errors?
Also, I didn't check why the prediction didn't work :)
Thank you in advance and kind regards!!

Hi @FrAoJm,
The problem you have fixed is correct, i.e., you have swapped -x  --phred64-quals  to this  --phred64-quals -x. Yes, this is an error and was not caught during our test runs. It is unusual because the phred64 format is set to 33 by default and (--phred64-quals) is being called in this case. We are adding a few extra features to IGV and will append this fix and make a new release soon.
Regarding novel miRNA prediction, can you try a sample with a large size? Most likely there might not be any novel miRNA or enough reads to support that call. However, keeping in mind the above issue, I will check the codes again before we make a new release.
Cheers,
Arun

May I ask where can I found the 'mirge2_tRF_a2i.py' from the system? Just meet the same problem, I used conda and pip for installing the miRge3.0, the 'miRge3.0' can be found from "~/.local/bin", but  I cannot find the  'mirge2_tRF_a2i.py'

@zyangx,
If you are using conda in a particular environment name say 'mirge38', then the script can be found at the following path: miniconda3/envs/mirge38/lib/python3.8/site-packages/mirge/libs/. Instead, if you are using the base environment, then you can find the script at: miniconda3/lib/python3.8/site-packages/mirge/libs/
For pip installation, on ubuntu, you can find it at .local/lib/python3.8/site-packages/. Also, please share the error message you have received. I will update this on Bioconda next week.

This is the error message what I got:
Summarizing and tabulating results...
Traceback (most recent call last):
File "/home/yangzhen/.local/bin/miRge3.0", line 8, in
sys.exit(main())
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/main.py", line 103, in main
summarize(args, workDir, ref_db, base_names, pdMapped, sampleReadCounts, trimmedReadCounts, trimmedReadCountsUnique)
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/libs/summary.py", line 1032, in summarize
a2i_editing(args, cannonical_4ie, isomirs_4ie, base_names, workDir, Filtered_miRNA_Reads, mirMergedNameDic, mirDic, ref_db, seqDic, onlyCanmiRNA)
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/libs/mirge2_tRF_a2i.py", line 1070, in a2i_editing
bowtie = subprocess.run(str(bwtCommand), shell=True, check=True, stdout=subprocess.PIPE, text=True, stderr=subprocess.PIPE, universal_newlines=True)
File "/home/yangzhen/Software/miniconda2/envs/py38/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'bowtie  --threads 12 -x  --phred64-quals /miRNA-seq_pipeline/miRge/miRge3_Lib/human/index.Libs/human_genome -n 1 -f -a -3 2 output_dir/miRge.2021-04-16_23-47-17/SeqToMap.fasta' returned non-zero exit status 1.
But I'm sorry I still can not find the script from .local/lib/python3.8/site-packages/. there are two folders called "mirge3-0.0.7.dist-info" and 'mirge', but no such a script was found from them. The method I used to install miRge 3.0 is to create an python3.8 environment by using conda, but I can not install 3.0 by using "conda install -c bioconda mirge3", instead, I uesd 'python3.8 -m pip install --user mirge3' in this conda environment. Then miRge3.0 works but stucks at this bowtie step. Sorry not very familiar with python, It is not necessary for me to use 2.0 as long as 3.0 works, but now I  get neigher of them run.

Oh, sorry, it seems out there, let me try

Update:
Now I can run miRg3.0 for small sample size data, but still get stuck when large sample size used. It seems due to the Memory limit, error report like:

Predicting novel miRNAs
Performing prediction of novel miRNAs...
Start to predict
Traceback (most recent call last):
File "/home/yangzhen/.local/bin/miRge3.0", line 8, in
sys.exit(main())
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/main.py", line 140, in main
predict_nmir(args, workDir, ref_db, base_names, pdUnmapped)
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/libs/novel_mir.py", line 332, in predict_nmir
convert2Fasta(pdUnmapped, infile, minLength, maxLength, countCutoff, outputdir2, species, speciesNameDic, base_names,rawReadCounts,filteredReadCounts)
File "/home/yangzhen/.local/lib/python3.8/site-packages/mirge/libs/novel_mir.py", line 52, in convert2Fasta
unmapList = unmap.reset_index().values.tolist()
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 5486, in values
self._consolidate_inplace()
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 5347, in _consolidate_inplace
self._protect_consolidate(f)
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 5336, in _protect_consolidate
result = f()
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 5345, in f
self._data = self._data.consolidate()
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 940, in consolidate
bm._consolidate_inplace()
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 945, in _consolidate_inplace
self.blocks = tuple(_consolidate(self.blocks))
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 1886, in _consolidate
merged_blocks = _merge_blocks(
File "/home/yangzhen/.local/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 3099, in _merge_blocks
new_values = np.vstack([b.values for b in blocks])
File "<array_function internals>", line 5, in vstack
File "/home/yangzhen/Software/miniconda2/envs/py38/lib/python3.8/site-packages/numpy/core/shape_base.py", line 283, in vstack
return _nx.concatenate(arrs, 0)
File "<array_function internals>", line 5, in concatenate
numpy.core._exceptions.MemoryError: Unable to allocate 12.0 GiB for an array with shape (66, 24368923) and data type int64

@zyangx,
Hi, thank you for the error report. Assuming your computer has sufficient memory, I believe it is related to overcommitting memory issues: I found some similar issues and solutions in this regard. Please use these solutions with caution  .
Basic information on overcommit handling modes here;
Solution for issue 1 here
Solution for issue 2 here
I hope this helps.


Rename TLMHybrid to TLMLite

name change is good and I like the copy saying TLM Lite.
Will leave checking the code actually works though to @jas2600 (and that it's been updated everywhere necessary)

The changes LGTM. But somehow when i tried to generate the docs website locally i got the following errors, which doesn't seems related though. And it only happens with this branch. The main branch works fine to generate the docs.
SyntaxError: /Users/jay/workspaces/cleanlab-studio-docs/docs/autogenerated/reference/cleanlab_studio.studio.inference.md: Expected corresponding JSX closing tag for . (9:163)
7 |
8 | {/* markdownlint-disable */}

9 |
|                                                                                ^
10 |
{module}{ }{cleanlab_studio.studio.inference}
11 | {Methods for interfacing with deployed ML models (to produce predictions). }
12 | {This module is not meant to be imported and used directly.}{Instead, use}{Studio.get_model()}{to instantiate a}{Model}{object.}
SyntaxError: /Users/jay/workspaces/cleanlab-studio-docs/docs/autogenerated/reference/cleanlab_studio.studio.studio.md: Expected corresponding JSX closing tag for . (9:160)
7 |
8 | {/* markdownlint-disable /}
9 |
|                                                                                ^
10 |
{module}{ }{cleanlab_studio.studio.studio}
11 | {Python API for Cleanlab Studio. }
12 |
SyntaxError: /Users/jay/workspaces/cleanlab-studio-docs/docs/autogenerated/reference/cleanlab_studio.studio.trustworthy_language_model.md: Expected corresponding JSX closing tag for . (9:180)
7 |
8 | {/ markdownlint-disable */}
9 |
|

Ah, it's bc you need to run the docs built with the updated docs config changes too (https://github.com/cleanlab/cleanlab-studio-docs/pull/536).
It worked locally for me:

Ah, it's bc you need to run the docs built with the updated docs config https://github.com/cleanlab/cleanlab-studio-docs/pull/536 too.

I believe I used the tlm-lite-rename branch of the docs repo to generate and got that error. But right now it works. Is that related to this change?


[NR-59017] TicToc time calculation

Ticket: https://new-relic.atlassian.net/browse/NR-59017
What's changed:

Use System.nanoTime() rather than System.currentTimeMillis() to prevent time difference due to a network change
Add unit tests

Q: What was the reasoning for doing this?
A: Use nanoTime() will prevent the scenario that when time got changed on the device due to wifi network, which causes inconsistent tictoc time period.
Q: What possible side effects might we see due to using System.currentTimeMillis() elsewhere in the agent?
A: I double checked most of other places of using System.currentTimeMills(), most of them are used as a timestamp. We could change few places that use to calculate time difference, but that will be a different request than this PR.


Lesson 1 revisions

This PR contains a few quick updates to the first CoCo lesson, including:

fixed banners not displaying on GitHub
removed stretch goals
added link to Software Carpentry lesson with nice overview of JupyterLab and Jupyter Notebook

This PR is superseded by #23.


[FLINK-15418][table-planner-blink] Set FlinkRelDistribution in StreamExecMatchRule

What is the purpose of the change
This pr fix that StreamExecMatchRule not sets FlinkRelDistribution trait.
Brief change log
1, Set FlinkRelDistribution in StreamExecMatchRule.
Verifying this change
This change added tests and can be verified as follows:
MatchRecognizeITCase.testPartitionByWithParallelSource
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)

@wuchong Thanks for the review, I've updated the code according to suggestions.


Add Initials and avatar fields to ResourceDto

Add Initials and avatar fields to ResourceDto

Bumping for visiblity

Thanks @jamescavanagh @gcmestre !  Happy to merge this change.  We generally do a release of the SDK after each API update and our code generator will rebuild all API fields automatically.


[Performance] Decoding Decodables from ByteBuffer is not zero-copy.

Expected behavior
The ByteBuffer's getJSONDecodable ought to be zero-copy.
Actual behavior
The ByteBuffer's getJSONDecodable doesn't use zero-copy flags to getData.
The getData's byteTransferStrategy: option at https://github.com/apple/swift-nio/blob/eb140159386d19fa2c9439af24286982a52fb0b3/Sources/NIOFoundationCompat/ByteBuffer-foundation.swift#L109 could have been set to .automatic, that copies everything less than 256kb.
Steps to reproduce
Observe https://github.com/apple/swift-nio/blob/eb140159386d19fa2c9439af24286982a52fb0b3/Sources/NIOFoundationCompat/Codable%2BByteBuffer.swift#L31

Also since the getData's result is not leaked outside of the decoder function, there's opportunity to perform zero-copy without allocating the deallocator function when constructing Data.

We should unconditionally use zero-copy here; copying is only used on small buffers to avoid retaining large ones for small amounts of data, and that’s not really relevant here.

Also since the getData's result is not leaked outside of the decoder function, there's opportunity to perform zero-copy without allocating the deallocator function when constructing Data.

I don’t think that’s entirely accurate. There is no guarantee in the JSONDecoder API that the passed in Data is not sliced to produce any output Data objects, at least not that I’m aware of. So we need to defend against the possibility that the Data we pass in may be propagated outside the function.

exactly because of what @Lukasa says, I decided not to do this but was thinking about asking the Foundation folks if they would be happy to make the guarantee that JSONDecoder never still references the Data that was passed in.


Provide initial GridGenerator as a constructor param to GameViewModel

Currently the initial grid generation logic is tied to GameViewModel.
It would be better to decouple the initial state generation logic from the GameViewModel.
If resuming the game becomes a feature, this would come in handy.

Issue #75 supersedes this issue, closing this one.


define FastUnicodeData to speedup unicode checks

Comparing
before

after

NOTE- ASCII test- is faster, Uni will be more faster!

project1, unit1?
holiday crossword puzzle? ;)

Fixed, renamed project make_unidata files

thank you!


iOS 8 devices crashing if the screen rotates during a scan.

This was being caused by a null reference in the previewLayer.Connection. I didn't dive too deep into it so I just added a simple null check. A better solution might be to find out why it is being set to null when the screen rotates during a scan.
Steps to reproduce:

Load sample.ios onto an ios 8 device
Select any of the scan options and let the camera be loaded
rotate the screen and do a scan

What happens is that on iOS 8.3, DidRotate gets called as the scanner is being dismissed. Because StopRunning completes before DidRotate is called, the Connection is null.
I came up with the exact same fix independently, and I can confirm that it works.

Thanks!


Support chunks without errors

Without this change trying to access a chunk inside a test will result in a 404 and a very cryptic test error.

:+1:

LGTM

@MikaAK - LGTM as well.


What should the ULP power consumption be in deep sleep? (IDFGH-6208)

Summary: the doco seems to suggest ULP uses 150uA but I see 450uA with the most minimal hwr and swr.
I want to run my project from battery so need to optimize power usage.
The hardware is a naked esp-wroom-32 module soldered to a carrier board. A low Iq LDO (RT9080) is used with a Li-Ion battery. No devices are attached yet. The sw is built with IDF 4.4.
When I go into deep sleep the current is 10-12uA, as expected. If I activate the empty ULP program it uses 450uA. The doco suggests it should be around 150uA.
I use this project. It now includes a hand drawn sketch of the board components.
TIA

try again after deleting these codes (https://github.com/makehackvoid/ESP8266Lib/blob/master/examples/eyal/null/main/ulp.c#L36-L37)

No change. BTW, seeing all the commented lines in udp.c should tell that I tried many combinations with no success.
From the schematics one can see that only the EN pin has a 10k pull-up. Others are either open (most) or attached to 1m+1m voltage dividers which should draw very little current even if they are active.
I wonder if there is another esp_sleep_pd_config() to set.
To make it even clearer, I measure current at the battery rather than after the LDO. I did at one point measure at the esp32 pin and the difference was negligible. And anyway, the same project but with the ULP inactive draws only 10-12uA (depending on the meter used).

I use the release/v4.4 version of the $IDF_PATH/examples/system/ulp_fsm/ulp example project to test on the ESP32-WROOM--32E module. When I delete the following two lines of code, the actual measurement results look normal.
rtc_gpio_isolate(GPIO_NUM_12);
rtc_gpio_isolate(GPIO_NUM_15);

The picture below is my measurement result:

I think the difference in our measurement results may come from the board or the measurement method. Perhaps you can provide a simple hardware schematic and measurement method to help troubleshoot the problem.

To simplify the situation further I soldered another ESP32 on an empty board. Connected a LiFePO4 battery (measured as 3.32V) to GND and 3.3V and installed a 10k pull-up from EN to 3.3V. Nothing else.
To program it I temporarily attached the necessary wires but the test ran without anything else attached. The results are the same, measured from one of the battery wires. I used a number of meters to confirm the readings and they mostly agree.
I think this is as basic a hardware setup as I can use. I will appreciate all informed suggestions.

I used your example project to test on the ESP32-WROOM--32E module, and the current is really large when deep sleep sleeps (refer to the figure below).

When I checked your code, I found that ULP is always running, so I modified some code and it looks OK (refer to the picture below)

You need to add a halt instruction here, You need to add a halt instruction here, it will trigger ULP to sleep.

Sorry, I got it wrong. As you mentioned, the current when ULP is working is higher than 150 uA (I measured 360 uA), I will confirm this problem.

Regarding the "halt": the actual program is, naturally, more involved than this trimmed down test which, on purpose, establishes a specific situation. The full application uses the ULP to count pulses from a flow sensors which can arrive at a few hundred a second so it runs permanently. The main processor is in deep sleep , waking up as required to report the status (using wifi).
If the current usage of the ULP is higher than 150uA when it is active then this should probably be documented, and I would like to know what to expect.

If the ADC, tsens sensor is not used in your project, you can modify the code according to the figure above, the current I tested is about 206 uA

Thanks, a good start. Do you think we can go lower, as the doco says, to 150uA?
Both changes 1 (.../rtc_init.c) and 2 (.../ulp.c)  are required to drop the current to 250uA for me. The 3rd change has no effect.
Are you suggesting that this change will go into the repository? I would rather not have to patch ESP-IDF after every git pull.
I think it is now time for me to open my second issue, which is actually the first I encountered, The new issue is [to be inserted].

You can try to modify it like this. The actual test current is as follows:

The actual test ULP can run, but there is no guarantee that there will be no problems after such modification. This requires more time to test and verify.

Yes, very nice, I now see 130uA. Hope it does not break anything. Tomorrow I will test my full app with these changes.
I will wait for more verification, and will probably not use these changes until they land in github. This project still has some time to go before being commissioned, as I am tying loose ends.

I see a status "Awaiting Response" but  I already responded "works OK". and it still does. My project uses the ULP just to count pulses on a digital pin so I cannot say how this change will affect other devices. Either accept the change or provide an API to control the feature (RTC_CNTL_BIAS_*_FOLW_8M) when required. Thanks.


added button and back button for help section

Added a "help" button on the main page, the button will navigate to the help page, on the help page, there's introduction to the E matrix, and the E matrix itself, and a "back" button to get back to the main page.

rebased with "main" branch

The code is causing issue, when adding in the task after clicking add task it won't go back to the dashboard.Please check the code and re submit a PR @Admiral-Monferrat


Upgrade to Helm v3

v2 is depricated.

also have helm set itself up in it's own little script in hal-scripts

I'd be careful with this - the vault helm chart IIRC warns not to use helm v3.

yes

@npmaile this is completed as of https://github.com/homedepot/spingo/pull/115. Please close if you agree.


Allow compilation with ghc 8.10.7

Need to allow GHC 8.10.7.
Current build on master:
λ  servant-hmac-auth git:(master) ✗ stack build --resolver lts-18.21

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for servant-hmac-auth-0.1.3:
    base64-bytestring-<IP_ADDRESS> from stack configuration does not match ^>=1.0  (latest matching version is <IP_ADDRESS>)
needed since servant-hmac-auth is a build target.

Some different approaches to resolving this:

  * Set 'allow-newer: true' in /Users/nitin/.stack/config.yaml to ignore all version constraints and build anyway.

  * Recommended action: try adding the following to your extra-deps in /Users/nitin/Documents/temp/servant-hmac-auth/stack.yaml:

- base64-bytestring-<IP_ADDRESS>@sha256:d5027ea6d63eab27cfa34eb1f5a05b6a5ee521dd176a06da3f7d80e117a88259,2352

It's possible you'll no longer be able to build with GHC 8.6.5, due to constraints introduced in https://github.com/Holmusk/servant-hmac-auth/commit/75599c259e276220d1bf90a4e200a55a22eb901e
In that case I'd recommend simply dropping ghc 8.6.5.
Also I'd recommend using GHC 8.8.4 (latest in 8.8 series instead of 8.8.3).

Or alternatively you can just add appropriate extra-deps to stack-8.6.5.yaml.


Obtain WASM/JS files from emscripten-forge

This PR removes the WASM/JS files from this repo and instead obtains them at build time from Emscripten-forge, based on the work in #25. Co-authored by @DerThorsten.
This passes the tests locally for me and works with cockle-playground. It currently doesn't work with the JupyterLite terminal extension but I think that is an existing issue in JupyterLite related to the recent upgrades and I will deal with that separately.

Looks promising. Probably the package release process is broken now, but I'll deal with that at the next release.


warning "unexpected middleware set after the adapter. This won't be supported from Faraday 1.0"

I got this issue when using oauth2 v1.4.1,  while using /master branch this issue is gone.  I think it's because on /master ,  the way how Faraday is built is different compared to v1.4.1  (see here for master .
Is there a way to get around it for v1.4.1?   e.g: pass in some kind of connection_opts when constructing the OAuth2::Client ?

@nahzz Could you puts your faraday versions and the code raise warning for more context?
Especially, the code create OAuth2::Client.
As I see in v1.4.1, oauth2 just leaves these config to caller.
Thanks.

@nahzz 1.4.1 is not supported.  If you upgrade to the latest release version, version 1.4.4, then Faraday issues should be resolved.  There should not be any breaking changes.
1.4.5 should be released within a day or so.

I am going to close this.  Please create a new issue if this continues to be the case with the current version of the gem.


How to parallelize the pipeline for cluster computing?

Hi there,
I'm interested in parallelization of the nf-core/rnaseq pipeline for running batch jobs on a computing cluster dedicated to parallel and multi-threaded application.
Unfortunately, I'm not very experienced with this and couldn't find any documentation on how to parallelize nf-core pipeline jobs. I'm currently not sure if this is supported at all?
Many thanks in advance for your help!

Hi Jan,
the pipeline already processes many samples and parallelizes the analysis in each step when the software used (e.g. STAR for mapping, Salmon ...) allows for that. That should cover your points ...?

Hi @apeltzer,
thanks a lot for the feedback!
I understand that the pipeline parallelizes the analysis but I'm still unclear about what its default settings are regarding parallelization? Is there an option to control for this in a HPC environment?
In comparison, I've seen that the GEO2RNAseq pipeline (https://bitbucket.org/thomas_wolf/geo2rnaseq/src/master/) provides the option to set a [MAX_CPUS] variable.

All the steps have settings that prove to be reliable - see the conf/base.config for details.  Labels are used for each step in the pipeline to request as many CPUs / Memory / Time as listed here in the config.
These processes are then for example submitted with the respective requirements to a scheduler on a cluster - if you use a scheduler in your institute, you could add this to here

That way you can use the pipeline on your cluster without further required changes

Many thanks @apeltzer!

No worries - also feel free to join the Slack  channel if you have more questions, I'll close this then 👍


Blazor Debug WASM updates: VS4Mac guidance

Fixes #19465
Internal Review Topic

Idk if we need to touch the Browser source maps section. Do we need to qualify those remarks now that VS4Mac is in the topic?
I couldn't get the Debug in the browser guidance to fully work. Shift+⌘+d gets the ball rolling with the Unable to find debuggable browser tab error page and the command seems to run. I can't get anywhere with it after that, and there is no file:// node in Sources. I added a remark on this PR that says the guidance is only for Chrome or Edge on Windows.
We'll have tooling tabs now.
I 🔪 the images. We have really, really, REALLY bad outcomes with IDE and Azure images around here. We don't have the staffing to support them. They go stale ... we get issues/confusion from readers ... we get time-crunched 🤯 trying to recreate them. I tailor the text here to avoid them. I await orders (at 👦🔫 gunpoint! 👦🔫 preferably 😄) for any that you feel we really need.
Is there matching Visual Studio (Windows) timeout guidance for VS4Mac?

We should get this in and live quickly. Idk that the plan was to put a live link to the docs into the blog post.
Thanks @mrward!

VS Mac does not currently have a way to configure/override the request timeout.

I tagged that for "Windows" now, so we should be good on that one.

We're going to go ahead and merge now. Message me for further updates. I'll be back online in about four hours (~2pm PST).


quiet pub get stdout in flutter create

Fixes #116227.
Pre-launch Checklist

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] All existing and new tests are passing.

FYI you might get a merge conflict with https://github.com/flutter/flutter/pull/117896

Closing as superseded by #118285


SVG imports in npm modules aren't transformed into React components

Describe the bug
Third-party npm modules that import SVGs in the following manner aren't transformed into React components:
import { ReactComponent as Logo } from "./logo.svg";

Instead, they remain as URL imports to the SVG file.
Did you try recovering your dependencies?

Yes.
Which terms did you search for in User Guide?

SVG
Environment

Environment Info:

  current version of create-react-app: 4.0.0
  running from C:\Users\Dylan\AppData\Roaming\npm-cache\_npx\19220\node_modules\create-react-app

  System:
    OS: Windows 10 10.0.19041
    CPU: (32) x64 AMD Ryzen 9 3950X 16-Core Processor
  Binaries:
    Node: 12.15.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.5 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 6.13.4 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 86.0.4240.111
    Edge: Spartan (44.19041.423.0), Chromium (86.0.622.56), ChromiumDev (88.0.680.1)
    Internet Explorer: 11.0.19041.1
  npmPackages:
    react: ^16.14.0 => 16.14.0
    react-dom: ^16.14.0 => 16.14.0
    react-scripts: ^3.4.3 => 3.4.3
  npmGlobalPackages:
    create-react-app: Not Found

Steps to reproduce

Import an npm module that imports SVGs using the ReactComponent as method.

Expected behavior

The SVG renders.
Actual behavior

Error: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: undefined. You likely forgot to export your component from the file it's defined in, or you might have mixed up default and named imports.

Reproducible demo

TBD

Additional Comments
I'm actually not sure if this is an intentional behavior or not. If it isn't intentional, and I should be able to import modules using this convention, please let me know and I'll put together a reproducible demo.

I'm running into this issue. Has anyone discovered a solution?

For future reference, this issue is caused because CRA does not look through node_modules in the current react-scripts webpack.config.js file.
// Process application JS with Babel.
            // The preset includes JSX, Flow, TypeScript, and some ESnext features.
            {
              test: /\.(js|mjs|jsx|ts|tsx)$/,
              include: paths.appSrc,
              loader: require.resolve('babel-loader'),
              options: {
                customize: require.resolve(
                  'babel-preset-react-app/webpack-overrides'
                ),

You need to change include: paths.appSrc to include: [paths.appSrc, paths.appNodeModules,].
You can do this without ejecting by using react-app-rewired, CRACO, or any other webpack extender for CRA using the following config:
const paths = require("./node_modules/react-scripts/config/paths");

module.exports = {
    webpack: function (config, env) {
        config.module.rules[1].oneOf[2].include = [
            paths.appSrc,
            paths.appNodeModules,
        ];
        return config;
    },
};

@dstaley did @CTLaChance's solution work for you?  I'm trying it out and with some adjustments rules[1].oneOf[3] I'm still getting a path for SVG files imported by npm modules.
Anyone have any further help here?


VSCode plugin seems to fail when trying to open ballerina module in stdlib in the language

Description:
Tried opening std lib module in VSCode, but there were no support from the plugin once it's opened.
Steps to reproduce:
Open ballerina folder inside src/main folder in the stdlib module. (tried doing project init as well) still the same issue.
Affected Versions:
0.990.3

In which stdlib submodule did this occur? Tried opening up few modules like auth, bir and builtin and the extension worked as expected. I get completions and diagnostics.

I tried bir module

When building a module which is packed with the ballerina distribution, compiler fails with a class cast exception (log attached herewith). Due to this reason, the
ballerina-internal.log
plugin fails to provide the suggestions.

Analyzed the error log;
org.wso2.ballerinalang.compiler.GenericPackageBinary cannot be cast to org.ballerinalang.repository.PackageSource

Seems like we have an unchecked cast to PackageSource. On the other hand; checking instanceOf will be costly for micro level performance concerns.
How about writing a compiler plugin to warn user, when his source code's orgName/alias conflicts with internally packed stdlibs. This will be beneficial for the tooling as well since compilation errors causes completions, definitions...etc fails without a user information.

This is fixed in SL.


Add inittab to run fsck on boot.

PR Overview:

Adds an inittab (based off buildroot's) to run fsck on boot.

The default buildroot inittab is here if you want to diff it.
https://raw.githubusercontent.com/buildroot/buildroot/master/package/busybox/inittab

circleci doesn't seem forthcoming with why this failed :(

Closing in favor of #147 since we've let this PR fall by the wayside for so long :sweat:


Not playing.

I want to play songs on the latest 1.10 version but it does not start. All the streams are not staring. If you need info tell me.

Here is a video of the issue:
https://www.youtube.com/watch?v=X7Fx9lI8Z98

I can't reproduce this in MP, or SP.  I'll need the relevant log files from client/server if applicable.


Microsoft.Dynamics.Nav.NavUserAccount.NavUserAccountHelper

I use the DotNet command shown in the subject line in a few places in my Solution and I can't see anything related to it on here so I need a non-DotNet alternative in AL please...

Help....anyone.... please?

@JesperSchulz

Closing due to lack of response. If the issue still exists, kindly open a new issue in this repository. Thank you!


SamuelOgboye

Hello @samuelogboye.
Nice work, however, please rename your folder to be in all lower case. Otherwise, excellent work!

Ok thanks
I’ve resolved and push to my branch. But can’t see a new pull request order.
Will it be merged automatically?
On Thu, Jul 13, 2023 at 1:47 PM Edwin Ade @.***> wrote:

@.**** requested changes on this pull request.
Hello @samuelogboye https://github.com/samuelogboye.
Excellent work with your code, however, please rename your directory to be
in all lowercase. Also in future, try to provide a short description with
your PR. Otherwise, good work!
—
Reply to this email directly, view it on GitHub
https://github.com/ALX-SE-Algorithmia/Demo-Project/pull/39#pullrequestreview-1528360576,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A557ZCSRIT62HS2KVE7IYQDXP7U4RANCNFSM6AAAAAA2IHP72A
.
You are receiving this because you were mentioned.Message ID:
@.***>

Ok thanks I’ve resolved and push to my branch. But can’t see a new pull request order. Will it be merged automatically?
…
On Thu, Jul 13, 2023 at 1:47 PM Edwin Ade @.> wrote: @.* requested changes on this pull request. Hello @samuelogboye https://github.com/samuelogboye. Excellent work with your code, however, please rename your directory to be in all lowercase. Also in future, try to provide a short description with your PR. Otherwise, good work! — Reply to this email directly, view it on GitHub <#39 (review)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A557ZCSRIT62HS2KVE7IYQDXP7U4RANCNFSM6AAAAAA2IHP72A . You are receiving this because you were mentioned.Message ID: @.***>

Perfect, ready for merging, thank you Sam I will do the merging


 However, in some conditions the default methods of sex detection can give incorrect results e.g. when homozygous reference alleles are omitted from the source such as in whole-exome or whole-genome. Therefore it is useful to be able to specify the sex detection method used when deciding how to deduplicate the sex chromosomes.
This PR both supports the existing use of a boolean for deduplicate_XY_chrom as well as using a String that can be passed to determine_sex() as the chrom parameter.
( I'll probably have another PR soon for improvements to determine_sex() itself. )

Codecov Report

Merging #114 (e535507) into develop (91bfccf) will increase coverage by 1.37%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop     #114      +/-   ##
===========================================
+ Coverage    92.98%   94.36%   +1.37%
===========================================
  Files            8        8
  Lines         1468     1419      -49
  Branches       252      252
===========================================
- Hits          1365     1339      -26
+ Misses          65       41      -24
- Partials        38       39       +1

Impacted Files
Coverage Δ

src/snps/snps.py
95.49% <0.00%> (+0.88%)
:arrow_up:

src/snps/io/reader.py
97.87% <0.00%> (-0.02%)
:arrow_down:

src/snps/resources.py
94.73% <0.00%> (+0.03%)
:arrow_up:

src/snps/ensembl.py
70.73% <0.00%> (+39.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91bfccf...e535507. Read the comment docs.

Hmm, interesting idea. I like it because it only has to be set in one place, and can be used in many as a default. It would still have to be overridable on specific invocations, but that shouldn't be a problem.
I'll have a try with that approach and see how it works in practice.

@apriha I've implemented the selection of sex detection method as an instance variable, so it can be shared between methods easily. Theres are some further improvements to be made (e.g. a separate threshold to detect low heterozygous X chromosomes as male, number of non-null Y snps as a proportion of the total snps) but they are separate issues for the future.


Auto switch light/dark-mode W10+Office theme

Description of the new feature / enhancement
As a round-the-clock PC-user I would like to have an option to automatically switch from W10 dark-mode to light-mode and vice versa.
Either via a self-defined schedule (e.g. 08:30 switch dark-mode into light-mode; 17:30 switch from light-mode into dark-mode).
Or perhaps via sunrise/sunset based on location.
Apart from switching the Windows-theme I would like Office-applications (MS Word, Excel, Powerpoint, Outlook,..., MS Teams) to switch in synch with switching the Windows-theme.
Scenario when this would be used?
Currently performing these tasks manually multiple times on a daily basis. Would be nice to automate this.
Supporting information
Blogs can be found on the internet describing how to automate switching dark/light-mode for Windows via the Task Scheduler (e.g. [https://www.windowscentral.com/how-switch-between-light-and-dark-colors-schedule-automatically-windows-10] ), but even then I need to manually switch to dark/light-mode for MS-Office applications (Word, excel, powerpoint, outlook) and then still this setting is not taken over by MS Teams. In MS Teams you need to switch manually as well.
All in all (without automating via Task Scheduler) you need to adjust settings on 3 different places manually when you want to switch from light to dark when working late.....

Yes. Has been requested many times (including myself) both in Powertoys ( see #1331) and in the Feedback Hub.

Thanks for the suggestion! We'll close this against #1331 to keep the conversation centralized to that thread. You're added points for updating MS-Office product themes as well have been noted.


Many unit tests related to file systems fail on Martijn-s laptop

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗
Describe the issue
On Martijn-s laptop, mvn clean install produces many failing unit tests, one of them being Samba2FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot. This issue does not appear on Tim's development, not in GitHub actions and not on Jenkins. The issue appears both on the master branch (commit 7e3fe213d47cb39de27709838ad4a7005474f6f9) and the 7.9-release branch (commit fbf05022202ec0b045302ef1706629f00ebc5dd6).
Martijn could reproduce the issue in Eclipse

He added some additional logging to the test as shown below:

The log output is:
10:58:11,784 INFO  Samba2FileSystemTest.setUp():42[] Test in Samba2FileSystemTest [org.frankframework.filesystem.smb.Samba2FileSystemTest.null]
*** JFileServer Enterprise edition licenced to Frank!Framework (Niels<EMAIL_ADDRESS>***  For version 1.*
10:58:11,908 INFO  BasicFileSystemTest.setUp():42[] Test in BasicFileSystemTest [org.frankframework.filesystem.smb.Samba2FileSystemTest.null]
10:58:11,908 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():482[] Test basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot
10:58:11,908 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():483[] Clean root folder
10:58:11,908 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():485[] Create folder 'folder'
10:58:11,909 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():487[] Create folder 'Otherfolder'
10:58:11,909 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():489[] Create file
10:58:11,910 INFO  BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot():491[] List file

The line 10:58:11,908 INFO  BasicFileSystemTest.setUp():42[] Test in BasicFileSystemTest [org.frankframework.filesystem.smb.Samba2FileSystemTest.null] comes from added code in BasicFileSystemTest.java:

This rule is applied in the setUp() methods to display the name of the test, although the name is rendered incorrectly as null.
When Martijn discussed with Tim van der Leeuw, it struck Jim that the createFile() calls in the test succeeded while the basicFileSystemTestListFile() call failed.
The test succeeded when it was run in Eclipse without running other tests. The information given here was produced by running all tests of package org.frankframework.filesystem.smb.
Reporter
Martijn Dirkse

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots
If applicable, add screenshots to help explain your problem.
Configuration
Please provide the configuration of the Pipe or Receiver with the problem.

Input
Please provide an example of the input message.
Alternatively, provide a Ladybug report.

Environment
Please go to the console of your Frank, click on 'Information' and copy-paste the results here.

Additional Environment

DBMS: [e.g. Oracle, MSSQL, MariaDB, MySql, PostgreSQL]
Browser: [e.g. Chrome, Safari, Edge, Firefox]

Additional Context
Add any other context about the problem here. (f.e. ladybug report / test adapter with larva test)

This issue is related to #6063

Maven and Java versions:
C:\Users\martijn\git\frankframework>mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Users\martijn\Programmas\apache-maven-3.6.3-bin\apache-maven-3.6.3\bin\..
Java version: 11.0.15, vendor: Eclipse Adoptium, runtime: C:\Users\martijn\Programmas\OpenJDK11U-jdk_x64_windows_hotspot_11.0.15_10\jdk-11.0.15+10
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"

C:\Users\martijn\git\frankframework>

Eclipse:
Eclipse IDE for Enterprise Java and Web Developers (includes Incubating components)

Version: 2021-09 (4.21.0)
Build id: 20210910-1417

(c) Copyright Eclipse contributors and others 2000, 2021.  All rights reserved. Eclipse and the Eclipse logo are trademarks of the Eclipse Foundation, Inc., https://www.eclipse.org/. The Eclipse logo cannot be altered without Eclipses permission. Eclipse logos are provided for use under the Eclipse logo and trademark guidelines, https://www.eclipse.org/logotm/. Oracle and Java are trademarks or registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

This product includes software developed by other open source projects including the Apache Software Foundation, https://www.apache.org/.

Lombok v1.18.22 "Envious Ferret" is installed. https://projectlombok.org/

What happens when you run the tests without the proprietary profile?

mvn -P!proprietary clean install does not work, but the error is different:
[INFO]
[INFO] ---------------< org.frankframework:frankframework-aws >----------------
[INFO] Building Frank!Framework support for AWS 8.1.0-SNAPSHOT          [16/25]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ frankframework-aws ---
[INFO] Deleting C:\Users\martijn\git\frankframework\aws\target
[INFO]
[INFO] --- flatten-maven-plugin:1.5.0:clean (clean) @ frankframework-aws ---
[INFO] Deleting C:\Users\martijn\git\frankframework\aws\.flattened-pom.xml
[INFO]
[INFO] --- maven-enforcer-plugin:3.4.1:enforce (enforce-lib-ban) @ frankframework-aws ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- maven-enforcer-plugin:3.4.1:enforce (enforce-classpath-lib-ban) @ frankframework-aws ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- directory-maven-plugin:1.0:directory-of (find-project-root-directory) @ frankframework-aws ---
[INFO] Directory of org.frankframework:frankframework-parent set to: C:\Users\martijn\git\frankframework
[INFO]
[INFO] --- flatten-maven-plugin:1.5.0:flatten (flatten) @ frankframework-aws ---
[INFO] Generating flattened POM of project org.frankframework:frankframework-aws:jar:8.1.0-SNAPSHOT...
[INFO]
[INFO] --- glassfish-copyright-maven-plugin:2.4:copyright (Check-CopyRight-Notice) @ frankframework-aws ---
[INFO]
[INFO] --- glassfish-copyright-maven-plugin:2.4:check (Check-CopyRight-Notice) @ frankframework-aws ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ frankframework-aws ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\Users\martijn\git\frankframework\aws\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.11.0:compile (default-compile) @ frankframework-aws ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 7 source files with javac [debug release 11] to target\classes
[INFO]
[INFO] --- maven-checkstyle-plugin:3.3.1:check (verify-code-style) @ frankframework-aws ---
[INFO] You have 0 Checkstyle violations.
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ frankframework-aws ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.11.0:testCompile (default-testCompile) @ frankframework-aws ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 5 source files with javac [debug release 11] to target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ frankframework-aws ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.frankframework.filesystem.AmazonS3FileSystemTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.01 s <<< FAILURE! - in org.frankframework.filesystem.AmazonS3FileSystemTest
[ERROR] org.frankframework.filesystem.AmazonS3FileSystemTest  Time elapsed: 0.01 s  <<< ERROR!
java.lang.NoClassDefFoundError: Lorg/junit/rules/TestName;
Caused by: java.lang.ClassNotFoundException: org.junit.rules.TestName

[INFO] Running org.frankframework.jms.AmazonSqsFactoryTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.79 s - in org.frankframework.jms.AmazonSqsFactoryTest
[INFO] Running org.frankframework.senders.AmazonS3SenderTest
[INFO] Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.531 s - in org.frankframework.senders.AmazonS3SenderTest
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   AmazonS3FileSystemTest » NoClassDefFound Lorg/junit/rules/TestName;
[INFO]
[ERROR] Tests run: 36, Failures: 0, Errors: 1, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Frank!Framework parent 8.1.0-SNAPSHOT:
[INFO]
[INFO] Frank!Framework parent ............................. SUCCESS [  8.170 s]
[INFO] Frank!Framework Commons ............................ SUCCESS [ 16.784 s]
[INFO] Frank!Framework Security ........................... SUCCESS [  6.455 s]
[INFO] CredentialProvider ................................. SUCCESS [ 11.368 s]
[INFO] Frank Management Gateway ........................... SUCCESS [  6.904 s]
[INFO] Frank!Framework DBMS ............................... SUCCESS [  4.012 s]
[INFO] Frank!Framework core ............................... SUCCESS [07:28 min]
[INFO] Frank Console ...................................... SUCCESS [  0.226 s]
[INFO] Frank Console Backend .............................. SUCCESS [ 13.083 s]
[INFO] Frank Console Frontend ............................. SUCCESS [01:29 min]
[INFO] Frank Console Webapp ............................... SUCCESS [  4.872 s]
[INFO] Frank!Framework Larva .............................. SUCCESS [  6.218 s]
[INFO] Frank!Framework Ladybug ............................ SUCCESS [  6.077 s]
[INFO] Frank!Framework minimal bundle ..................... SUCCESS [  0.368 s]
[INFO] Frank!Framework support for Akamai ................. SUCCESS [  9.763 s]
[INFO] Frank!Framework support for AWS .................... FAILURE [ 21.338 s]
[INFO] Frank!Framework support for Batch processing ....... SKIPPED
[INFO] Frank!Framework support for Cmis ................... SKIPPED
[INFO] Frank!Framework support for Aspose ................. SKIPPED
[INFO] Special classes for NN ............................. SKIPPED
[INFO] Frank!Framework support for Messaging Systems ...... SKIPPED
[INFO] Frank!Framework full bundle ........................ SKIPPED
[INFO] Frank!Framework webapp ............................. SKIPPED
[INFO] Frank!Framework Test ............................... SKIPPED
[INFO] Frank!Framework example ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10:53 min
[INFO] Finished at: 2024-01-25T10:11:03+01:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "proprietary" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project frankframework-aws: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\martijn\git\frankframework\aws\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :frankframework-aws

C:\Users\martijn\git\frankframework>

Strange.. The error (not being able to find org.junit.rules.TestName) implies that we use that class somewhere, but I cannot see it being used anywhere in our code!
We're trying to migrate away from all JUnit 4 classes in order to avoid these type of problems. If you add -PL AWS it should ignore the AWS module for now.

I tried mvn -P!proprietary --projects !aws clean install but now the file system errors are back. I have this:
[INFO] Running org.frankframework.xslt.XsltSenderTest
[WARNING] Tests run: 41, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.352 s - in org.frankframework.xslt.XsltSenderTest
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestCopyFile:320 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestExistsMethod:355 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestGetParentOfTheDeletedFile:572 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileFromEmptyFolder:471->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot:480->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromRootWhenReadingFromFolder:497->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestModificationTime:225 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestMoveFile:249 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestNotExists:77 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestReadSpecialCharsFails:190 » FileSystem
[INFO]
[ERROR] Tests run: 4865, Failures: 0, Errors: 10, Skipped: 290
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Frank!Framework parent 8.1.0-SNAPSHOT:
[INFO]
[INFO] Frank!Framework parent ............................. SUCCESS [  7.972 s]
[INFO] Frank!Framework Commons ............................ SUCCESS [ 13.107 s]
[INFO] Frank!Framework Security ........................... SUCCESS [  6.231 s]
[INFO] CredentialProvider ................................. SUCCESS [ 12.467 s]
[INFO] Frank Management Gateway ........................... SUCCESS [  7.162 s]
[INFO] Frank!Framework DBMS ............................... SUCCESS [  4.113 s]
[INFO] Frank!Framework core ............................... FAILURE [08:16 min]
[INFO] Frank Console ...................................... SKIPPED
[INFO] Frank Console Backend .............................. SKIPPED
[INFO] Frank Console Frontend ............................. SKIPPED
[INFO] Frank Console Webapp ............................... SKIPPED
[INFO] Frank!Framework Larva .............................. SKIPPED
[INFO] Frank!Framework Ladybug ............................ SKIPPED
[INFO] Frank!Framework minimal bundle ..................... SKIPPED
[INFO] Frank!Framework support for Akamai ................. SKIPPED
[INFO] Frank!Framework support for Batch processing ....... SKIPPED
[INFO] Frank!Framework support for Cmis ................... SKIPPED
[INFO] Frank!Framework support for Aspose ................. SKIPPED
[INFO] Special classes for NN ............................. SKIPPED
[INFO] Frank!Framework support for Messaging Systems ...... SKIPPED
[INFO] Frank!Framework full bundle ........................ SKIPPED
[INFO] Frank!Framework webapp ............................. SKIPPED
[INFO] Frank!Framework Test ............................... SKIPPED
[INFO] Frank!Framework example ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:07 min
[INFO] Finished at: 2024-02-01T11:19:47+01:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "proprietary" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project frankframework-core: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\martijn\git\frankframework\core\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :frankframework-core

C:\Users\martijn\git\frankframework>

It strikes me that profile proprietary could not be found because it appears in the parent pom.xml.

@mhdirkse, I am using Java <IP_ADDRESS> and Maven version 3.9.3. Can you try using that?
Tip: you don't need to do install on maven. Just run mvn clean test to get the same results.
Perhaps clean your local maven cache partly (~/.m2/repository/org/frankframework/), if you think things are corrupted. All unit tests in master should just run fine.
The profile proprietary is defined in the root pom.xml file. When that can not be found, that's an error for sure. Perhaps that's solved when you upgrade your maven?

Thanks. The tests still fail after erasing my .m2/repository folder. I have:
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestCopyFile:320 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestExistsMethod:355 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestGetParentOfTheDeletedFile:572 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileFromEmptyFolder:471->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFolders:508 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromOtherFoldersWhenReadingFromRoot:480->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestListFileShouldNotReadFromRootWhenReadingFromFolder:497->BasicFileSystemTest.basicFileSystemTestListFile:368 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestModificationTime:225 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestMoveFile:249 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestNotExists:77 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestReadSpecialChars:172 » FileSystem
[ERROR]   Samba1FileSystemTest>BasicFileSystemTest.basicFileSystemTestReadSpecialCharsFails:190 » FileSystem
[INFO]
[ERROR] Tests run: 4846, Failures: 0, Errors: 12, Skipped: 291
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Frank!Framework parent 8.1.0-SNAPSHOT:
[INFO]
[INFO] Frank!Framework parent ............................. SUCCESS [ 31.264 s]
[INFO] Frank!Framework Commons ............................ SUCCESS [ 24.781 s]
[INFO] Frank!Framework Security ........................... SUCCESS [ 12.325 s]
[INFO] CredentialProvider ................................. SUCCESS [ 19.468 s]
[INFO] Frank Management Gateway ........................... SUCCESS [ 11.912 s]
[INFO] Frank!Framework DBMS ............................... SUCCESS [  3.862 s]
[INFO] Frank!Framework core ............................... FAILURE [10:15 min]
[INFO] Frank Console ...................................... SKIPPED
[INFO] Frank Console Backend .............................. SKIPPED
[INFO] Frank Console Frontend ............................. SKIPPED
[INFO] Frank Console Webapp ............................... SKIPPED
[INFO] Frank!Framework Larva .............................. SKIPPED
[INFO] Frank!Framework Ladybug ............................ SKIPPED
[INFO] Frank!Framework minimal bundle ..................... SKIPPED
[INFO] Frank!Framework support for Akamai ................. SKIPPED
[INFO] Frank!Framework support for AWS .................... SKIPPED
[INFO] Frank!Framework support for Batch processing ....... SKIPPED
[INFO] Frank!Framework support for Cmis ................... SKIPPED
[INFO] Frank!Framework support for Aspose ................. SKIPPED
[INFO] Special classes for NN ............................. SKIPPED
[INFO] Frank!Framework support for Messaging Systems ...... SKIPPED
[INFO] Frank!Framework full bundle ........................ SKIPPED
[INFO] Frank!Framework webapp ............................. SKIPPED
[INFO] Frank!Framework Test ............................... SKIPPED
[INFO] Frank!Framework example ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11:59 min
[INFO] Finished at: 2024-02-05T16:56:09+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project frankframework-core: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\martijn\git\frankframework\core\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :frankframework-core

C:\Users\martijn\git\frankframework>

C:\Users\martijn\git\frankframework>mvn -v
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: C:\Users\martijn\Programmas\apache-maven-3.9.6-bin\apache-maven-3.9.6
Java version: 11.0.22, vendor: Eclipse Adoptium, runtime: C:\Users\martijn\Programmas\OpenJDK11U-jdk_x64_windows_hotspot_11.0.22_7\jdk-11.0.22+7
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"

C:\Users\martijn\git\frankframework>

The command mvn help:active-profiles shows:
The following profiles are active:

 - proprietary (source: external)
 - default-modules (source: org.frankframework:frankframework-parent:${revision})
 - proprietary (source: org.frankframework:frankframework-parent:${revision})

Thanks for trying out. Too bad it didn't work out. TIp: you don't need to flush your whole M2 repo, just the org/frankframework part should suffice, but maybe in this strange situation, it was just fine.
I am wondering what is going on. Can you do one more tryout? I was thinking of a fresh git clone of the frankframework-repo and then building that with mvn test. Just to make sure nothing from the past is sitting in the way.
I also just noticed that your first experiments could build core but in the last experiment, it was failing at core. That makes it even more weird.

TIm had vandaag precies dezelfde errors op zijn laptop, begreep ik. We weten nog niet waarom dit wel door onze CI/CD is gegaan. Zijn er nog mee bezig.

Oke, mijn conclusies hierboven blijken niet valide te zijn. Tim schrijft vanmorgen namelijk: "Bij mij gisteren waren het niet altijd dezelfde tests en na herstarten laptop ging alles weer goed." Ik neem aan dat dit bij Martijn niet het probleem oplost en dus blijft het zo helaas toch een mysterie.

I just ran the unit tests for the first time with Java 17, after pulling. I have commit 9e5a9b6b65d87d7478f42e93467353fbde72d2ee now. This way the build succeeded. I am curious whether I was lucky or whether the issue has been fixed.

I mean: running the tests was successfull. I did mvn test.

I just tried with Java 21. Now I have the following error:
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   FtpFileSystemTest>FileSystemTest.writableFileSystemTestCopyFileToNonExistentDirectoryCreateFolderTrue:568 » FileSystem
[INFO]
[ERROR] Tests run: 4947, Failures: 0, Errors: 1, Skipped: 128
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Frank!Framework parent 8.1.0-SNAPSHOT:

I scrolled up to find more details:
[INFO] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.171 s - in org.frankframework.filesystem.FtpFileSystemSenderTest
[INFO] Running org.frankframework.filesystem.FtpFileSystemTest
[ERROR] Tests run: 51, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 170.063 s <<< FAILURE! - in org.frankframework.filesystem.FtpFileSystemTest
[ERROR] writableFileSystemTestCopyFileToNonExistentDirectoryCreateFolderTrue  Time elapsed: 120.424 s  <<< ERROR!
org.frankframework.filesystem.FileSystemException: Cannot connect to the FTP server with domain [localhost]: (IOException) Timed out waiting for initial connect reply: (SocketTimeoutException) Read timed out
Caused by: java.io.IOException: Timed out waiting for initial connect reply
Caused by: java.net.SocketTimeoutException: Read timed out

[INFO] Running org.frankframework.filesystem.LocalFileSystemActorTest
[INFO] Tests run: 71, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.574 s - in org.frankframework.filesystem.LocalFileSystemActorTest

Closing this as it seems to be a fluke. A new issue can be opened when there are test failures.

I still have this issue on the following commit:
commit 30ef8c8fd4146a9624d0edfc95f33a16990ee0d3 (HEAD -> master, origin/master, origin/HEAD)
Author: Niels Meijer<EMAIL_ADDRESS>Date:   Tue Mar 12 15:38:47 2024 +0100

    Support Ladybug database storage (#6385)

I built with Java 17 and got the following errors:
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.051 s - in org.frankframework.filesystem.mock.MockForEachAttachmentPipeTest
[INFO] Running org.frankframework.filesystem.SftpFileSystemActorTest
[ERROR] Tests run: 67, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.924 s <<< FAILURE! - in org.frankframework.filesystem.SftpFileSystemActorTest
[ERROR] fileSystemActorMoveActionTestRootToFolderExistsAndAllowToCreate  Time elapsed: 0.005 s  <<< ERROR!
java.net.BindException: Address already in use: bind
        at org.frankframework.filesystem.SftpFileSystemActorTest.setUp(SftpFileSystemActorTest.java:44)

[INFO] Running org.frankframework.filesystem.SftpFileSystemSenderTest
[INFO] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.063 s - in org.frankframework.filesystem.SftpFileSystemSenderTest
[INFO] Running org.frankframework.filesystem.SftpFileSystemTest
[ERROR] Tests run: 56, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 15.638 s <<< FAILURE! - in org.frankframework.filesystem.SftpFileSystemTest
[ERROR] writableFileSystemTestCreateNewFile  Time elapsed: 0.002 s  <<< ERROR!
java.net.BindException: Address already in use: bind
        at org.frankframework.filesystem.SftpFileSystemTest.startNewSshDaemon(SftpFileSystemTest.java:51)
        at org.frankframework.filesystem.SftpFileSystemTest.setUp(SftpFileSystemTest.java:41)

[INFO] Running org.frankframework.filesystem.smb.Samba1FileSystemActorTest

In the summary near the end of the Maven build I had:
[INFO] Frank!Framework FileSystems ........................ FAILURE [05:06 min]

And below that the errors were summarized as follows:
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project frankframework-filesystem: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\martijn\git\frankframework\filesystem\target\surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]

I still have this issue on the following commit:

Hi @mhdirkse, likely this is not related. I've randomized the bind ports for the local SFTP server, but it also still sometimes fails at GitHub builds. I am now working on a fix. Please ignore these errors for now.

The sftp server should automatically find an unused port. @jkosternl your solution by doing exactly the same won't fix this. I've personally never seen this fail on GH by the way...

The sftp server should automatically find an unused port. @jkosternl your solution by doing exactly the same won't fix this. I've personally never seen this fail on GH by the way...

Some evidence: https://github.com/frankframework/frankframework/actions/runs/8251487224/job/22568642354
My solution just tries a few ports, in order to pass through a free one to the SFTP server. This increases the chances that it works.

Nu het pr er door is, zou je het nogmaals kunnen proberen?

Gelukt, de testen slagen als ik met Java 17 bouw. Ik heb met deze commit getest:
commit bb172e86db48ab54629e104f7e42807343574a01 (HEAD -> master, origin/master, origin/HEAD)
Author: MarkvdVorst<EMAIL_ADDRESS>Date:   Wed Mar 13 17:39:45 2024 +0100

    Remove unnecessary information from GrayBox view (#6371)

Gelukt, de testen slagen als ik met Java 17 bouw. Ik heb met deze commit getest:

Great! Than likely the new SFTP-unit tests work fine, in finding a free port. I'll close the issue again.


Is there a way to disable the selection?

Hello, @evrencoskun
Currently i'm having issues with the selectiongHandler. First, ok it does unselect the item but when i scroll, i'm using tabs by the way, to another tab and return the selection still there. I think it is a bu, also if if i have many columns when i unselect and scroll horiontally on the table it shows me the selection again, so might be a bug. So i went to anther solution, disable the selection is there a way to disable? set the selection to false or should i null the selection handler?

Hi @tuvirus,
Probably, 0.8.8 version has got a selection bug as you mention.
To ignore setting selection colors that are displayed by user interaction, the below line can be used.
 tableView.setIgnoreSelectionColors(false);

Where should I use it? I'm using it in my activity class. But doesn't work. I was planning just comment the selection states so it doesn't change the color.

when you initialize your tableView, you should use it.


Phpcpd names option?

Hello,
I have a doubt. Does the phpcpd analyzer tool has any defined way to obey the extensions configuration option?
Is It not possible with the "--names" parameter in phpcpd command?
thank's for your time and your work

--names isn't supported by current phpcpd analyzer.

It could be configured in .phpqa.yml (PR is welcomed) or write custom analyzer.


Observation messages time resolution

I cannot fully recall the consensus on the time resolution; currently, we are using "%H:%M:%S:%f" which is up to 6 digits on the floating-point, therefore, has a few ms resolutions. There were discussions about using 64 bits or two time strings where the latter indicates ns after seconds.
Let's keep this issue to track.

Also related to this issue; #10.

snews has not set time format/resolution yet.

Fixed with ISO time format update https://github.com/SNEWS2/SNEWS_Publishing_Tools/commit/89acd9097dcdac01e27e09f5b0ce87a41e842575


Bump gds-api-adapters to get better errors back

Since 27.1.0, gds-api-adapters includes the request body in the error message,
allowing you to see much more detail by default. This is particularly helpful
when trying to debug Whitehall pushing to the publishing-api.
I've bumped to the latest gds-api-adapters because there's no reason not to,
and it includes some useful changes.

:+1:


Fixed whole contact page

@PriyaGhosal @samarth-6 please review and give it level 3
before

after

@AsifQamar issue link or number pls

@AsifQamar issue link or number pls

@MastanSayyad https://github.com/PriyaGhosal/BuddyTrail/issues/1970

@MastanSayyad please give it in higher level

@AsifQamar assign issue has level 1 so I have given level 2

@AsifQamar assign issue has level 1 so I have given level 2

its auto assign thats why its showing level one , you can give level 3 600 lines of code is added


fix: allow to pass custom ndkVersion to the runtime

@FestplattenSchnitzel This answer the need for this PR https://github.com/NativeScript/android-runtime/pull/1670
Instead of fixing the version i went for allowing to set it through command line.
You can then build the android runtime like this:
./gradlew -PndkVersion=20.1.5948944
@FestplattenSchnitzel why version 20.x and not 21.x ? I understand 22 might be too new but 21 should be good, right?
I just want to make sure we use a recent ndk for the runtime. I dont want any crash to appear on f-droid as we would be using an old ndk (which the dev would not know about as he builds with a newer one)

@farfromrefug when building locally on my machine Gradle automatically used NDK version 20.0.5594570. But if you're sure that 21.x works, we can use that one.

I dont want any crash to appear on f-droid as we would be using an old ndk (which the dev would not know about as he builds with a newer one)

I don't like crashed either, but every version built by the F-Droid test CI so far worked perfectly fine, so the chances seem to be good.

@FestplattenSchnitzel it is not a question of me being sure it is good. As we dont define it right now it is gradle which chooses it. So it must often be quite recent.

maybe related:
https://github.com/NativeScript/android-runtime/issues/1659

@FestplattenSchnitzel will be released with the runtime 8.2


feat: Migrate to jsonb column type for meta and component fields

This closes #736

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 70.97%. Comparing base (32de5ac) to head (e11d863).
Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #817      +/-   ##
==========================================
+ Coverage   64.86%   70.97%   +6.10%
==========================================
  Files          48        3      -45
  Lines        5946      534    -5412
  Branches      144      144
==========================================
- Hits         3857      379    -3478
+ Misses       2089      155    -1934

Components
Coverage Δ

apel_plugin
70.97% <ø> (-10.83%)
:arrow_down:

auditor
∅ <ø> (∅)

slurm_collector
∅ <ø> (∅)

slurm_epilog_collector
∅ <ø> (∅)

htcondor_collector
∅ <ø> (∅)

priority_plugin
∅ <ø> (∅)


Support for model validation

First off, thanks for sqlboiler! It's been a huge timesaver for our team.
Although, there may already be a defined pattern for handling this w/ sqlboiler, how best should model validation be handled? We would like to validate our models up front in our request handlers. Is it feasible to auto-generate validation struct tags based on the schema definition (e.g. go-playground/validator)?
If you're interested in this as a core capability, we could take a stab at it and submit a pull request.

Very glad to hear people are using it successfully (or I hope successfully!).
I think validation could be generated and have a place in SQLBoiler but I would put in a vote for a much more unobtrusive approach. A library that takes validation out of the struct might be nicer: https://github.com/kat-co/vala - it also opens up the options for people to later add more validation as they see fit without having to change the struct further. Consider the use case where the generation is not enough, users still don't have the ability to modify the struct directly and therefore they'd be unable to set any extra validation requirements that the inference didn't pick up.
Thoughts on this?

@n3integration Hello. Also just wanted to add that we added hooks as a means of validation as well: https://github.com/vattle/sqlboiler#hooks -- Not sure if this fits your model, but figured I'd mention it just in case.

@aarondl - I completely agree that there would be too many cases that cannot be predicted up front from the schema alone. Many times users would still be required to write their own validation logic, so it makes sense to provide an extension point to provide custom validation. What are your thoughts about something similar to the following?
func (m *Model) BindJSON(reader io.Reader) error {
    if err := json.NewDecoder(reader).Decode(obj); err != nil {
        return err
    }
    return Validate(m)
}

func (m *Model) Validate() error {
    ...                           // 1. Perform generated validation
    OnValidate(m)                 // 2. Perform custom validation, if hook is registered
}

@nullbio - Thanks for the hook pointer. A custom validation hook might be a good extension point.

I've been thinking more about this problem. I'm wondering how far ahead a user would truly get by using generated validation. One of the biggest problems being the errors that are generated from this stuff may need to be i18n'd and translated.
I can imagine a scenario where the error messages only serve English users well. And then everyone else would have to be able to have access to some sort of "pluggable" system to translate the errors into their language and that brings all kinds of fun.
So I guess questions to be answered in order to move forward are:

How insufficient are the current hooks at providing validation?
How much boilerplate could we truly save (considering error messages must be provided by the user)?

@aarondl - i think that ozzo-validation may be a good way to go to support validation within sqlboiler. thoughts?

Yeah, it looks very nice to me. Looks like a souped up Vala.

Not going to leave this opened as it won't be added unless someone opens a PR for it.

How insufficient are the current hooks at providing validation?

One problem with using hooks for validation is a point of consistency.  Ideally you would write the hook once and have it Just Work™, meaning thereafter your actions will always be validated.
With the current hook mechanism, you have to import your models and call models.AddSomethingHook().  If you're using these models in multiple packages, you have to remember to call init() and register the hooks each time, or make your own hooks package and import it to trigger the registration.
I thought of somehow adjusting the SQLBoiler settings to generate import statements to a non-generated package that would then add the hooks (thereby forcing their registration whenever you import the models) but that creates an import cycle.  To achieve "global validation hooks" in a type-safe fashion, they would need to reside in the generated models package.
I would suggest some sort of hooks.go file that is part of the models package but not auto-generated, so you can put the hooks there, but I have read elsewhere your opinion on this.
This could be done through settings in your config file, using the whole column-matching mechanism, and then something like ozzo-validation (as mentioned) for the validation...but it's not terribly exciting.
This particular problem is a result of making the data-representation package also do the manipulation, and making this package auto-generated.  (Not a criticism really, just an examination of tradeoffs.)
One thought is that, since the database is treated as the source of truth, maybe it makes sense to put the data validation rules in the database, a la CHECK clauses.  That might be a decent official stance, given that there is currently not an official stance, although I'm not sure how versatile this route is.

One other way to do this is to use a custom type with your own driver.Valuer implementation.  You can do some per-field validation this way that will Just Work™ although you might cause the acceptance tests to explode.

You're right that if you want validation to be part of the models package, there is literally only the option to keep it in the same package. And also correct that that sort of idea is frowned upon by us.

With the current hook mechanism, you have to import your models and call models.AddSomethingHook(). If you're using these models in multiple packages, you have to remember to call init() and register the hooks each time, or make your own hooks package and import it to trigger the registration.

There's a few oddities in this paragraph:

init is a magic Go function and is called automatically, and does not need to remember to be called. Only the import of the package containing the init does. Typically a modelsx (models extension package) is a perfect place for this.
models.AddXHook affects the global memory of the models package, that's to say it doesn't matter how many packages in a binary import models, if at ANY point in any package inside that same binary you've called models.AddXHook() the hook is now being called by all packages in that binary so the hook registration is not a per-package TODO, but a per-program todo which is slightly better.

The way I recommend doing this to utilize the above correctly is: Use a modelsx package to extend your models with extra helper functions AND an init function that registers all your validation hooks. At the top of your main / the place where you side effect import your database driver (eg lib/pq) also side effect import the modelsx package so you can now be sure that the entire program has the hooks registered.
If in the unfortunate event you forget to import modelsx for the side effects at some hierarchically high-up file, it's almost certain that you've accidentally imported it in order to use it for other helpful functions and the same result is achieved.
It is not fail-safe unfortunately, but I also don't think it's -that- bad or error prone given you only have to do this once per program.
Also FWIW I also use check clauses because I can't be sure that only Go clients will ever talk to the database and at least some basic validation should still be used.

init is a magic Go function and is called automatically, and does not need to remember to be called. Only the import of the package containing the init does. Typically a modelsx (models extension package) is a perfect place for this.

That thought came out scrambled.  I meant "call Add to register the hooks in the init() of each package".  I don't know if that mistake quite merited having init() explained to me slowly and carefully, but I apologize for the confusion.

models.AddXHook affects the global memory of the models package, that's to say it doesn't matter how many packages in a binary import models, if at ANY point in any package inside that same binary you've called models.AddXHook() the hook is now being called by all packages in that binary so the hook registration is not a per-package TODO, but a per-program todo which is slightly better.

This is a good point!

The way I recommend doing this to utilize the above correctly is: Use a modelsx package to extend your models with extra helper functions AND an init function that registers all your validation hooks. At the top of your main / the place where you side effect import your database driver (eg lib/pq) also side effect import the modelsx package so you can now be sure that the entire program has the hooks registered.

This is good advice, thanks.
I guess the one other factor is unit-tests; since these each have their own entry point, you will likely need to import modelsx in every applicable package.

Also FWIW I also use check clauses because I can't be sure that only Go clients will ever talk to the database and at least some basic validation should still be used.

Also a good point, thanks.

Entirely separately, I realized that "always-on validation" as a feature would conflict with "acceptance tests" as a feature, since the acceptance tests would have no way to ensure that the structs they're inserting & updating have valid data as per your arbitrary hooks.
So yes, the only viable route is to register them manually, and you've made some good points about how to do that.

I don't know if that mistake quite merited having init() explained to me slowly and carefully, but I apologize for the confusion.

Take no offense. People of all levels of Go understanding post issues here. Easier to assume no knowledge than to go back and forth filling in blanks.
Glad some of the advice was useful at least.
You are right about the test packages, but in general I find I'm not worried about validation happening there because it's tested properly elsewhere and also it's somewhat assumed the test writer knows how to manipulate the database properly. But it is a valid concern nonetheless. Ideally the same code would be running in both places for parity.


Plain language checklist: broken link

Page feedback alerted me to a broken link on Plain language checklist. It's the one in Written for easy reading by the average reader: https://www.plainlanguage.gov/guidelines/audience/). There's an extra close parentheses on it. When removing that, the link is good.
@YeseniaGtz OK to do as a hot fix instead of putting in our monthly maintenance release?

Merged with April release.