ERROR in Patch_segmentation_baysor: Unable to find compatible target in cached code image. [Bug]

Hi there, thanks so much for developing a great software!
I am running the cellpose_baysor.yaml config file for Xenium data, which is able to complete "cellpose resolve", but does crashes at the "patch_segmentation_baysor". I've been trying to run it starting from "checkpoint patchify_baysor" to no avail. What happens is that patchify works, and some of the baysor jobs do complete, but the rest crash with the following error:
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=128000, mem_mib=122071, disk_mb=1000, disk_mib=954
Select jobs to execute...

[Tue Mar 26 19:42:35 2024]
rule patch_segmentation_baysor:
    input: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/patches_file_baysor, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10
    output: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10/segmentation_polygons.json, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10/segmentation_counts.loom
    jobid: 0
    reason: Forced execution
    wildcards: index=10
    resources: mem_mb=128000, mem_mib=122071, disk_mb=1000, disk_mib=954, tmpdir=/scratch/lsftmp/5002506.tmpdir

ERROR: Unable to find compatible target in cached code image.
Target 0 (cascadelake): Rejecting this target due to use of runtime-disabled features

[Tue Mar 26 19:42:35 2024]
Error in rule patch_segmentation_baysor:
    jobid: 0
    input: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/patches_file_baysor, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10
    output: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10/segmentation_polygons.json, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10/segmentation_counts.loom
    shell:

        if command -v module &> /dev/null; then
            module purge
        fi

        cd /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/10
        ~/.julia/bin/baysor run --save-polygons GeoJSON -c config.toml transcripts.csv :cell

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Any guidance whatsoever would be greatly appreciated! Thank you!
Joe

Hello @jmchan88,
It's difficult to know what happens without the Baysor error log (what you gave is the snakemake log, but not the actual rule error). If you use the Slurm profile, the logs are in the sopa/workflow/logs directory.
Else, can you show me your Snakemake command line (it may help me tell you where the log file is located)?
Just some guesses of potential errors:

Out of memory error. In that case, two solutions: (i) you can reduce the patch_width_microns to run on smaller patches, or (ii) you can increase the required memory under rule patch_segmentation_baysor in the sopa/workflow/Snakefile file.
This could also be related to this issue: #41

Let me know, and I will be happy to help you!

Indeed the log file doesn't say much, and I'm not familiar with LSF clusters so I don't understand what is happening either.
Concerning the amount of transcripts, yes it is widely enough. But, since we make patches, it may happen that one patch is at the boundary of the slide and therefore contains very few transcripts. Can you check the transcripts.csv file under the /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/34 directory?

Ah gotcha! I looked at the transcripts.csv file in this folder, which had:
42835 lines
292 unique values under "cell_id."
Would that explain the error? Thanks Quentin!
Joe

Just to note, I have not done any interactive ROI selection in this pipeline (as I thought it was optional), so I do imagine that there could be patches that have no transcripts whatsoever .... is that an issue? Would this pipeline crash if there is a patch without any transcripts whatsoever?

Just as an update, I found that each time I do a round of Patch_segmentation_baysor, a fraction of the patch jobs complete and another fraction crash.  If I keep running sopa many times over, eventually all the patch jobs complete! But I wonder why a fraction of these patch jobs fail with the above error each time.

The amount of transcript you indicated should be enough, and the number of "prior" cells is also good. No, you don't need to run an ROI selection: we will simply not run Baysor on tiles with too few transcripts (if any)!
Concerning the fact that the runs sometimes crash and then succeed: it's weird to experience this kind of indeterminism. Since memory usage is not determinism, I think it really could be an "out of memory error", and this would also explain why the job crashes without any error logged.
Can you try to decrease the patch width in microns, and see if this behavior still happens?

Hey Quentin! I tried to decrease the patch width in both microns and pixels in half, but the same problem still remains ... I don't get the sense that it's even getting a chance to run for some reason
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=200000, mem_mib=190735, disk_mb=1000, disk_mib=954
Select jobs to execute...

[Tue Apr  2 16:51:01 2024]
rule patch_segmentation_baysor:
    input: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/patches_file_baysor, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136
    output: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136/segmentation_polygons.json, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136/segmentation_counts.loom
    jobid: 0
    reason: Forced execution
    wildcards: index=136
    resources: mem_mb=200000, mem_mib=190735, disk_mb=1000, disk_mib=954, tmpdir=/scratch/lsftmp/5132942.tmpdir, partition=mononoke

ERROR: Unable to find compatible target in cached code image.
Target 0 (cascadelake): Rejecting this target due to use of runtime-disabled features

[Tue Apr  2 16:51:01 2024]
Error in rule patch_segmentation_baysor:
    jobid: 0
    input: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/patches_file_baysor, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136
    output: /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136/segmentation_polygons.json, /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136/segmentation_counts.loom
    shell:

        if command -v module &> /dev/null; then
            module purge
        fi

        cd /data/chanjlab/Xenium.lung.NE_plasticity.010124/preprocessing.fine/20231013__183137__JC_PILOT_RU151_RU263_RESECTION/output-XETG00065__0005504__RU151__20231013__183151.zarr/.sopa_cache/baysor_boundaries/136
        ~/.julia/bin/baysor run --save-polygons GeoJSON -c config.toml transcripts.csv :cell

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Ok, thanks for trying this. So maybe the Julia (or Baysor) installation has an issue. I found this Julia issue related to yours, did you already have a look?
Can you try to run Julia and/or Baysor without Sopa? Just to see if the issue is related to (i) Sopa, or (ii) Baysor, or even (iii) Julia itself.

Hey Quentin! Thank you for pointing out that Github link, setting that environmental variable may be the answer! We are actually on the same wavelength, and I started running Julia Baysor last night directly at this step which preliminarily seems to work, but I'll keep you posted!

Great, thanks! If setting this variable works for Baysor/Julia, then you can try to add the variable directly in the patch_segmentation_baysor rule of the Snakefile

Actually, I'm not sure if I'm understand the solution in the Julia post. Is the solution to go into Julia and type ENV["JULIA_CPU_TARGET"] = "generic; native"

As another query, in trying to run julia baysor directly per patch, I ran the command in the Snakefile
/home/chanj3/anaconda3/envs/sopa/bin/snakemake --profile lsf --config data_path=$mdir --configfile=/home/chanj3/chanjlab/shared/sopa/workflow/config/xenium/cellpose_baysor.yaml --use-conda
However, this command doesn't seem to create the output json and loom files that would allow me to continue running the sopa snakemake pipeline after running Julia Baysor directly. How are these output files below created?
        f"{paths.smk_baysor_temp_dir}/{{index}}/segmentation_polygons.json",
        f"{paths.smk_baysor_temp_dir}/{{index}}/segmentation_counts.loom",

The ENV["JULIA_CPU_TARGET"] = "generic; native" line has to be run inside Julia, which is not very convenient for what we want to do. Instead, you can run export JULIA_CPU_TARGET="generic; native" in your shell.
For instance you can modify the patch_segmentation_baysor rule as below:
rule patch_segmentation_baysor:
    input:
        patches_file = paths.smk_patches_file_baysor,
        baysor_patch = f"{paths.smk_baysor_temp_dir}/{{index}}",
    output:
        f"{paths.smk_baysor_temp_dir}/{{index}}/segmentation_polygons.json",
        f"{paths.smk_baysor_temp_dir}/{{index}}/segmentation_counts.loom",
    params:
        args_baysor_prior_seg = args.baysor_prior_seg,
    resources:
        mem_mb=64_000,
    shell:
        """
        if command -v module &> /dev/null; then
            module purge
        fi

        export JULIA_CPU_TARGET="generic; native"

        cd {input.baysor_patch}
        {config[executables][baysor]} run --save-polygons GeoJSON -c config.toml transcripts.csv {params.args_baysor_prior_seg}
        """

Concerning your last comment, I'm not sure but can you try to add --rerun-triggers params to the snakemake command? It should re-run only the jobs whose params changed.

NB: just in case you pull the latest changes from sopa, please upgrade the dependencies via pip install -e . --upgrade or pip install sopa --upgrade, else it will not be consistent with the latest changes

Thanks Quentin! It's quite strange  ... with the new sopa version, even if I set the JULIA_CPU_TARGET to generic; native, there are random cases where I get the same error and julia baysor still seems to have JULIA_CPU_TARGET set to cascadelake

Just to be sure, Julia and Baysor work fine on their own? It breaks only when running Snakemake, is that correct?

I think there’s some issue with some of the nodes on my LSF cluster … some
insist on using JULIA_CPU_TARGET of cascadelake despite me setting the
environmental variable otherwise
On Mon, Apr 8, 2024 at 5:23 AM Quentin Blampey @.***>
wrote:

Just to be sure, Julia and Baysor work fine on their own? It breaks only
when running Snakemake, is that correct?
—
Reply to this email directly, view it on GitHub
https://github.com/gustaveroussy/sopa/issues/42#issuecomment-2042272588,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC7LTUBNOU6KWK26336QFNTY4JOYTAVCNFSM6AAAAABFOP2XMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBSGI3TENJYHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Okay, I see. Then, if it's specific to your cluster, I'll close the issue
Don't hesitate to re-open if needed! And let me know if you find a solution, I'm interested to see how to fix it, and it can help other users :)


Table align is not honored

The "align" attributes of table is not supported?
My environment is "Asciidoctor PDF 1.5.0.alpha.9 using Asciidoctor 1.5.2". This Asciidoctor PDF  changes this adoc into this image.
[.text-center]
.ほげほげ一覧
[align="right",width="50",cols="<1,^1,^1",options="header,strong"]
|=======================
|Col 1|Col 2      |Col 3
|1    |Item 1     |a
|2    |Item 2     |b
|3    |Item 3     |c
|=======================

ああああ

[.text-center]
.ほげほげ
[align="center",width="50",options="header"]
|=======================
|Col 1|Col 2      |Col 3
|1    |Item 1     |a
|2    |Item 2     |b
|3    |Item 3     |c
|4    |Item 4     |d
|=======================

If not support, is it possible to center the table in the page in a different way?

I guess we hadn't thought of it yet because we always had tables that are 100%. Prawn Table supports it, so we just need to implement it.

It will be necessary to use the role "left", "center" or "right" to be consistent with how the HTML works.
[.center, width=50%]
|===
|Column A |Column B

|A1
|B1

|A2
|B2
|===

I decided to implement the align attribute as well as an alternative (though this doesn't currently work in the HTML converter).

Thanks. Using .center , I can place the table in the center.
.ほげほげ一覧
[.right,width="50",cols="<1,^1,^1",options="header,strong"]
|=======================
|Col 1|Col 2      |Col 3
|1    |Item 1     |a
|2    |Item 2     |b
|3    |Item 3     |c
|=======================

ああああ

.ほげほげ
[.center,width="50",options="header"]
|=======================
|Col 1|Col 2      |Col 3
|1    |Item 1     |a
|2    |Item 2     |b
|3    |Item 3     |c
|4    |Item 4     |d
|=======================


Enable HA mode for Sentry

Closes #2180
This PR adds an additional check for Sentry to see if credentials are already created, and if so wait for them to be detected on the filesystem before loading them.
In addition, this PR removes a redundant channel.
This fixes a (*possible) regression in 0.11.0 where multiple sentry pods would create the certs at the same time and until the pods were restarted, the cluster would exist with 3 different root CAs loaded in memory.

possible because, although weird, this wasn't reproduced in 0.10.0

/cc @mchmarny

Please add tests for this change.

enabling HA on e2e test would not be able to 100% catch this bug. I would add unit-test to validate detectCertificates and shouldCreateCerts to prevent the regression.

enabling HA on e2e test would not be able to 100% catch this bug. I would add unit-test to validate detectCertificates and shouldCreateCerts to prevent the regression.

Writing the tests for these two cases now.
However, HA on e2e is actually what I did to catch this bug.

enabling HA on e2e test would not be able to 100% catch this bug. I would add unit-test to validate detectCertificates and shouldCreateCerts to prevent the regression.

Writing the tests for these two cases now.

Thanks.

However, HA on e2e is actually what I did to catch this bug.

I see. I agree on having HA mode on e2e test. The targeted unit-test will help a lot to find the issue earlier.


Make submodules private

Prefix submodule names with an underscore to indicate that they're not intended to be accessed by downstream projects.
Only symbols exposed in the top-level namespace should be considered public. We don't want to treat the locations of symbols in individual submodules as part of the API. Instead, we'd prefer to be able to reorganize things in the future without causing breaking changes.
In the future, we might add public submodules (such as a tophu.typing submodule for custom type annotations) to store public symbols that don't belong in the top-level namespace. But, for now, every symbol that's intended to be public should be importable directly from tophu.

any chance you could make coarse_unwrap exposed too? or do you think it's possible to get the same result with multiscale_unwrap?
(i found it useful enough that I stuck it in as a temporary easter egg while tophu was being remodeled
https://github.com/opera-adt/dolphin/blob/main/src/dolphin/unwrap.py#L424 )

any chance you could make coarse_unwrap exposed too?

Yeah, it's on my list. I have an old half-finished branch somewhere with the necessary changes. Just need to clean it up and add a unit test. Thanks for the heads up.


[8 pt] Clean up logging to make less verbose

For some of the larger HUCs, we are getting close to 10,000 lines of unit log output. This is cumbersome and difficult to sift through when debugging. There is also a cost associated with the size of these logs on AWS, albeit it is quite small.
Current behavior
There is currently a significant amount of verbosity in the logs that can be condensed.
Expected behavior
Decrease the amount of logging output.
Steps to replicate behavior (include URLs)
Run fim_pipeline.sh and look at the log files:
<$outputDestDir>/logs/unit/<HUC>_unit.log file.
<$outputDestDir>/logs/branhc/<HUC>_branch_<branch>.log file.
Or if there are errors: <$outputDestDir>/unit_errors/<HUC>_unit.log
Suggested fixes (starting point):
In the scripts run_unit_wb.sh get rid of:  date, Tstart, & Tcount
For all of the shorter routines, less than 20 seconds, we should remove the timing associated.
For the routines that take a longer amount of time, leave the timing commands.
For run_by_branch.sh - get rid of all of date, Tstart & Tcount lines, as this script takes a short amount of time to process. Remove line 3 and 4 in run_by_branch.sh as well.
In process_branch.sh , add the following to the tee command: tee --eta=never to suppress ETA log.
Additional lines that could be removed from logs:
0...10...20...30...40...50...60...70...80...90...100 - done.
-----------------------------------------------------------------
Polygonize Reach Watersheds 01100003<PHONE_NUMBER>
Split Derived Reaches 01100003<PHONE_NUMBER>

We can get rid of quite a bit of logs, but anything calling TauDEM (sometimes through mpiexec) can't be "quited" down. TauDEM has no verbosity or quiet flag options. But TauDEM is likely going away soon.


How to translate a tag, which include a inner tag

Start your  Free 10-day Trial

translate directive will grab all text in current DOM Element, so the key will be
Start your  Free 10-day Trial

My bad @hyliker regex will grab 'til first opening <
It means for your example, key will be: Start your.
It will be hard to maintain if I update all regex to take care of this.
Maybe if others guys ask for that, I'll do the job.


Refactor setup

The main thing standing out to me is in the lookup tables.  I'd much prefer the keys to be the hexes and values to be the human readable flavor.

Out of curiosity, what is your reasoning behind that?
My main reason behind that design was, that I'm using these lookup tables mainly to map/translate the human readable value to hex and not the other way around and with this design I can assure, that there is no option multiple times in that list.
But in the end that should also be true for the hexes.
I was refactoring that part multiple times and now I'm thinking that a solution I had earlier might have been the cleanest solution:
const optionsArray = [
  { name: 'On', value: 0x02 },
  { name: 'Off', value: 0x03 }
]

The only reason I dropped that design was the repeating  { name: ..., value: ...} pattern

@Kaniee the main reason is that the hex is a fixed value for what it represents but the human readable text can be subject to change.  e.g.
exports.powerModeArray = {
    'On':            0x02,
    'Off (Standby)': 0x03
}

If for some reason Off (Standby) needed to be changed to Off or Off (standby) that change would break existing selections because of the key change where-as the hex hasn't changed.  I see where you're going with the array but if you mapped them like this instead they would directly translate to the choice dropdowns, which is acceptable and done in plenty of examples.
const optionsArray = [
  { label: 'On', id: 0x02 },
  { label: 'Off', id: 0x03 }
]

@krocheck I fully agree with the with the issue on changing labels.
I remember now that this was the reason I moved to use the object array before I got annoyed by the repeating bits of code and went back to my original solution (and forgetting my initial reasoning behind the change 🤦♂)
I'll go ahead and make that change. But I'm not sure yet on the key names. I'm planning the move the VISCA specific code into a npm package so it can be used for other projects as well (including other Companion modules). Therefor I don't want to design/name stuff specifically for Companion but nevertheless it's the main driver for that code.


integrations: Add Zabbix.

Add integration for Zabbix
Testing Plan:
Ran ./tools/test-all on dev platform. All passed OK.

@eeshangarg can you review this?

@eeshangarg I'll get onto it in the next couple of hours. Just with regards to the change you mentioned about using check_send_webhook_message, is this instead of check_send_stream_message? I'm guessing the webhook message one is fairly new. I started building this in October, did most of it, got side-tracked and ended up picking it back up this week with not much left to do on it. So that might be why I used check_send_stream_message instead.

@eeshangarg I've made the changes and rebased.

@adambirds, @timabbott: Resubmitted as #9708, with a few changes to the documentation on top of the existing commit! Thanks! :)

Closing as we just merged #9708; thanks again for contributing this integration @adambirds!!


Add check for null or short length raw resource

Description
Check in both service code and database schema for small length raw resource
Related issues
Addresses [issue AB#90113].
Testing
Describe how this change was tested.
FHIR Team Checklist

[X] Update the title of the PR to be succinct and less than 50 characters
[X] Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
[X] Tag the PR with the type of update: Bug, Dependencies, Enhancement, or New-Feature
[X] Tag the PR with Azure API for FHIR if this will release to the Azure API for FHIR managed service (CosmosDB or common code related to service)
[X] Tag the PR with Azure Healthcare APIs if this will release to the Azure Healthcare APIs managed service (Sql server or common code related to service)
[X] CI is green before merge
Review squash-merge requirements

Semver Change (docs)
Patch

/azp run


Review observations: plumage

observations: plumage indicates the plumage with a letter code:
https://github.com/ices-tools-dev/esas/blob/77b9faf016e03c650c0e80ade37f6162470d0f9e/_data/table-schemas/observations.yaml#L168-L190

[x] @ices-tools-dev/data-and-information Does ICES already have a controlled vocabulary for this? no

Need new vocab
Plumage types:
•	B Breeding (summer) plumage
•	T Transient plumage (moulting between winter & summer plumage or vice-versa)
•	W Non-breeding (winter) plumage
•	L Light morph (skuas/double light ‘LL’ fulmars)
•	C Coloured morph (‘L’, ‘D’ & ‘DD’ fulmars)
•	I Intermediate morph (skuas)
•	D Dark morph (skuas)
•	P Pullus
For immature auks (Guillemot & Razorbill):
•	A1 ½ or less than adult size
•	A2 Over ½ adult size
•	A3 About same size as adult
For immature Northern Gannets:
•	G1 Plumage 1
•	G2 Plumage 2
•	G3 Plumage 3
•	G4 Plumage 4
•	G5 Plumage 5

Need new vocab
Plumage types:
•	B Breeding (summer) plumage
•	T Transient plumage (moulting between winter & summer plumage or vice-versa)
•	W Non-breeding (winter) plumage
•	L Light morph (skuas/double light ‘LL’ fulmars)
•	C Coloured morph (‘L’, ‘D’ & ‘DD’ fulmars)
•	I Intermediate morph (skuas)
•	D Dark morph (skuas)
•	P Pullus
For immature auks (Guillemot & Razorbill):
•	A1 ½ or less than adult size
•	A2 Over ½ adult size
•	A3 About same size as adult
For immature Northern Gannets:
•	G1 Plumage 1
•	G2 Plumage 2
•	G3 Plumage 3
•	G4 Plumage 4
•	G5 Plumage 5

I updated the issue body so it links to the current vocab.

I updated the issue body so it links to the current vocab.

Brief description or reference to a publication for what f.ex. "‘L’, ‘D’ & ‘DD’ fulmars" or "Plumage 1-5" in Northern gannets means would be beneficial.
As i understand, LL and DD are not valid codes, right?

All these mainly refer to traditional ways of coding, only described by means of drawings actually. The gannet codes were kept on explicit demand of the ESAS community. Despite this, future data submissions should ideally only use age coding for gannets, and distinguish between light and coloured morphs for fulmars.


Add a patch to RO2 lidar, polish up GUI of segmentation classes options

Change vector to vector
Add validation

I'm opening it here as our fork of o3de-extras doesn't have github issues.

Resolved in latest commit to pr


CLI options don't override config.rb options

In #1f3bf47, @tdreyno added support for CLI options. Although this is still undocumented it's in master. It's a bit broken (thus this issue), but the idea is you can do, e.g.,
bundle exec middleman build --build_dir some_other_dir

It works for all possible application config variables. However the logic is a little bit broken currently, as config.rb will always override the CLI arguments (whereas the opposite should be true).
This is happening in application.rb:302, the call to instance_eval.

This should help: https://github.com/middleman/middleman/commit/aab7bac8a6f531cea02572bb0a1614f9de88a932
As a bonus, config[:cli_options] is now available for looking at what was passed in.

+1, thanks!


Support for BDSP Repacker?

Maybe change from just evScript support to BDSP tools

This would really fit better as a separate extension


Double output length for claude

When the model "claude-3-5-20240620" is used in tge fubction prompt_claude in _endpoints.py, we should call client.messages.create with additional parameters:
max_tokens=8192
extra_headers={ "anthropic-beta": "max-tokens-3-5-sonnet-2024-07-15"}

This will double the length of files we can process.

git-bob solve


Add TypeScript typings

It would be nice to have TypeScript support in the project

Is handled in #7


amplify clone  existing projet

Note: If your issue/bug is regarding the AWS Amplify Console service, please log it in the
Amplify Console GitHub Issue Tracker

Describe the bug
On pulling an existing project , the cli is struck at the following message :
Amplify AppID found: XXXXXXX. Amplify App name is: XXXXX
Backend environment dev found in Amplify Console app: XXXXX
Amplify CLI Version
4.29.1
To Reproduce
Run amplify pull --appId XXXXXXXX --envName XXX on an existing amplify project.
Expected behavior
The command should be able to pull the env
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [Ubuntu]
Node Version. v13.12.0

Tried it with different apps in amplify console. Unable to pull any app using amplify pull . The cli is  struck at same message

The issue was not amplify cli specific but because of aws account


Update local-storage.js

Type: bug
Description:
Delete versions and history in package.json, to fix issue #1359.

unit-test failed

@lirantal https://github.com/verdaccio/verdaccio/pull/1389 this is the right PR :-) this is closed.

eeek, losing track fast :)


[MNT] Add warning message in README.md

Merge this PR when Nilearn make a release.
Closes #150.

LGTM. There is just a minor typo "to on nilearn" -> "to nilearn". Otherwise ready to merge.

Not sure nilearn will be released next yet. Once the release happens, this will be a one click job!
Thanks for looking out for these details!

@htwangtw now that nilearn 0.9 is out, should we merge this?

Sure! I shall update the links

LGTM! Thanks.


Publishing [x] Science! V4.9 for KSP1.0.4

Publishing [x] Science! V4.9 for KSP1.0.4
Thank you!

This looks like it needs a rebase.

Was I supposed to do something?  I did "git rebase -i" if that helps.

That's perfect, merging with my thanks.


'N4BiasFieldCorrectionImageFilter' object has no attribute 'GetLogBiasFieldAsImage'

I am running the N4 bias field correction using simpleITK's implementation.  In the example here (https://simpleitk.readthedocs.io/en/master/link_N4BiasFieldCorrection_docs.html), it has a line of "log_bias_field = corrector.GetLogBiasFieldAsImage(inputImage)".  However, when executing, the error appears "AttributeError: 'N4BiasFieldCorrectionImageFilter' object has no attribute 'GetLogBiasFieldAsImage' ".  Can someone let me know why the N4 filter does not have the attribute of 'GetLogBiasFieldAsImage'?
Thank you very much

Hello,
You are looking at the documentation for the master branch. This method was added since the 2.0 release. You are download the latest development version. Or look ad the example from 2.0:
https://simpleitk.readthedocs.io/en/v2.0.0/link_N4BiasFieldCorrection_docs.html

Thank you very much for your quick reply.  I used the simpleITK version 1.2.4 when reporting the errors.  As you suggested, I have updated to the 2.0.0.  However, I still got the error that "'N4BiasFieldCorrectionImageFilter' object has no attribute 'GetLogBiasFieldAsImage' ".   Did I mis-understand something?

Hello @dakjin
What @blowekamp suggested was that you install the latest binary files which are pre-release (you installed the official 2.0.0 release):
pip install --upgrade --pre SimpleITK --find-links https://github.com/SimpleITK/SimpleITK/releases/tag/latest

@zivy Thank you very much for the clarification. Using the lastest binary files, it works now.


VIPER Issues

Sorry to repost here, but figured this would be better than the ObjCIO Github.
Here's a few of things I am having a hard time figuring out using VIPER architecture:
If I have an entity that is just a data structure, and I have a presenter which is responsible for formatting data, why would I not want to pass the entity from interactor to presenter? I find myself unnecessarily creating two separate data structures (one on the interactor/entity side and one on the presenter side) which the interactor does a 1 to 1 mapping on. Is this correct?  Also, if you need to modify an entity based off of user interactions, how do you do this without violating layer boundaries and passing some reference to the entity to the presenter and/or view?
How does data get passed from one module to the next? Let's say I have a module that lists categories and a module that lists items in a category. When a user selects a category, that selections needs to be passed from the category module to the item module so that the item module interactor knows which category to retrieve. Data should be owned by the interactors, yet the transition between modules happens at the presenter/wireframe level. It seems I am stuck passing data from one interactor to another through their respective presenters/wireframes which results in a lot of unnecessary code and seems to violate the single responsibility of the presenter only presenting data to a view and a wireframe just transitioning between views.  There is also the layer boundary issue again because if you pass enough information for the second interactor to interact with the entity, then you are basically passing the entity... that or you have to do another ugly remapping the in second interactor. Should the interactors in different modules know about each other or communicate via a protocol? What's the best way to pass data between modules?
And what is the proper role of wireframes?  Shouldn't they instantiate the entire stack (presenter, interactor, etc.) for their module and wire them up?  Seems like this is mostly happening in the AppDependencies class which feels like it will get out of hand quickly.  If wireframes own and configure the entire stack, then it feels like this might help a bit in passing information from one interactor to another, but would still violate boundaries or require interactors to communicate.

+1 — I have a few ideas about the problems you're addressing.
For me, I think many developers including myself are left confused because the VIPER article is very vague about wireframes and relationships between modules — vitally, how they come together to form an application. A good post, but it leaves the community to fill the gaps. Your post is a good place to start.
The concept of 'wireframes' in the original VIPER post is just wrong. I would take what's been written about the routing section with a pinch of salt. The problem lies in that the wireframe class is acting as a a façade for the entire module's instantiation as well as being an endpoint (an internal interface, if you like) that lets the presenter send messages to something outside the module. There are two responsibilities here, and they must be separated.
What's more, there's no 'W' in VIPER! The intention was good but it's a bad execution. If you let wireframes talk to other wireframes directly (thereby creating a dependency), then you get something really anti-modular: a situation like this:

VIPER modules be the 'dots' ONLY. Routing and navigation is what ties the dots together (embed segues / container view controllers count as navigation!). They cannot belong in the modules themselves, otherwise your modules won't be reusable.
The key is to recognise that modules live within the context of an application and therefore we can split the wireframe's internal responsibilities from its external ones. I call the internal responsibility the 'wireframe' in this case, and the external one 'a router'.

'wireframes' are interfaces (a @protocol, even) that lives in VIPER modules. The presenter is programmed against this interface -showFooBar in response to a button press, for instance. It has no idea how it'll be done, but has faith that the thing implementing the wireframe will deal with it somehow.
router objects lives in the context of the application and is responsible for tying modules together. It typically conforms to the wireframe protocols so that it can handle calls to -showFooBar by performing a segue, or showing a view controller or whatever.
use dependency injection — it's not worth managing how objects get instantiated manually (as in the original 'wireframe' concept). Use something like Typhoon, where each VIPER module has its sown TyphoonAssembly.

It's no wonder really how virtually nobody's managed to write a demo that actually involves two VIPER modules -_-.

The original post also says that you shouldn't pass entities from Interactor to Presenter. This is too over the top and others have interpreted VIPER differently, like [over at Brigade]:
They embody some type of data and act as the “payload” that gets passed around between the other classes. For example, the data manager returns an entity to the interactor, which returns an entity to the presenter, which then uses that entity to tell the view what it should display.

So by all means do pass entities to the presenter. Ideally these should be domain objects. There's a question as to whether these entities should be rich or anaemic... up to you and your application really.

This would be so much easier if someone official actually commented on that issue. Which leaves me to think that they didn't really solve it themselves.

@fatuhoku Thanks for the thoughtful comments.  I think we have arrived at exactly the same point as to what needs to be changed to make VIPER workable on an app scale.  Unfortunately, I dove into the architecture head first for a green pasture project and now find myself doing a lot of refactoring to undo my naive application of the architecture.  At the end of the day, it seems that the key innovative piece of VIPER over standard MVC is the presenter/router layer and extracting some view logic and navigation out of the view controller.  In this way, it has a lot of similarities to MVVM or the Presentation Model pattern.
@sebastianwr I agree that the lack of official commentary on the issue is concerning.  An idea, even a good one, without proper community support tends to stay just an idea.

What I find is that a very reasonable and efficient approach is to have a class that does all the "dirty work" accompanying each view controller. This class is the entry point to the view (as in the Wireframe) and gives the VC his data (like the Interactor and Presenter). That means the view is decoupled from the implementation (if you like you can convert the entities passed to the VC to a "stupid" POOO... POObjCO... POSO... whatever) to even more decouple the view from the data.
Each "worker" or "service" has a designated method for the entry point that is called by other workers.
Also, when using MagicalRecord or something you can skip the data store and still have decoupling so that the VCs don't touch the database layer.
So basically, I combine Wireframe/Interactor/Presenter and maybe sprinkle a View-Model onto it, if it's necessary to decouple. And of course, if anything is worth having its own class (because it's an own concern) I split it from the "service" class to a separate class.
Key principle as usual is dependency injection and testability. I think that's also the basic message behind VIPER (although DI is terribly implemented in the example).

In this sample application data manager classes are contained within each module.  It seems that works in this case but in other applications data managers might need to be shared between modules.  Am I misunderstanding the role of data managers?

That's less answered by the requirements of that architecture and more by
the requirements of optimization/code readability and maintainability.
There are good reasons to go both routes.  For example:
If your data manager is creating a large memory footprint, is needed for a
prolonged period, requires a good amount of time to establish a session,
and might have session dependence (e.g. a network connection to an SQL
database using a database driver) you might want to share the object that
owns that session.
If your data manager is creating a smaller memory footprint, is only needed
sporadically, is fast to establish a session, and has no session dependence
(e.g. retrieving an object from a memory or disk-based cache) you might
want to have each module with its own data manager that is only kept alive
when needed and separates data responsibilities on a per-module basis.
The common case of retrieving objects from a HTTP-based RESTful service
that might require authentication falls somewhere in between.  Ultimately
this will also depend on how you break out data responsibilities between an
interactor and a data manager and also whether or not your data manager is
doing the heavy lifting or is an adapter to another service that might
already be shared (e.g. core data, icloud or a BaaS).
I find myself thinking a lot about these sort of (potentially unnecessary)
optimizations.  To axioms I wind up proving true time and time again are
Keep It Simple Stupid, and premature optimization is the root of all evil.
On Thu, Feb 5, 2015 at 2:21 AM, Sam Wyndham<EMAIL_ADDRESS>wrote:

In this sample application data manager classes are contained within each
module. It seems that works in this case but in other applications data
managers might need to be shared between modules. Am I misunderstanding the
role of data managers?
—
Reply to this email directly or view it on GitHub
https://github.com/mutualmobile/VIPER-TODO/issues/4#issuecomment-73024148
.

--
[image: Wild Onion Labs] http://wildonion.io/
Scott Hoyt / CoFounder
708.704.6804/<EMAIL_ADDRESS>Wild Onion Labs
http://wildonion.io
[image: Facebook] https://www.facebook.com/scotthoyt [image: Twitter]
https://twitter.com/scotthoyt [image: Linkedin]
https://www.linkedin.com/in/scottrhoyt
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Wild Onion Labs
LLC is not responsible for errors or omissions in this message and denies
any responsibility for any damage arising from the use of e-mail. Any
opinion and other statement contained in this message and any attachment
are solely those of the author and do not necessarily represent those of
the company.

@scottrhoyt Thanks for the comments.  I've been messing around with viper a little and watching some of Robert C. Martin's videos on clean architecture which help illustrate the architecture a little further.  I definitely see where you are coming from about KISS. It seems quite easy to over engineer.

@scottrhoyt Yes — in practise I have also found the Interactor and DataManager can easily be excess baggage if all your VIPER module does is to display data. i.e. not every module has business logic or needs to fetch data.
I ended up creating 'Presentation-only'  VIPER modules, which consists only of the View+ViewController, Presenter and Wireframe.

I like this discussion a lot. There is not much discussion at all about VIPER, so I ask my questions here.
Is there a good example of reusing a VIPER module? I think of a module with View, Presenter, Interactor and DataManager. How could this be reused in another app? It's tied to application and business logic through bundling all layers together. Even a reuse in the same app is quite unlikely or not? A good example would be helpful.
The whole concept of Clean Architecture is layer independence through following The Dependency Rule.
@fatuhoku I like your advice of separating the wireframe. But I have a small problem with this one:

The presenter is programmed against this interface. e.g. the presenter might send the wireframe: [self.wireframe showFooBarWithData:data fromViewController:vc].

I think this violates layer independence because the presenter has to know a framework class (a kind of UIViewController).
So do we really need module independence or do we need layer independence? Or do we need both? How can we accomplish both with an external router in place? I can't figure out the best way to do this. I'm searching for a way to accomplish an ui framework independent router working in the same abstraction layer as the presenter.
Thanks a lot to all of you in advance

Ok. So module independence is not intended for module reuse instead it is targeting on organisational concerns in the app itself.

There are a few benefits to designing your app as a set of modules. One is that modules can have very clear and well-defined interfaces, as well as be independent of other modules. This makes it much easier to add/remove features, or to change the way your interface presents various modules to the user.

@torstenlehmann I think reusability is still about the same as a strictly MVC approach.  The problem VIPER intends to sole is not one of reusability as much as testability.  The main difference between VIPER and MVC is that the navigation/UI logic/data formatting that is normally thrown into a view controller in MVC is now pulled out into a router/presenter/presenter, respectively, in VIPER.
This makes the code easier to test, read, and maintain.  But just as a view controller in MVC, there is still a high degree of coupling between a presenter and router and a specific "screen", "view", or module in VIPER.  The likelihood of reusing these components is dependent on how often you reuse the same UI paradigm in your app.  The equivalent of the MVC model layer--interactor, entities, data manager--should be just as reusable as model components in MVC and perhaps more so because of a more explicit separation of different components in the model layer.
So, in summary, module reuse will probably be low.  Layers, on the other hand, become more reusable as you move lower from UI to data/model.

I found this discussion really helpful, however I have a question.
In an app I'm developing, I have a conditional navigation at app startup.
In case the app is first launched, I have to show a walkthrough, otherwise I will land directly on one of two screens: a password view controller, or the main interface.
The view controller that should be shown is inferred by a couple of values stored on the device (user defaults, keychain and so on).
The question is: where should I implement all this logic?
It doesn't seem to belong to the presenter (or to the interactor), since I don't have a single one.
It seems to fit well inside the wireframe, but I'm not sure, since it seems to break SRP (it would implement both the animation logic and the what-to-show-next logic, based on stored data).
Or would be better to simply create a companion object that mimics the interactor purpose but coupled with the wireframe instead of the presenter?
Ideas are welcome :)

@alessandroorru  really good questions that I had been wondering for a while too. I like your thinking starting from ground up. I ran into similar issues and have some thoughts.
People using VIPER seem to have found the concept of a 'root' module useful. Check out https://github.com/mutualmobile/VIPER-TODO/blob/master/VIPER TODO/Classes/Common/View/VTDRootWireframe.m for example.
Work with more than one root view controller

In an app I'm developing, I have a conditional navigation at app startup.
In case the app is first launched, I have to show a walkthrough, otherwise I will land directly on one of two screens: a password view controller, or the main interface.

The first thing iso recognise that these are all rightfully top-level, or root view controllers of your application. i.e. they have a right to be shown directly in your UIWindow without any trickery from storyboards or segues or other VIPER modules.  Yes, this is contrary to just about every tutorial on iOS, but I think it's important to see this.
How can you even manage root view controllers without resorting to the storyboard (and thus being tied to one 'decision making' ViewController... quite horrible)?   Well, I found https://github.com/timshadel/TSAnimatedRootViewSwitcheroo.  It helps you switch between roots.
Cool. Let's elaborate a little bit.
Root VIPER Wireframe provides the Window and manages navigation between root view controllers
Starting with a blank slate, you have an application with storyboards disabled, so when you Cmd+R, all you see is black.

AppDelegate creates / gets a RootWireframe  injected. AppDelegate performs some basic  initialization tasks and then asks the wireframe for a window to display (https://gist.github.com/fatuhoku/d4f39cb6e7075c3a70d1). This window is, from now on, managed by your application.
RootWireframe thinks to itself. Okay, do I show the login screen, the walkthrough, or the main interface right now?. This is essentially a navigation decision. This is normally driven by presenters. So okay, let's make a RootPresenter and ask it which interface should be shown.
RootPresenter might need additional information like "Is the user signed in?" or "Is this the user's first launch?" to decide whether it should show the walkthrough, signup or main interface screens. This information might come from your database. In any case, RootPresenter goes and asks a RootInteractor for those details.
RootInteractor comes back and says, for example, the user is signed in already. RootPresenter decides to tell RootWireframe that it should show the main interface.
RootWireframe goes off and tells the MainInterfaceWireframe to initialize itself, and asks it for a UIViewController to display.

When a user decides to log out...

MainInterfaceWireframe tells the RootWireframe that the sign up screen should be shown again.
RootWireframe asks SignUpWireframe for a UIViewController, and then uses something like TSAnimatedRootViewSwitcheroo to  switch from the main interface to the sign up screen.

@torstenlehmann about layer independence, I've been thinking about it the last few days... you are right — Presenters really shouldn't know anything about UIViewControllers. So perhaps working with storyboards in VIPER is unsatisfactory for this reason: your Wireframe class absolutely has to own UIViewControllers for the presenter to not have to touch UIKit at all.

@fatuhoku thank you for your helpful answer.
I was already ending up with a solution similar to the one you proposed, but I was using a sort of interactor directly linked to the wireframe. My mistake was that I was missing the concept that is the presenter that leads the navigation, while the wireframe/router duty is to just perform the requested navigation.
I have another question for you. I'm trying to integrate a DI framework (typhoon) in this project, and I have some doubts about what should be injected. I surely would construct with DI the object graph of wireframe, presenter and interactor, but what about view controllers? Do you think it is ok to instantiate them (and link to the presenter through protocols) directly in the wireframe, or should they be injected too? I use storyboard only as containers, so I instantiate VC by code through storyboard IDs.

@alessandroorru I'm exploring the topic as well myself — so I don't really have the perfect formula. As with these things, there's no silver bullet.
What I will say though is that I've found that using a DI container like Typhoon has been useful in providing a declarative way of laying out your components on the table, and by allowing you to group your application's dependencies logically, it enables much clearer thinking about how you might manage those dependencies (i.e. actually understand your application architecture) and lets you begin the process of optimising the dependencies.
I've found Typhoon's storyboard support to be quite solid, and so you could probably have them injected!
Again, segues though aren't so great, because then UIKit is instantiating your view controllers, and it's not so easy to hold references to them from your wireframe.

My observation is that this discussion on issue #4 is going to grow. :D
Any recommendations for a better communication channel? What about a VIPER discussion repo? Or architecture case studies in general.
I have some reservations regarding too much flexible architecture upfront. Like premature optimization is the root of all evil, I tend to say that premature flexibility could be also problematic.
So I like this discussion a lot to review my own approaches.

I have a question about communication between modules, I think @fatuhoku was discussing something similar here earlier, so I hope you could give me some advice. The use case I’m working on is dead simple - I have a list of categories, user selects one of it and I push a view controller displaying category contents. The first question is wether this should be one module or two modules.
I think that it should be 2 modules (list module and category module) so the second question is how I should pass an entity between modules? What kind of entity should it be?  Probably not a NSManagedObject, may be entity created by the interactor from NSManagedObject? I guess not a struct that a presenter operates with? Although it was a presenter that got notified that a particular category was selected, so should it ask interactor for its entity and then pass it to wireframe so that it could pass it to the next module? Is it okay for a wireframe to know about entities at all?
It seems like this is the easiest problem and I’m the only one missing it, so I would appreciate any thoughts on this.

@daria-kopaliani My 2 cents:

Surely you have to abstract away all the persistence layer. Nobody should know about CoreData but one or many data stores, and Interactor should only talk with data stores. Managed object should be converted to application models by the data stores, possibly with structs if you are using swift in order to keep their immutability. Keep in mind that models should not have any business logic inside, just variables and convenience methods.
Interfaces between interactor and data stores should be implemented through protocols. In this way, if one day you decide to totally or partially remote CoreData, you should only switch to different data stores and you don't have to change all the app. Moreover, writing tests, you won't have to relay on fixtures on CoreData or heavy mocks! You just need a different data store that conforms to the same protocol the interactor expects and that returns the data you want to pass around.
Anyway read again the VIPER guide, it describes this process a lot better than me.

Following VIPER original guidelines, the interactor should not pass models to the presenter, but should convert them to a minimal aggregation of information that the presenter need. Anyway, IMHO, this is quite limiting and a bit redundant. Many times I have found myself writing another struct that just contains the same data. So I think that it's not worthy to introduce another layer here, and I think it is ok to pass around models, as long as all the presenter do with them is just consume their data to update the UI.

About passing around data, I have some doubts too, maybe @fatuhoku has better ideas. I think it is ok to pass the data to the wireframe as long as it purpose is just configure the next view, for example I think it is ok if you have something like this:

func presentDetailInterfaceForCategory(category: Category) {
       let viewController : CategoryDetailVC()
       viewController.category = category
       // present view controller ...
}

Has anyone used storyboard embed segues with view controllers?

@torstenlehmann hmmm. I could make my Presenter not have a direct reference to the UIViewController by using delegates, but Presenter acts as a data source, and therefore knows about UIKit. Perhaps it's time to move all of my UITableViewCell logic back into the ViewController. Eek.

@fatuhoku Yeah. Teasing details. At first I think you have to decide if it's really worth it to completely isolate the UIKit stuff. As I mentioned maybe it's too cumbersome to do this and you don't have enough resources (time, money, co-workers) to make it this clean. That's a point where I'm searching for the sweet spot and guidelines.
But if you want to divide the two layers completely, you can try the following ways maybe. It all depends on what you want to archieve. Is your view only asking for data? Should your view being updated by external events?

One View Controller meets One Presenter (updating entire list in one go)
Put all you CollectionViewDelegate and DataSource methods to the viewController. The presenter should not know, how the view gets its work done. A good test is to imagine how it would be if you switch your collection view with a webView. This must not change your presenter.

The pattern now is to pipe all this data through one method. Define a method showCollectionItems on the viewController and add this method to the view protocol which the viewController implements. In this method reload the collectionView with the items you are passing as argument to showCollectionItems. This updates your entire list in one go. But in some cases you have to pass a really big list. And updating individual cells is also impossible this way.

One View Controller meets Multiple Presenters (updating each cell)
The pattern here is to pass multiple mini presenters from your main presenter to the viewController. This gives you fine grained control over which cell to update instead of getData-mapData-pushData in your main presenter. With regards to layer independence its okay that the view has to handle the mini presenters. This pattern is a step into the direction of view binding. I don't have much experience with this. I thought of this last time I build a list with downloads and each item should show its progress. That's quite annoying to do with one presenter only, because the main view has to handle each update on any cell. I don't know how good ReactiveCocoa fits in here. So the main presenter pushes the mini presenters to the view if the collection of data changes. The mini presenter updates a cell if the cell content for this data item changes. This sounds for abstraction and a framework in the long run questioning the clean architecture way. :D

Your idea to let the presenter implement the delegate methods is also a nice one I did not think of. Sure, you have to create separate methods which the delegate methods can call on the presenter. This is kind of a pulling concept and is nice for big data sets. The collection view can even ask the main presenter for a range of items or mini presenters (a kind of window on the whole data set). This makes it possible to execute batches of fetches on the database or webservice. You can also push a kind of virtual proxy to the view which encapsulates this on demand methods. Its kind of a NSFetchedResultsController but independent of the data layer.

Yes. Maybe this combi is the most powerful. I will consider this.

@alessandroorru

Following VIPER original guidelines, the interactor should not pass models to the presenter, but should convert them to a minimal aggregation of information that the presenter need.

I have my problems with this, because it violates layer independence. The interactor should do nothing what the presenter is there for. You can increase reusability of the interactor if the presenter maps the items from the interactor to view items (if this is really needed).
I like your common sense to use the entities if they are the same for the view. All what this shows to me is that apps often aren't business level systems. They can evolve to that later. Immutable data is no problem at all and a presenter that uses entities still follows the dependency rule to depend on inner layers.

@daria-kopaliani I would pass the minimal amount of data needed to do the work in the category module. Most of the time this is a categoryID (but not a rowId of a database, a database independent id, maybe an id coming from your webservice, a system wide id if you have a webservice and app combination). The categoryPresenter asks a categoryInteractor which asks a dataManager which asks a concrete dataStore for this category with this id. Yeah a lot of layers which are often not needed. This is caused by the simplicity of getting data. It's kind of the simplest use case imaginable. You have no filtering, searching, calculating in there. That's what I mean by not using too flexible architecture upfront. View, presenter and dataManager and dataStore is enough flexibility for this case. And you can reuse the dataManager and dataStore on many other modules in this app. If you have more logic than getting data introduce an interactor between presenter and dataManager. But even if you replace your local database with a webservice api you only need to introduce a second concrete dataManager. I tend to say that you can even omit the presenter if fetching and presenting this category is the only thing that you are going to do. It makes no difference if you build this new layer on demand at a later time or upfront. But you take the risk that this flexibility is not needed later. You can argue that you want to test the presenter. But testing this glue code has no real value. Testing domain logic has real value. Testing mapping code in the presenter has value (yes I have thrown in 4 bugs last time I wanted to put a forename, surname and a separator together: null, "", "someName" on both fields, and you have 9 possible states to handle – doubled by the separator included or not, results in 18 states). That's what tests are for – to narrow down this complexity.
So like alessandroorru said. First step wrap core data and use plain objects. Then choose alessandroorru's advice or my advice. I think they both work quite well, because it's not a complex thing at all unless you have any external events which should update the category view (e.g. some new items arrived in the database and should be shown). VIPER (and clean architecture) is a pattern for managing complexity. I have written an app recently and it felt quite bloated because its such a simple app and i used VIPER. That's why I'm searching for this process to introduce new layers on demand. I liked the book Refactoring by Martin Fowler a lot. But it aims on refactoring objects not layers. So I appreciate any advices.

What about NSFetchedResultsController?
They have a great value through index path based updates, but their place should be the data manager layer, before the interactor, abstracting away the CoreData dependency. This becomes a VIPER limit, if I cannot reproduce the same behavior all the way up to the presenter.
What are your thoughts or ideas about it?

@alessandroorru my thoughts on this
What are the requirements?

On demand fetching of data to save memory
Using strictly separated layers
anymore?

The important point to get is: If you have memory constraints you have to pull the data on demand. This is what NSFetchedResultsController basically does. Don't push a big list of data on viewDidLoad.
A shortcoming of Uncle Bobs explanation on clean architecture (as far as I understand) is this entity fetching behaviour. He often speaks about systems where every object can practically fit into RAM. For him a database is only the persisting mechanism. As you can see Core Data and SQLite is a little bit more. It's also a search and filter mechanism that is quite fast and memory efficient. If Core Data would only be the persisting mechanism you should fetch everything into RAM and do searching and filtering in RAM with predicates on collections. For big data sets this is not realistic (or am I wrong?).
So lets add a third requirement:
3. We want fast and memory efficient searching and filtering on large data sets.
Can we build this on our own? I don't think so. We have to made a decision now. I don't know of any library that does this for us in memory. But Core Data solved our problem quite good. This presents an important insight. Frameworks are here to help us! Sometimes you can write things on your own to not depend on a framework and sometimes you want to safe time or are overchallenged. For my own skills writing the SQLite database would be a bit hard. So thx to the SQLite-Team and thanks to Apple. This is how life works. If you need help, you depend on someone else.
Now we want to use this searching and filtering capabilities of the database. The definition of fetching now changes from loading all data into RAM to loading searched and filtered data into RAM. This leads us to the following solution:
Use the NSFetchedResultsController only in the Core data layer. That way you need a dataManager per fetchRequest because the NSFetchRequest belongs to Core Data. You can build a flexible dataManager taking every configured fetchedResultsController (configured with a fetchRequest) as a parameter in the constructor. You can ask you presenter for viewModelAtIndexPath: which asks an interactor for resultModelAtIndexPath: which asks the dataManager for entityAtIndexPath:. So it all stays the same but you are mapping only one entity and you are doing this on demand. If this mapping is worth while depends on the complexity of the system. Passing the indexPath trough all layers is ok, because it's a complex version of a position.

@torstenlehmann Thanks for your answer, anyway I think that the biggest problem is the other way around. Think about the fetched results controller delegate: its methods are called whenever a new object is added/modified/deleted from the current result set. This is a behavior that I want to preserve: I want to be able to get notified at presenter level about new objects that has been updated and at which index path.
I think that your idea of a dataManager per fetch request is a way to go, I have to give it a thought ;)

@alessandroorru If it's not about persistence at all you can use KVO on arrays to build your own EntityListObserver in your use case layer.
If you want to preserve layer independence every delegate call has an equivalent delegate call in every layer if the event is originating from inner layers. Is this good design? Hard to see why this should be. The use of an NSFetchedResultsController means that you are only presenting data from a database. There is business logic than presenting data. I would wrap Core Data, because I don't like the API that much and call this wrapping API from the presentation layer (presenter/view). No interactors needed in this case.

@fatuhoku The gist you reference above i.e. "https://gist.github.com/fatuhoku/d4f39cb6e7075c3a70d1"
is no longer available. Any chance you still have it around - would love to read it.
Many Thanks.

If you have an application that talks to both a web api and a database how many dataManagers should you have one, two or three?
Case:
a) dataManager
b) APIDataManager and LocalDataManager
c) dataManager, APIDataManager and LocalDataManager
Where in
a) The interactor talks to a single dataManager that talks to any services you may have (remote or local).
b) The interactor knows the difference between local and remote information - and calls either the APIDataManager or the LocalDataManager, which talk to remote and local services respectively.
c) The interactor on talks to a general dataManager, the general dataManager then talks to the APIDataManager and LocalDataManager
"But even if you replace your local database with a webservice api you only need to introduce a second concrete dataManager." - @torstenlehmann. Any thoughts?

@sao101 It depends on where the abstraction lies within your app, that is distinguishing what you do from how you do it. Who is defining that there are two different data stores?
If local and remote data stores are part of the problem domain itself (e.g. sometimes the problem requires fetching remote data, and other times it requires fetching local data), it is sensible for the interactor to know about the two different data stores.
If the Interactor only cares about what data is requested, but it does not care about how the data is retrieved, it would make sense for a single data manager to make the determination of which data source to use.
There are two different roles at play here—the business designer, and the data designer. The interactor is responsible for satisfying the needs of the business designer, i.e. the business logic, problem domain, etc. The data layer is responsible for satisfying the needs of the data designer, i.e. the server team, IT team, database team, etc.
Who is likely to change where you look to retrieve data, the business designer, or the data designer? The answer to that question will guide you to which class owns that responsibility.

@sao101 "But even if you replace your local database with a webservice api you only need to introduce a second concrete dataManager."
What I'm describing here is an example of programming against an interface/protocol not implementation, so you can switch out data managers on demand.
 It's a question of making the data retrieval process visible to the interactor or not. Try to write the interfaces and objects for both cases. What is your intuition? What feels better? Can you find facts why this feels better? What is the most likely change that can occur to the requirements? Which things must be rewritten at that point?
I don't use VIPER upfront. I write, grow/rewrite and refactor until the modules fall into place. Software requirements are changing constantly. For me it's not reasonable to have one pattern for all of them. VIPER reminds me that I ask this question more often: Can I separate concerns better here?

@jeffgilbert @torstenlehmann - Thank you very much for your explanations and feedback guys. I really appreciate it.

@fatuhoku Hi. Your gist https://gist.github.com/fatuhoku/d4f39cb6e7075c3a70d1 is not available anymore. Can you please post it again?

@egze Sorry I think I've removed it!  Might not be able to recover that snippet now. :/

Hi, I am using VIPER for application written in Swift. I have a situation when app contains a lot of actions based on dialogues with reusable views. So I decided to split each dialog to the following modules:

Dialog - dialog module that is responsible for navigation between dialog screens and for updating state of the dialog target.
Dialog Screen - dialog screen module that is responsible for displaying screen UI, fetching data from data source. It does not know anything about next dialog screen and about dialog target, but it can notify listeners through module output protocol about changes.

With this approach I can reuse any view of the dialog (and I think it is pretty SOLID solution), but is this a good approach? It seemed to me that VIPER article implies that module should know how to advance to the next module.
Also I have decided to use Swift module (separate build target) for every VIPER module and I have a question about it. What is the best practice about providing entry point to VIPER module. Currently I use module function that creates router and presents module.

Hmmm. I think 'VIPER' as a concept is given more credit than it deserves. As we've discussed there, there's components whose responsibility are half-baked.
I recently saw this video and understood where VIPER came from.
Then I found this: http://clean-swift.com/clean-swift-ios-architecture/. A certain Raymond Law wrote up how he thought of applying Clean to Swift.
You really want to be pushing your implementation details to the edges. Protect the details of your business logic.

Who should own whom?
I've seen in some VIPER demo, that no weak references are used at all, probably resulting in massive memory leaks. In another blog I read that the wireframe should own the viewcontroller, which makes sense hierarchically, but on the other hand a navigationcontroller will retain its viewcontrollers too. So when our wireframe retains the view, we need a rootwireframe that manages the navigationcontroller, and watches its delegate calls, as it might pop the viewcontroller due to the back button, and the rootwireframe would need to release the child wireframe too. For simplicity's sake, in our case we decided that the viewcontroller owns the presenter, and the presenter owns both the wireframe and interactor, but it kind of feels wrong.
I'd be interested on other takes on this.

If there are no weak references between view–presenter–interactor, that is a bug. Typically, no-one should have a strong reference to the view controller, because we can depend on iOS to control the lifetime of the view controllers.
Assuming the view controller is properly managed by iOS, you want strong references to flow from the view controller to its dependencies: view -> presenter, and presenter -> interactor. You would want weak references flowing back to the view controller: interactor -> presenter, and presenter -> view. This also makes sense because the weak references can be considered channels to communicate results. As far as dependency management, I like strong references to objects that you need to perform your work, and weak references to communicate results.

That's like we have it now + the presenter has a strong reference to the wireframe, same for you?

My current approach is to have a factory and wireframe for each different lifetime within the app. e.g. for objects with the lifetime of the app, I have an AppFactory, and an AppWireframe; for objects with the lifetime of a modal transaction (i.e. modal view), I have a Factory, Wireframe, etc. The lifetimes are usually associated with some containing view in iOS. e.g. app life objects are associated with the main window, modal lifetimes are associated with a modal view, etc.
I use objc_setAssociatedObject to associate the wireframe with its containing view. e.g. AppWireframe is an associated object of the main UIWindow, a Wireframe is associated with its modal view, etc. This way, when the modal view is dismissed, the wireframe, etc. is released.
Doing the above, I use a weak reference from the presenter to its wireframe, since a wireframe will have a longer lifetime than a particular presenter.
But, if the wireframe is already going to outlive the presenter, it's probably ok to have a strong reference to the wireframe :-)

Nice architecture @JeffGilbert-MM! Though the objc_setAssociatedObject makes it a little less applicable to Swift.
I have to ask the question though. How do you pass data between views? Is it through all the presenters and wireframes?
V1 -> P1 -> W1 -> W2 -> P2 -> V2

Or is there another channel created to pass data directly from presenter to presenter or view to view?
I have done it the former way, but it certainly adds a lot of boilerplate.

You can do it from Swift. I wrote this code:
extension NSObject
{
    /**
     controlLifetimeOf(navigator:) allows the lifetime of a navigator to match the lifetime of another object.
     This is useful to avoid retain cycles between a view controller, navigator, and presenter.

     - parameter navigator: the navigator whose lifetime you want to match to the target of the function.
     */
    func controlLifetimeOf(navigator: AnyObject)
    {
        objc_setAssociatedObject(self, &AssociatedKeys.Navigator, navigator, .OBJC_ASSOCIATION_RETAIN_NONATOMIC)
    }

    fileprivate struct AssociatedKeys
    {
        static var Navigator = "navigator"
    }
}

Depending on what needs to be passed, the presenter can pass data to it's wireframe. Other times, it's appropriate for the data layer to implicitly pass data between interactors. e.g. one interactor can write data into its data store, and the next interactor can read data from the data store.

That would definitely work for Swift classes that inherit from NSObject. We like to stay in pure Swift land as much as possible though. It also forces your hand on using reference semantics, and I think value semantics have a place particularly in I, P, E, and W layers to discourage creating shared global state.
The direct interactor to interactor passing of data is something i have tried, but not using a data store to accomplish it--again trying to reduce global state as much as possible. Instead I used a protocol that allowed the two to communicate directly. But that basically boils down to passing the data:
I1 -> P1 -> W1 -> W2 -> P2 -> I2

because you have to pass the protocol object to the other interactor and wireframes don't have direct access to interactors.

We discarded using structs for I, P, W, as it prohibits to have weak references to them, and we consider every object having a property of some class object to be a class itself, as it encapsulates changing state.
http://faq.sealedabstract.com/structs_or_classes/

@JeffGilbert-MM: In an experiment I tried the Wireframe owning the ViewController. Works very nicely, and makes sense IMHO, as the Wireframe manages the ViewController, not the other way around. We have to track state very closely though, so the Wireframes (and thus their ViewControllers) get deallocated properly when the system executes a navigation action beside Wireframes (e.g. using the back button or back swipe).
Here is the example project: https://github.com/willhaben/Wireframes

@JeffGilbert-MM  you explanation about strong vs weak helped me a lot.
@scottrhoyt I'm also still confused about passing data between views (modules for that matter). I guess I'm still confused about passing objects (entities) between modules. If the presenter or the router don't know about the entity. I'm not sure if the confusion comes from the VIPER diagrams on the internet. Why can't we just simply initialize the view controller (module) that we're navigating to and pass the object that we need?

@jeffgilbert @scottrhoyt @sebastianwr @torstenlehmann @fatuhoku @samwyndham
Guys I got a Facebook like feed in my view controller so I decided to rewrite code with VIPER .
I got one question, where should I maintain my array of posts (MyPostArray) in the VIPER .
Any suggestion would be highly appreciated.

@octopusbaba Presenter ....


Single node multi GPU training using train_detector

How to use this API for multi GPU training?
train_detector(model, datasets, cfg, distributed=False, validate=True)

It needs to set distributed=True and set up distribued training to train with multi GPU. You can refer to tools/train.py.


Add more concise documentation of arguments for the attributes

Hi Jonathan,
In the list of datatypes on the website, it could be helpful to give an example string for each datatype (or at least those where strings are not lengthy). In most cases the sytax is obvious, but with Boolean values, lists, or dates, for example, the syntax may be less clear. If this makes the website to crowded (which I think it may do), maybe there could be an extra dataset file that contains one item for each datatype, just for the sake of an example?
Best,
Wolf

There a quite a bit of optional arguments. There is already a link to more detailed documentation at http://docs.karrlab.org/obj_tables/master/1.0.8/source/obj_tables.html.

I realized that the constructor arguments weren't displayed in the documentation. This is now fixed.

In addition to the API documentation, it would be helpful to have more concise documentation of the arguments to the constructors for the attributes.
This could be assembled by parsing the docstrings of the constructors and compiling the docstrings to HTML with sphinx.

Hi Jonathan,
that's a good change, but originally I meant something much simpler. For example, with "Boolean", users may not know what's the required format, eg "True", "true", or "1", or whether several of these are allowed. Similar for lists, the user may not know whether brackets are needed, which separator to use, etc. Of for numbers, whether an exponential notation is allowed, etc. So my idea was to put some examples, for those cases that are not self-evident. But I think it will make the webpage too crowded, so I don't really know where this could be put.
Best,
Wolf

I see. I could examples to the compiled documentation and then extract that into a separate document.
I've tried to make ObjTables flexible. For example:

Boolean -- XLSX booleans or strings that encode any of the above
Numbers -- can be XLSX numbers or XLSX strings that encode numbers (exponential notation is supported)
Lists: essentially this supports the Excel dialect of CSV with , as the default separator. Values that includes the separator can be encoded with " delimiters. Example: "val-1", "val-2", val3 would be parsed to ["val-1", "val-2", "val-3"]. The separator can be specified in the argument to the attribute, e.g. List(separator=','). White spaces before and after the separator are ignored.


fix Jenkins URL to match github repo

The redirect for Youscribe/htpasswd-cookbook => redguide/htpasswd works on github, but not for Jenkins.

@mburns thanks ;)


chore: new contract and config

New contract version v6
Updates addresses for ganache and staging.

That fix is thanks to @nduchak who found it out, I was stuck 😅


[riverpod_generator] Add the ability to change the generated Provider's name

Is your feature request related to a problem? Please describe.
Most of the time I work with riverpod_generator, I struggle to find a consistent naming system according to the current riverpod_generator implementation.
Let's go straight to 2 fast examples.
Example 1
Sometimes I would like to name the Provider's class/function and the model the same
[Filename: user.dart]

import 'package:freezed_annotation/freezed_annotation.dart';
import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'user.freezed.dart';
part 'user.g.dart';

@freezed
class User with _$User {
  const factory User({
    required int id,
    @Default('User') String displayName,
  }) = _User;
}

@riverpod
class UserState extends _$UserState {
  @override
  User build() => // ...

  void changeName(String name) =>
    state = state.copyWith(displayName: name);

  // ...
}

/* ---------------------------------- */

[Filename: my_app.dart]

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

import 'user.dart';

class MyApp extends ConsumerWidget {
  // ...

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    final anotherUser = User(id: 4);

    // ...

    return MaterialApp(
      // ...
      home: MyScreen(
        user: ref.watch(userStateProvider);
        onNameChanged: (name) => ref.read(userStateProvider.notifier).changeName(name),
      ),
      // ...
    );
  }
}

Problem: I have to rename the Provider's class to something else (UserState in this case), creating a userStateProvider instead of the desired userProvider
Example 2
In this example I would like to use ThemeMode inside a Provider in order to dynamically change the app theme mode (system/light/dark).
[Filename: theme_states.dart]

import 'package:flutter/material.dart' as material;
import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'theme_states.g.dart';

@riverpod
class ThemeMode extends _$ThemeMode {
  @override
  material.ThemeMode build() => material.ThemeMode.system;

  void set(material.ThemeMode mode) => state = mode;
}

/* ---------------------------------- */

[Filename: my_app.dart]

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

import 'theme_states.dart' hide ThemeMode;

class MyApp extends ConsumerWidget {
  // ...

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    return MaterialApp(
      // ...
      themeMode: ref.watch(themeModeProvider),
      home: MyScreen(
        onMyButtonTap: () => ref.read(themeModeProvider.notifier).set(ThemeMode.dark),
      ),
      // ...
    );
  }
}

Problem: I have to put an alias on package:flutter/material.dart and hide ThemeMode wherever I use flutter/material and the themeModeProvider provider together.
Describe the solution you'd like
My solution is to add a prop to the @Riverpod() annotation called name.
What this prop does it's to change the resulting variable name associated with the Provider's class/function to the annotation's name prop + configSuffix (ex. Let's say that we use the default prefix: @Riverpod(name: 'user') + 'Provider' = 'userProvider').
Let's go through the examples I wrote before and fix them using the proposed solution.
Example 1
[Filename: user.dart]

import 'package:freezed_annotation/freezed_annotation.dart';
import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'user.freezed.dart';
part 'user.g.dart';

@freezed
class User with _$User {
  const factory User({
    required int id,
    @Default('User') String displayName,
  }) = _User;
}

@Riverpod(name: 'user')
class UserProvider extends _$UserProvider {
  @override
  User build() => // ...

  void changeName(String name) =>
    state = state.copyWith(displayName: name);

  // ...
}

/* ---------------------------------- */

[Filename: my_app.dart]

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

import 'user.dart';

class MyApp extends ConsumerWidget {
  // ...

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    final anotherUser = User(id: 4);

    // ...

    return MaterialApp(
      // ...
      home: MyScreen(
        user: ref.watch(userProvider);
        onNameChanged: (name) => ref.read(userProvider.notifier).changeName(name),
      ),
      // ...
    );
  }
}

Solution:

UserState changed to something more meaningful like UserProvider and added name: 'user' prop to @Riverpod() annotation
The resulting provider name will be @Riverpod(name) + configSuffix = userProvider

Example 2
[Filename: theme_states.dart]

import 'package:flutter/material.dart';
import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'theme_states.g.dart';

@Riverpod(name: 'themeMode')
class ThemeModeProvider extends _$ThemeModeProvider {
  @override
 ThemeMode build() => ThemeMode.system;

  void set(ThemeMode mode) => state = mode;
}

/* ---------------------------------- */

[Filename: my_app.dart]

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

import 'theme_states.dart';

class MyApp extends ConsumerWidget {
  // ...

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    return MaterialApp(
      // ...
      themeMode: ref.watch(themeModeProvider),
      home: MyScreen(
        onMyButtonTap: () => ref.read(themeModeProvider.notifier).set(ThemeMode.dark),
      ),
      // ...
    );
  }
}

Solution:

ThemeMode changed to something more meaningful like ThemeModeProvider and added name: 'themeMode' prop to @Riverpod() annotation
The resulting provider name will be @Riverpod(name) + configSuffix = themeModeProvider
Since the classes names are different now, I can remove the aliases and the hide keyword when importing

Describe alternatives you've considered
Another solution could be not to add the suffix to the generated provider's name if the annotated class/function already ends with it.
Example
Let's suppose that I didn't changed my build.yaml, so the suffix is Provider:
[Filename: user.dart]

import 'package:riverpod_annotation/riverpod_annotation.dart';

part 'user.g.dart';

@riverpod
class UserProvider extends _$UserProvider {
  // ...
}

Since the suffix is Provider and the end of UserProvider too, we get:
[Filename: user.g.dart]

// ...

final userProvider = Provider(...);

Instead of the current implementation:
[Filename: user.g.dart]

// ...

final userProviderProvider = Provider(...);

Additional notes
Maybe I'm missing something. If it is like so, please tell me.
If this solution will be approved I could try to take care of the implementation myself through a PR, if you prefer.
Thank you for your time.
Niccolò Gorraz

Possible overlap with issues #2244, #2261, #2279, #3130, #3130, #3189 and #3203

Duplicate of https://github.com/rrousselGit/riverpod/issues/2244


This extension not working in Espocrm 7.0.9

Unfortunately Espo 7 was a major code refactorization and many extensions became obsolete.
This is a free extension thus there are no plans to upgrade it,


build(composer.lock): patch upgrade of dependencies

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
98

Relevant Lines:
100

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
98

Relevant Lines:
100

💛  - Coveralls


start error ,cannot load library 'libcairo.so.2 ,Error importing flask-weasyprint!

python manage.py run0
Error importing flask-weasyprint!
PDF support is temporarily disabled.
Manual dependencies may need to be installed.
See,
    `http://weasyprint.org/docs/install/#by-platform`_
    `https://github.com/Kozea/WeasyPrint/issues/79`_

cannot load library 'libcairo.so.2': libcairo.so.2: cannot open shared object file: No such file or directory

2015-06-16T05:57:21.702+0000 I NETWORK  [initandlisten] connection accepted from <IP_ADDRESS>:40963 #9 (1 connection now open)
/data/quokka_venv/lib/python2.7/site-packages/flask_script/init.py:153: UserWarning: Options will be ignored.
warnings.warn("Options will be ignored.")

Running on http://<IP_ADDRESS>:8000/ (Press CTRL+C to quit)

Restarting with stat
Error importing flask-weasyprint!
PDF support is temporarily disabled.
Manual dependencies may need to be installed.
See,
http://weasyprint.org/docs/install/#by-platform_
https://github.com/Kozea/WeasyPrint/issues/79_
cannot load library 'libcairo.so.2': libcairo.so.2: cannot open shared object file: No such file or directory
2015-06-16T05:57:22.383+0000 I NETWORK  [initandlisten] connection accepted from <IP_ADDRESS>:40964 #10 (2 connections now open)
/data/quokka_venv/lib/python2.7/site-packages/flask_script/init.py:153: UserWarning: Options will be ignored.
warnings.warn("Options will be ignored.")

Weasyprint will be dropped in favor od PDFkit


run n170 experiment

After you have have run through the first 'good first issue' and completed installation, the next thing to do is to run one of the experiments.
Note, this only applies to users/developers with access to an EEG device.
As with the installation issue, we would like to gather feedback and any errors encountered from this process.
Please follow through the instructions here. Be sure to select the correct EEG device, choose the visual N170 experiment, and for this test run, just enter '30' (30 seconds) for the duration option.
Please leave comments on this thread with any issues encountered, or suggestions for improvement, from your experience of this process.

Hello everybody. I got it to run (had to use Python 3.7 and psychopy-2021.2.3) and when I get data the CSV file is missing the last column in the firsts rows (please see the attachment, I also include the one from the example with the last column Marker0). @JohnGriffiths mentioned on NTX slack that other people where having this problem too. Thanks

Closing as is stale


minPwdLength > 14 is not supported at this time

minPwdLength > 14 is not supported at this time

OK, so looking at this suggested change, may I conclude that the change I made based on the recommendation from Robert Durff was poorly founded?
@MSRobertD : Would you mind commenting on this case, knowing that you requested the current content?

@illfated , happy to comment.  Sorry if this wasn't clear in my original feedback / post.
Although I'm an external consultant, I work embedded with an MS team that puts Windows through security certifications.  Our own testing showed that this GPO does support values >14.  (And that's a good thing, as we wouldn't pass certification without it, which would have generated a different sort of bug / request to the feature team.)  So my original feedback was simply that: hey, this GPO does seem to support > 14, so can we change the documentation to show that?  That's it.  If this is, in fact, a bug in the GPO itself, then let's have a different conversation.
For an FTE contact to loop in from our team (if needed), please contact Mike Grimm (mike.grimm@microsoft.com).

Hello,
I am also a Microsoft FTE, my alias is jories, and I'm a debug engineer on the Windows debug team. While the Group Policy GUI was updated a couple years ago to allow users to modify the Minimum Password Length in Group Policy and Local Policy to 20, the corresponding changes were never actually made to the SAM (Security Accounts Manager) to actually allow that policy to be successfully applied. In practice, if one attempts to apply a Group Policy where the Minimum Password Length has been configured to > 14 characters, an event from SceCli will be generated in the event log of the machine stating that there was an error applying security policy.
Full support for MinPwdLen > 14 may be revisited again later in the future, but it's not a hot issue for the Feature Team right now, despite many customers asking. It has proven to not be an easy or risk-free change. Changing this restriction is similar to changing MAX_PATH which has been painstaking to change and causes lots of things to break.

#sign-off

@martyav @Dansimp The PR seems ready to be merged. Thank you.


[Bug]: Microsoft.Playwright.CLI installs incorrect version of Chromium

Playwright version
1.13.0 (CLI 1.0.2)
Operating system
Windows
What browsers are you seeing the problem on?
Chromium
Other information
dotnet --version: 6.0.100-preview.6.21355.2
What happened? / Describe the bug
playwright install installs Chromium 888113, but _playwright.Chromium.LaunchAsync() requires  Chromium 901522
Code snippet to reproduce your bug
dotnet tool install --global Microsoft.Playwright.CLI
playwright install

in C# unit test:
_playwright = await Playwright.CreateAsync().ConfigureAwait(false);

  Browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
  { Timeout = 5000, Headless = _headless })

Relevant log output
playwright install
Playwright build of chromium v888113 downloaded to C:\Users\romfi\AppData\Local\ms-playwright\chromium-888113
Playwright build of firefox v1271 downloaded to C:\Users\romfi\AppData\Local\ms-playwright\firefox-1271
Playwright build of webkit v1492 downloaded to C:\Users\romfi\AppData\Local\ms-playwright\webkit-1492
Playwright build of ffmpeg v1005 downloaded to C:\Users\romfi\AppData\Local\ms-playwright\ffmpeg-1005

from unit tests:
Microsoft.Playwright.PlaywrightException
Executable doesn't exist at C:\Users\romfi\AppData\Local\ms-playwright\chromium-901522\chrome-win\chrome.exe
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝

This sounds like it's picking up the "wrong" library version. A couple quick questions, to help me diagnose:

Are you running the command from the project location?
Do you also have the node/npm version installed?
Can you try running dotnet .\bin\Debug\net5.0\Microsoft.Playwright.dll -- install? Does that install the correct version?

Are you running the command from the project location?

No, I was runnig from solution folder, when I run it from project folder it downloaded correct version - chromium-901522

Do you also have the node/npm version installed?

yes, 16.5.0

Can you try running dotnet .\bin\Debug\net5.0\Microsoft.Playwright.dll -- install? Does that install the correct version?

dotnet ./bin/Debug/net6.0/Microsoft.Playwright.dll
dotnet : Cannot use file stream for [C:\Users\romfi\source\repos\HtmlRaportGenerator\HtmlRaportGenerator.Tests\bin\Debug\net6.0\Microsoft.Playwright.deps.json]: No such file or directory
At line:1 char:1
+ dotnet ./bin/Debug/net6.0/Microsoft.Playwright.dll
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Cannot use file...le or directory:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\Users\romfi\source\repos\HtmlRaportGenerator\HtmlRaportGenerator.Tests\bin\Debug\net6.0\'.
Failed to run as a self-contained app.
  - The application was run as a self-contained app because 'C:\Users\romfi\source\repos\HtmlRaportGenerator\HtmlRaportGenerator.Tests\bin\Debug\net6.0\Microsoft.Playwright.runtimeconfig.json' was not found.
  - If this should be a framework-dependent app, add the 'C:\Users\romfi\source\repos\HtmlRaportGenerator\HtmlRaportGenerator.Tests\bin\Debug\net6.0\Microsoft.Playwright.runtimeconfig.json' file and specify th
e appropriate framework.

So playwright install on Windows must be run from project folder, but on linux it can also be run form solution? I have a github runner that installs playwright and it works ok

What I think is happening is, playwright install is picking up an older version of the DLL somewhere in your solution folder structure. Have you had a previous version of the NuGet installed there, by any chance? Is there a reference by another project to an older version?
That will also explain why it's working (as expected) on CI - there's likely no older versions there.

In my sln structure there are just 2 projects, blazor wasm app and playwright test app, the older version was installed in the test app previously, but the new (1.13.0) was installed before running playwright install

When you updated the reference, did you clean/rebuilt the project before doing install?

I thought so, but mb I forgot. Now I manually deleted bin and obj folders, uninstalled CLI tool and removed ms-playwright folder, rebuilt and reinstalled CLI and playwright install installs correct version 901522. Thanks for Your help, I'm closing this issue.