Write authoring documentation

To be added to CW docs.

[ ] How to use module
[ ] How to write tests with it
[ ] Style guide

I'm starting to much like kebab-case as a naming convention. No Shift key necessary anywhere, complete freedom in initial capitalisation, doesn't interfere with anything that I can think of. Could recommend not ending a name in -, but absolutely no need to enforce this. This would work well for me as a standard library style.
I have much fewer and less developed ideas on indentation and parenthesisation.
Style Guide could suggest n-1 as an argument name for use with Scott Numerals.
Lots of things can actually be ad-hoc though, and our syntax ( by design ) allows for many different conventions in naming, indentation and line continuation. "Be consistent" would be my most important advice.
For something like a Church succ, I'd like to recommend succ = \ n . \ f x . f (n f x) to show the conceptual difference between succ arguments and num constructor arguments.

Having the Style guide recommend certain, limited, assigned names might help preventing ( some ) clashes.
All single capitals plus KI reserved for standard combinators ( as documented here ).
I'd like zero, succ, pred, add, sub, mul, div, mod, pow, gcd, lcm, LT, EQ, GT, compare, lt, le, eq, ge, gt, True, False, not, and, or, xor, Pair, fst, snd to not have surprising implementations. Specifically plus, times might be usefully repurposed for other operations - they don't need to work on numbers only.
Constructors ( except for numbers, and possibly lists ) preferably capitalised, mainly to prevent clashes with JS true, false. I irrationally prefer zero, one, cons, nil &c uncapitalised.
Using Haskell standard list manipulation function names ( head, tail, .. plus adaptations like append for ++ ) are probably also not the best idea to alternatively define. ( It's a shame null can never be exported. See here for a comprehensive list of reserved words and literals. Note that, amongst others, undefined, module,require can be overwritten - but that doesn't necessarily make it a good idea to do so. )

Maybe for the Style Guide, maybe for the formal specs: comments should have whitespace following the #.
# immediately followed by non-whitespace is reserved for future expansion ( ie, compiler directives ).

Probably recommend setting verbosity Concise for Submit tests. Solvers can't change that; Calm is very sparse.

How to use module -> Verbosity levels:
Setting verbosity to Verbose shows references in use from the environment while compiling. This evaluates those references, which normally might never happen if a definition is never used. So this might (!) also create eval errors that would not happen without debugging.
I think it's a small price to pay, but it's something to document.

Yes I think an "introductory" document which defines syntax and gives a very brief introduction to LC (with links to wiki/etc) would be good. Then some beginner katas, and some of the existing multilingual LC katas can provide a link to the document in the initial solution.

For the Testing Guide: recommend ending any code in preloaded.lc with a dummy definition like _ = () to prevent cheating by starting solution.lc as a line continuation.
Code has to be compiled as a single block of text if user solutions are to be able to access Preloaded definitions, so preventing solution.lc from starting with a continuation is not possible - nothing you can put in preloaded.lc will uncontinue the last definition without causing a syntax error.
Note that Preloaded can be compiled separately if only the test suite needs access to definitions in it - but this offers no advantage over simply defining them in JS.


The focus indicator does not display properly for some of the headings in the navigation menu

Under certain circumstances* the headings in the sidebar (which are collapsible) are not wrapped within <span> causing the heading not to be highlighted when focused/tabbing through. Failing the WCAG 2.1. success criterion 2.4.7 (focus visible).
See the example here.
*I think it's the combination of the toc_level config and a use of nested pages
This fails WCAG 2.1 success criterion 2.4.7 (focus visible).

Can reproduce this on the tech docs template website too: https://tdt-documentation.london.cloudapps.digital/#technical-documentation-template


Decide whether to_numpy/from_numpy/lincomb should work with column vectors

For reference our previous discussion on this matter in pymor is in https://github.com/pymor/pymor/issues/545.
The performance benchmark does show, that if you heavily rely on BLAS 1 operations, then having vectors as contiguous data is quite relevant. The issue can be avoided by using as_fortran_array in from_numpy. Of course, that will cause lots of unnecessary copies when the user creates the array themselves. We could hack pyMOR to check if using as_fortran_array will cause copies in unwanted places.
Thoughts @pmli, @drittelhacker, @artpelling?

I would vote for column vectors (if there is only one vector, to_numpy could accept both a 1D vector and a 2D column vector).


chore(Blog): create “2022-06-28-jeudi-soir”

Automatically generated by Netlify CMS

👷 Deploy Preview for lucileetguillaume processing.

Name
Link

🔨 Latest commit
ac60af24f9a2a7a44ecc4d65e599ffaa05133c3d

🔍 Latest deploy log
https://app.netlify.com/sites/lucileetguillaume/deploys/62badab0dd71fd000a6f2211


Show "files skipped due to complete coverage" warning in HTML report

I ran into this issue where one of the modules I was testing was disappearing from my HTML coverage report when I wrote tests for it. I couldn't figure this out until I ran the plain text summary which had the following warning: "43 files skipped due to complete coverage".
I didn't know this feature existed or that it was activated by default. It would help if that warning appeared in all reporting formats.

This is implemented in a99b27f7.
BTW: this setting is not the default.  You probably have [report] skip_covered = true in your .coveragerc file.

Thanks for the fix. I checked our repository and we don't have a .coveragerc file anywhere. We are running Coverage.py, version 5.5 with C extension.

Hmm, how about a setup.cfg or tox.ini or pyproject.toml?

Ah, setup.cfg was the culprit. Sorry for the noise!

This is now released as part of coverage 6.0.


Should we use syntax, filetype or filename?

If we do filename we'll need a regex and mapping for file extensions to different filetypes
I think filetype is good

I think we'll need the filename if we are downloading the content as a file, or uploading a file

we could always just use a filename like notebin-download.js


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-yml-archive-non-versioning-archive/pull/232 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-yml-archive-non-versioning-archive.git

Docs Build status updates of commit de0042a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

docs/TOC.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-3.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-3.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Derivative reference volumes for 4D series

Your idea
In fMRIPrep, we generate _boldref.nii.gz files from some combination of the source bold file, an associated sbref (if available) and within-space transforms (such as SDC warp). This is used for performing SDC, coregistration, visualization, and as a resampling target for a full BOLD series.
I was initially going to put this into #519 as _boldref and _cbvref, but is it more general? Does this make sense to put into Imaging data types?
Would something like the following ever be generated?

_dwiref (cc @oesteban, @arokem, @Lestropie, @bids-standard/derivatives-mri-dwi)
_aslref (cc @patsycle, @HenkMutsaerts)
_petref (cc @mnoergaard @melanieganz, @bids-standard/derivatives-pet)

ASLPrep generates aslref files.

In the dMRI world, I think such a "dwi reference" is typically a $b = 0$ (b0 or bzero). So _dwiref would force people to change their standard. On the other hand, I see great usefulness in standardizing _<modality_suffix>ref as the suffix across modalities (and I would bet a similar concept of "individual alignment reference" may exist even for EEG, MEG or fNIRS)

I think in cases where there is a definite physical meaning, it makes sense to have a specialized suffix. We could describe reference volumes and then simply have a table that maps the raw suffix onto the derived reference suffix:

Data type
Modality
Derived reference

func
bold
boldref

func
cbv
cbvref

dwi
dwi
b0ref

perf
asl
aslref

...

DWI stands out a little, but the concepts are still linked in the spec and will make sense to each modality's practitioners.

Actually, there are occasions where the reference is calculated with
nonzero (but low) $b$ values. I'm not sure there's a big difference between
_b0ref and _dwiref while the latter is always theoretically correct and
more consistent with the other.
WDYT @yasseraleman ?
On Wed, Jun 28, 2023, 20:23 Chris Markiewicz @.***>
wrote:

I think in cases where there is a definite physical meaning, it makes
sense to have a specialized suffix. We could describe reference volumes and
then simply have a table that maps the raw suffix onto the derived
reference suffix:
Data type Modality Derived reference
func bold boldref
func cbv cbvref
dwi dwi b0ref
perf asl aslref
...
DWI stands out a little, but the concepts are still linked in the spec and
will make sense to each modality's practitioners.
—
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/1532#issuecomment-1611883475,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAESDRRYLSJTOCXK34NIKN3XNRZDHANCNFSM6AAAAAAZXNJ5YQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

I wonder if it would make sense someday to differentiate between b0 and b=0 in bids somehow. Since there are already metadata fields that include b0 in their name and are referring to something totally different. fwiw qsiprep has been writing out _dwiref files, which are the average b=0 images

If qsiprep already writes _dwiref, I guess that settles it right?
On Wed, Jun 28, 2023, 20:55 Matt Cieslak @.***> wrote:

I wonder if it would make sense someday to differentiate between b0 and
b=0 in bids somehow. Since there are already metadata fields that include
b0 in their name and are referring to something totally different. fwiw
qsiprep has been writing out _dwiref files, which are the average b=0 images
—
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/1532#issuecomment-1611919421,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAESDRU7RJ7XBBO23IVRO5TXNR4YLANCNFSM6AAAAAAZXNJ5YQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Yeah, sounds reasonable. I'll write up a PR, as it should be pretty quick.

Just a note that this was not an uncontroversial proposal. Additional input on #1533 would be appreciated.

https://github.com/bids-standard/bids-specification/issues/1532#issue-1779365385 - definitely! We will also have a _petref to indicate the file to be used with registration between PET and anatomical. However, there may be multiple refs present, as one might be using one ref for motion correction, and then another for registration. For example, it is not uncommon for PET preprocessing that a single frame is chosen for motion correction, and then once the data is motion corrected, then the motion-corrected data is either averaged or summed across frames to create a new reference that is then used for co-registration. So we just need to be absolutely clear on the meaning of _petref.

Definitely a good point that the motion correction and coregistration references need not be the same. In BOLD, we generally want to do motion correction with an uncorrected reference, but then use a distortion-corrected reference for coregistration. Assuming we're okay with a common suffix for these two cases, it's just a matter of a desc- field.


home-assistant-custom-components.solis-sensor: 3.7.1 -> 3.7.2

Things done
Bump solis-sensor to latest upstream version to fix a breakage with HA 2024.12

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 362440

x86_64-linux

:white_check_mark: 1 package built:

home-assistant-custom-components.solis-sensor

This PR touches 1 files
File pkgs/servers/home-assistant/custom-components/solis-sensor/package.nix is owned by @mweinelt
Requesting reviews from: {
"reviewers": [
"mweinelt"
]
}
gh: Validation Failed (HTTP 422)
Failed to request reviews: {"message":"Validation Failed","errors":["Could not add requested reviewers to pull request."],"documentation_url":"https://docs.github.com/rest/pulls/review-requests#request-reviewers-for-a-pull-request","status":"422"}


Hide "replace with templates" behind feature flag

Related to https://github.com/camunda/camunda-bpmn-js/issues/211#issuecomment-1337263134

Slightly simplified the implementation: https://github.com/camunda/camunda-bpmn-js/pull/214/commits/0916c9e8d114f57e28bba1c38b5c333f0b801453.


Add Travis CI builds

I believe a trusty image would have to be used since precise does not have sdl headers in it's repos.

Added travis.yml in 91474c8f35437a4c2ac46cc9233240b3ea7b660b.


OCI distribution - referrer metadata proposal

Add a proposal for tracking referrer metadata for each manifest and using this ability to store and retrieve signatures. The idea is illustrated using a prototype implementation.

Please see https://github.com/notaryproject/nv2/pull/14 instead.


Support for other response types then code & token.

The current codebase already provides many hooks for customizing the behaviour like enhancing tokens, allow custom grant types, pre-approval of authorization requests and so on. What it currently does not support is hooking in custom behaviour for handling other response-types then code & token in the Authorization endpoint.
The oauth 2.0 authorization framework specification has a section about new authorization endpoint response types.
Allowing for custom response types handlers would enable users of the framework to easily implement features like the id_token & none response types defined in the openid oauth2 specification without having to change the actual spring oauth2 framework.
All the needed hooks are already there in the AuthorizationEndpoint and a minimalistic implementation would mean introducing a CustomResponseTypesHandler that is capable of handling custom response types other then code & token.
I would like to write a pull request for this, but before putting in the effort I was wondering if this is something the core committers would consider viable?

After googling - probably should have done that first - this issue is linked to #619 and #220

The https://travis-ci.org/spring-projects/spring-security-oauth/builds/91321307 build fails, but it seems totally unrelated to my code. Also locally when running mvn clean test -P bootstrap everything is green. False negative?

Duplicates #619


Feat/use rich printer for init

What
Refactor contract init command to usesoroban-cli/src/print.rs.
Closes #1553
Why
To be more consistent without our output, and to reduce duplication.
Known limitations
n/a

I have some nits, but in general I think that we should have a the root command handle the print.errorln of the errors. The map_err's with print.errorln here are providing context to the error and should be an error case so that the caller would have that context.

@willemneal that's a good point - instead of printing the error messages and then swallowing the errors, we should return the error for those cases. I can work on that change in a separate PR.


class_embed

self.class_embed = nn.Linear(hidden_dim, num_classes)
self.bbox_embed = MLP(hidden_dim, hidden_dim, 4, 3)
self.num_feature_levels = num_feature_levels
self.use_dab = use_dab
self.num_patterns = num_patterns
self.random_refpoints_xy = random_refpoints_xy
self.label_enc = nn.Embedding(num_classes + 1, hidden_dim - 1)
请问为什么这里class_embed使用的是91类，而label_enc使用的是92类？标准的DETR里class_embed里似乎是
nn.Linear(hidden_dim, num_classes+1)
Why class_embed  use class 91 here and label_enc use class 92?

Hey, because label_enc is also used to encode empty object (content query of the matching part), we add another category here.
tgt = label_enc(torch.tensor(num_classes).cuda()).repeat(num_queries * num_patterns, 1).
You can refer to the prepare_for_dn function indn_components.py for more details.

@FengLi-ust  Thanks for the answer~
我之前参考的是deformable DAB-DETR 的代码，并没有这个操作

dn_dab_deformable_detr/dn_components.py

indicator0 = torch.zeros([num_queries * num_patterns, 1]).cuda()
tgt = torch.cat([tgt_weight, indicator0], dim=1) + label_enc.weight[0][0]*torch.tensor(0).cuda()
标识符直接与tgt拼接
Identifiers are directly spliced with tgt

DN_DAB_DETR/dn_components.py
tgt = label_enc(torch.tensor(num_classes).cuda()).repeat(num_queries * num_patterns, 1)
tgt = torch.cat([tgt, indicator0], dim=1)

所有tgt被encode成背景类
All tgt are encoded as background classes
请问为什么两者对于tgt有两种截然不同的操作
Why are there two different operations on tgt?

Because dab_detr uses all zero vectors as the initial content query embedding, therefore we directly embed them into the background category to distinguish from the dn part.
However, dab_deformable_detr uses learnable content query embedding, which contains information learned from the dataset. So we cat the indicator with the tgt to preserve the original content information.
The above two ways are just used to distinguish the dn part and original matching part, which do not have a significant impact on the performance. Thank you.

thank you for you anwser~ @FengLi-ust


Elasticsearch deep paging

Description
When building remote system integrations, it's often necessary to compare the entire product data. What you do, you iterate over all articles of Shopware via API, compare them locally and then issue create/update/delete accordingly.
If you have many articles (> 10k), you're gonna run into an issue with elasticsearch. The default limit for the result window is 10000. It's not possible to use regular pagination deeper than that (for good reasons).
In elasticsearch there are 2 ways around this:

Search After (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-search-after)
Scroll (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-scroll)

It's really necessary that both of this features are implemented in the elasticsearch integration to have the possibility to build efficient integrations with a large number of products.

https://issues.shopware.com/issues/NEXT-7819

hey @hhoechtl
thanks for raising this up. as you already opened a ticket on our issue tracker, I will close this here, to avoid duplication


Behavior test for SolrSearch, plus placeholder.

This includes a phpspec test describing the basic behavior of SolrSearch.

:+1:
:zap:

:100: / :100:


Create paper wallet

[x] change buttons on make backup https://github.com/swarmcity/sc-boardwalk-production/wiki/Make-backup to be able to get a paper wallet.

[x] Create layout of pdf This is the link to an example of MEW (there is no money on it, so don't even try)
mew.pdf

@bkawk changed the txt already.
I've created a html-page with all the styling inline. So I just want to look with @n3xco or @bkawk at it, to see if everything it is correct, and where to put it.


Embedding_bag_out does not have boundary check and causes IPEX UT fail

🐛 Describe the bug
Background:  IPEX added boundary check assertion in gpu kernel https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-gpu/pull/4555 and a UT(ipex/tests/gpu/example/test_embedding_bag.py::TestTorchMethod::test_embeddingbag_out_of_bounds) to test the debug log being printed when input is out of boundary.
Ops in torch-xpu-ops do not have this check and hence fail in this UT.
I check that CUDA does not have this boundary check, this feature is IPEX only.  In IPEX2.5, we override this Ops with IPEX implementation to make this UT pass.
Please decide whether we need to add this boundary check in torch-xpu-ops.
Versions
ipex/tests/gpu/example/test_embedding_bag.py::TestTorchMethod::test_embeddingbag_out_of_bounds
Assertion vec_idx < num_row failed

pr merged, suggest to close this issue.


NPM package 0.5.3 doesn't match master, but 0.5.1 release

After installing the package using npm, version number is 0.5.3 but code matches 0.5.1 release

I discovered this because when it's FSA compliant and an error occurs, it doesn't return the rejected promise, but the dispatched action, so when used with async/await and try/catch, catch is never executed.

+1


CoreML / ONNX Export

Hi,
Thanks a lot for the great work since EfficientSAM.
Any plan for releasing the trained weights and CoreML / ONNX Export?
Cheers.

Hi @SimonZeng7108,
Thanks for your interest in our EfficientSAM and efficient track anything.
Our trained checkpoints are going review for public release. When it is ready, we will provide examples for model export.

It would be nice to have also a torch.export example.

Hi @SimonZeng7108 @bhack,
We just released a family of 🤗Efficient Track Anything Checkpoints. Feel free to play with it.

@SimonZeng7108 @bhack,
We will share an example to run on MacBook Metal Performance Shaders (MPS) backend.

@yformer An onnx version would be great, the checkpoints are small enough to build something that runs in the browser

It would be nice to have a torch.export script so we can have both onnx and aoti.

I'm looking forward to an MPS example as well


Update package.json

Description

A clear and concise summary of the change and which issue (if any) it fixes. Should also include relevant motivation and context.
Resolved or fixed issue:
Affirmation

[ ] My code follows the CONTRIBUTING.md guidelines

⛔ Snyk has found 1 issues.

Icon
Severity
Issues

Critical
0

High
1

Medium
0

Low
0

⛔ security/snyk check is completed. Snyk found 1 issues. (View Details)
✅ license/snyk check is completed. No issues were found. (View Details)


stdenv: shorten evaluation errors when in Hydra

Hydra's page showing evaluation errors is about a mile long, showing
buckets of user-friendly errors, like this:
in job ‘seyren.aarch64-linux’:
Package ‘oraclejre-8u191’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix:71 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

in job ‘jetbrains.webstorm.x86_64-linux’:
Package ‘webstorm-2018.3.1’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/applications/editors/jetbrains/default.nix:230 has an unfree license (‘unfree’), refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnfree = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnfree = true; }
to ~/.config/nixpkgs/config.nix.

This makes it extremely hard to find actual issues in the output. This
patch set makes the output much more condensed in Hydra:
Failed to evaluate nifticlib-2.0.0: «unsupported»: is not supported on ‘x86_64-apple-darwin’
Failed to evaluate dmd-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
Failed to evaluate dmdBuild-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
Failed to evaluate ldc-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
Failed to evaluate ldcBuild-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
Failed to evaluate ldc-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
Failed to evaluate ldcBuild-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

I got a :tada: from @edolstra so I think I'll merge once this passes ofborg.

this branch:
real	1m46.976s
user	2m54.181s
sys	0m3.512s

master:
real	1m48.878s
user	2m56.880s
sys	0m3.651s

:man_shrugging:


[Feature] : service page can't work Dark & Light mode

Hello @khushi-joshi-05 ,

I will finish it..

closes #1202
Before

After

please merge it..

@khushi-joshi-05  please merge it..


Events logging

🚀 Feature
It can be reasonably hard for newcomers and even experienced users to understand which events are fired when. One potential quick solution would be to give create a napari logger, then use logging.debug() calls in in the Event class each time an event is called. I'm not sure whether this has been proposed and/or explored before, but I couldn't quickly find any evidence of it looking at the EventEmitter source.

Have you seen this plugin by @dalthviz
https://github.com/dalthviz/napari-ui-tracer
I think the event logging option addresses this?
Maybe we just need to document it?
I personally find it super helpful when trying to figure out where in the code to look for GUI issues.

I think that better granulation is required. Without this. There could be to many events to keep log readable.
Maybe should be a single logger per class. Then we could use logging.getLogger("class name") to control it for the interesting arena. As EventEmiter store source, then it could be easy to introduce.
Logggin supports hierarchy, so using full class path as a key allows to have napari logger, and napari.labels.Image logger so could control globally, or enable logging only for a given class.

It might be helpful to use something like structlog for this - it allows binding data/context to the logger.
There could also be a plugin to parse the events into a readable/interactive tree or something (similar to @dalthviz's plugin mentioned above).

Hi, just in case, the current logic available related with events logging/debugging is here:
https://github.com/napari/napari/blob/616a5af8f4bb6bc6fd96a6e5d1e03c191cb1b5b1/napari/utils/events/debugging.py
https://github.com/napari/napari/blob/616a5af8f4bb6bc6fd96a6e5d1e03c191cb1b5b1/napari/utils/events/event.py#L1240-L1254


Options appear empty for some references when editing/creating a new entry

I've found that when creating or editing a resource with a has_many association, sometimes there are no options in the list. Here are some cases:
The client and importer fields both refer to a Client record. This is what it looks like in the Ot model:

class Ot < ApplicationRecord
  belongs_to :workflow
  belongs_to :client
  belongs_to :importer, class_name: 'Client', foreign_key: 'importer_id'

  has_one :category, through: :workflow
  has_one :workflow_type, through: :workflow
  has_one :workflow_subtype, through: :workflow

  validates :name, presence: true
end

But when editing this Ot entry, this is what I get:

The Client dropdown works fine, but the Importer dropdown has no options to choose from. Ideally, it should have the same options that the Client dropdown has, since the class_name option is set to Client in the association.
Only after typing something into the input, some options appear. This can sometimes be a bit confusing, because only one option shows up unless I erase the content that the field already has, as I show in this short video:
https://user-images.githubusercontent.com/65265704/122690180-62895d80-d1fe-11eb-812b-0581f4a6d089.mp4
I'll be ready to provide more information if needed! Thanks!

Thanks @dantevvp, those issues should be fixed in the newest release which includes custom form enhancements:
https://rubygems.org/gems/motor-admin/versions/0.1.61
Would really appreciate if you could check if everything works as expected in the newest version and close the issue if it does.

Works beautifully! Thanks!


How to properly use acquire_packet_id together with C++ 20 coroutines?

It's not an issue per se, it's rather a question because I kinda don't fully understand how to properly use the acquire_packet_id method. It's probably due to my misunderstanding of how to use it.
Recently I've stumbled upon the error:
td::optional<typename async_mqtt::packet_id_type<PacketIdBytes>::type> async_mqtt::basic_endpoint<Role, PacketIdBytes, NextLayer>::acquire_unique_packet_id() [with async_mqtt::role Role = async_mqtt::role::client; long unsigned int PacketIdBytes = 2; NextLayer = boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0> >; typename async_mqtt::packet_id_type<PacketIdBytes>::type = short unsigned int]: Assertion strand().running_in_this_thread()' failed.`
It took me to take a close look at the documentation and the code of the library.
There is a part saying that: "f you call those APIs out of strand, then assertion failed.". This statement is followed by a table where one of the entries is: "acquire_unique_packet_id".
Now, let me briefly show (using pseudocode) how I use your client in my code, the code is simple and short so it should be quite easy to understand.
// ClientWrapper - is a wrapper that is using async_mqtt::endpoint<async_mqtt::role::client, async_mqtt::protocol::mqtt>; under the hood
// rethrow - is a function that just rethrows an exception
class ClientWrapper
{
boost::asio::awaitable<void> async_send(....)
{
    auto packet_id = ep_.acquire_packet_id();
    [...]
}

boost::asio::awaitable<...> async_receive()
{
  [...]
}

private:
async_mqtt::endpoint<async_mqtt::role::client, async_mqtt::protocol::mqtt> ep_;
};

auto ioc = io_context{};
auto client1 =  ClientWrapper{...};
auto client2 = ClientWrapper{...};

boost::asio::co_spawn(ioc, [&client1, &client2]() -> boost::asio::awaitable<void>()
{
  co_await client1.async_send(...);
  co_await client2.async_send(...);

  co_await client1.async_receive(...);
}, rethrow);

Notice that I'm using SYNC version of acquire_packet_id.
So, the error I showed before, and the documentation that says that there is an async version acquire_packet_id took me to the other solution where I replaced SYNC call to acquire_packet_id with async version with boost::use_awaitable completition hander.
So my questions are:

should I always use the async version of acquire_packet_id in case of using coroutines?
is my fix correct?
when should I use SYNC version of acquire_packet_id?
why is the return value of async call to acquire_packet_id optional? When can it be returned as nullopt?

Good question.
should I always use the async version of acquire_packet_id in case of using coroutines?
I recommend always using async version of acquire_packet_id.
It is always safe.
is my fix correct?
Yes.
when should I use SYNC version of acquire_packet_id?
You don't need to use SYNC version. But you could call SYNC version in the same endpoint's strand.
For example,
In the CompletionToken handlng code of the same endpoint.
Receiving CONNACK then send PUBLISH to the same endpoint.
Receiving PUBLISH then send PUBLISH to the same endpoint (echo back).
See
https://github.com/redboltz/async_mqtt/blob/d876fc1c0bd1fd4f7f4d7744241d14efa05c31b2/example/ep_cpp20coro_mqtt_client.cpp#L61-L97
The line 83 is runnning in the strand of the same endpoint as CONNACK is received, and sending SUBSCRIBE packet with SYNC acquired_unique_packet_id().
You cannot call ASYNC version out of the strand.
For example, receive PUBLISH packet from endpoint ep1, then send (forward) the PUBLISH packet to ep2. In this case, you shouldn't call SYNC version of acquire_unique_packet_id() of ep2 because the code is runnning on ep1's strand.
If you use C++20 coroutine, you wouldn't have much motivation to use SYNC version because C++ coroutine doesn't introduce annoying nested indents.
If you use callback functions instead of use_awaitable, then ASYNC version introduces one callback nest. SYNC version can avoid that kind of nest in the allowed case I mentioned above.
If I call ASYNC version in the same strand, is there any performance penalty compare to SYNC version? (I added the question)
You can ignore the penalty.
Here is ASYNC version implementation:
https://github.com/redboltz/async_mqtt/blob/d876fc1c0bd1fd4f7f4d7744241d14efa05c31b2/include/async_mqtt/endpoint.hpp#L248-L266
https://github.com/redboltz/async_mqtt/blob/d876fc1c0bd1fd4f7f4d7744241d14efa05c31b2/include/async_mqtt/endpoint.hpp#L755-L778
Simply calling boost::asio::dispatch to run on the strand, and then call ep.pid_man_.acquire_unique_id().
Here is boost::asio::dispatch document:
https://www.boost.org/doc/html/boost_asio/reference/dispatch/overload2.html
If boost::asio::dispatch is called from the same strand of the first parameter, then calling the second parameter in the dispatch funtion immediately.
The function call is usually inlined when you enable optimization like -O2.
So the cost is only checking the Executor ex is running on the strand or not. It is very small.
why is the return value of async call to acquire_packet_id optional? When can it be returned as nullopt?
MQTT spec allowed 1-65535(0xffff) packet_ids. If you use all of them and then calling acuire_unique_packet_id(), then it returns nullopt.
async_mqtt allocated unique packet_id efficiently. I implement it the previous project mqtt_cpp and port it to async_mqtt and number-allocator for JS https://www.npmjs.com/package/number-allocator . JS version is used by MQTT.js.
JS version has the better document so you can refer it to understand the performance and algorithm.
The important point is packet_id is managed by endpoint. It doesn't generated simple increment algorithm. It can avoid conflict. It is very important.
When you acquire or register packet_id, then the packet_id becomes using status. Then you pass to the packet_id to the some packet, and call send(). At that time, the management responsibility of the packet_id is moved to the endpoint.
The packet_id would reusable when the following events happned on the endpoint.

PUBACK is received when the packet_id is used for sending PUBLISH QoS1.
PUBREC with error reason_code is received when the packet_id is used for sending PUBLISH QoS2.
PUBCOMP is received when the packet_id is used for sending PUBLISH QoS2.
SUBACK is received when the packet_id is used for sending SUBSCRIBE.
UNSUBACK is received when the packet_id is used for sending UNSUBSCRIBE.
MQTT session is cleared. (e.g. recieve CONNACK with session_present false)

@redboltz Thank you very much for the in-depth explanation!

Recently, the strand treatment is updated.
See https://github.com/redboltz/async_mqtt/blob/main/test/unit/ut_cpp20coro_exec.cpp
It is the same behavior as Boost.Asio APIs. So async_mqtt respect it.
I recommend that BOOST_AUTO_TEST_CASE(same) approach.


Create ROS2 package

Use Case
There needs to be a place to store custom configurations and code.
Acceptance Criteria
GIVEN a developer
WHEN we want to contribute to c3pzero
THEN there will be a existing package to make contributions to.
Notes
Use https://github.com/ros-planning/navigation2_tutorials/tree/master/sam_bot_description as a starting point and rename everything.
Possibly improvements would be to have a meta package and a bringup package but we can make those determinations when we come to them.

#7 should develop in this monorepo


[Question] Any hints on best way to support swagger response/model parsers?

Basically i want to be able to support https://github.com/ruby-grape/grape-swagger#response-documentation-
It looks like you can provide custom parsers, but I haven't really dug much more into this
https://github.com/ruby-grape/grape-swagger#model_parsers
Just wondering if you have tried adding response support for swagger + grape + fast_jsonapi, or if there is an approach that isn't going to end up being a major headache?

Hi @rromanchuk, unfortunately I haven't found any simple way to provide such support.
The Best way that I have figured is to code a custom model parser that would take a serializer (including Jsonapi::ObjectSerializer), get its attributes and returns a hash containing the schema with its attributes.
But the big problem with this approach is : Impossible to figure the attribute type directly from the serializer. If your serializer serializes a database model, then we have to manage the dependencies with the different ORMs... like you said, that's a headache.
I'll keep you posted if I manage to find a better solution, meanwhile, feel free to share any ideas you might have on that matter

@rromanchuk I think you might be interested by this branch I've been working on. There is also a open pull request but it needs some testing and some reviews before merging.
When you have time, would you let me know if it is any close to answer your question?
# Gemfile
gem 'grape_fast_jsonapi', git: 'git://github.com/EmCousin/grape_fast_jsonapi.git', branch: 'feature/swagger_model_parser'

@EmCousin that's what i figured.  i know it kind of defeats the entire purpose, but i'm almost tempted to make non-jsonapi decorator just for documentation purposes, although good luck to me keeping that maintained.
In context of my client code, the jsonapi response is actually obfuscated away and is flattened to be used in mapping the local object graph. I'm basically just trying to avoid (or non backend developers) having to look at serializer files to look at which properties/datatypes are returned. I mean i might be even overthinking it, i think swagger has an example property where i could even generate fixtures with a simple rake task and render some examples via fixtures. Right now i basically just execute the API call and inspect the live results/dump responses from client console/read the serializer.

sweet, looking now!

closing, tracked in #5


not navigating to other user's page from one users page even after clicking.

current behaviour :
In home page, when searching for a userand clicked on that user its opening that users page for once. If i search for other user on the previous users page and click on it its not navigating to the 2nd users page. it is remaining on same page.But the url is changing when clicked on other user.
expected :
While we are on a users page, if we search for other user in search field on top, it should navigate to that users page.

2.

3.

4.

I have checked and confirm that this is a bug.

can you assign me this.

if its not fixed yet assign me


Your FFProbe version is too old and does not support -help option, please upgrade

Q
A

Bug?
no

New Feature?
no

Version Used
latest

FFmpeg Version
N-84014-g9e6b269

OS
Windows 10

PHP
7.2

Actual Behavior
First I have tried to use Create Singleton Method without inserting array then I got this error :
Your FFProbe version is too old and does not support -help option, please upgrade.
then I set the directories which are terms of ffmpeg and ffprobe , now I got the same error
Expected Behavior
I want to know how to fix it.

Could you post the complete code? Is FFProbe installed on the help system?

That happened to me just today, it was working yesterday,
I tried to use shell_exec & exec and noticed they return null.
I don't know if PHP-FFMpeg uses them or other way.
you could try to use
var_dump(shell_exec("ffprobe -help"));

perhaps a bug of php or the operating system? I'd like you to file a php bug then, because this is the wrong address for this behavior.. :/
We're using proc_open under the hood: https://github.com/symfony/process/blob/768debc5996f599c4372b322d9061dba2a4bf505/Process.php#L290
I'm sorry.

And then I replaced 'Process.php ' , but the error had no change What do you mean by that?

My PHP is crashing :sob:
`$ffmpeg = FFMpeg\FFMpeg::create();
                    $video = $ffmpeg->open( $destination );

                    $frame = $video->frame(FFMpeg\Coordinate\TimeCode::fromSeconds(1));
                    $frame->save($thumb_destination);`

it crash every time when code hit ffmpeg probe :/

I had the same issues on windows server 2016. The c:/windows/temp folder needs the proper rights to write stuff. In my case the IISUSR.

Closing due to inactivity.


Adjust documentation for kore-syntax and kore-JSON to multi-ary and/or

To follow up on the changes to \and and \or syntax in textual kore and kore-JSON, we should

update the kore syntax documentation (which currently features a multi-or within \left-assoc or \right-assoc)
adapt the json schema file to reflect the field changes for and and or. (earlier PR)

Originally posted by @jberthold in https://github.com/runtimeverification/hs-backend-booster/pull/337#pullrequestreview-1683537212

#3681 addresses part of this (not the json schema)


python310Packages.pytapo: init at 2.9.2

Description of changes
Python library for communication with Tapo cameras from TP-Link. This is e.g. required when using the custom "tapo" integration from HACS in Home Assistant.
Tests require access to an actual camera, therefore I disabled them.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Just added this package into my HomeAssistant installation and successfully integrated a Tapo camera and can communicate and control it ✅


exceptions in biggraphite.plugins.graphite._merge_cached_points

Hello,
i running a render cluster with biggraphite (version: 0.8.11) and i get sometimes exceptions in biggraphite.plugins.graphite._merge_cached_points.

First exception is a IndexError at line 104.

Second exception is that type 'float' is not iterable at line 98.

Currently i dont have the original stacktraces anymore because i running biggraphite with this patch:
diff --git a/biggraphite/plugins/graphite.py b/biggraphite/plugins/graphite.py
index 64fde85..0f86843 100755
--- a/biggraphite/plugins/graphite.py
+++ b/biggraphite/plugins/graphite.py
@@ -96,12 +96,16 @@ class Reader(BaseReader):
         if not cached_points:
             return points

+        if isinstance(cached_points, dict):
+            cached_points = cached_points.items()
+
         for (timestamp, value) in cached_points:
             step = int(timestamp - (timestamp % stage.precision)) / stage.precision

             index = step - start_step
-            if index < 0 or index > len(points):
+            if index < 0 or index >= len(points):
                 continue
+
             points[index] = value

         return points

cached_points is sometimes a dict instead of a list
points index should never be accessed when index is equal to len(points)

To be honest we never really used CarbonLink. The new version of Graphite will have merge_with_carbonlink that will do all of that for us (https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/readers/utils.py#L102).
I'd be happy  to merge your patch while waiting for a new release of graphite-web :)

Looking forward than for the next release :)

Should be fixed in master


Does this work for multiple websites?

Hi,
I am unclear on whether or not this can be setup to work for multiple websites with multiple domains.  I think it does, but the comments in the configuration file leave me a bit unsure.
For example I have three different independent websites setup in my WF account, and each site has multiple domain entries such as www.example.com, example.com, and blog.example.com.  So Three websites, three domains each, for a total of nine domains across three websites.  Do I just put all nine of those in the domains variable, and then list each website in the public variable? Or would I need to setup multiple configuration files one for each website?

sigh Ok I really should have finished going through the whole config file before opening this issue.  Now I feel like a dolt.
If I understand it, I basically just create one [[certificate]] entry for each website right?

@jwvanderbeck
Not the repo maintainer, but yah, I think you're correct in your second comment there:
[[certificate]]
domains = [ "foo.com", "www.foo.com", "blog.foo.com"]

public = "~/webapps/path_of_first_website_in_webfaction"
name = "first_website_cert_name_you_choose"

Then another one for:
[[certificate]]
domains = [ "bar.com", "www.bar.com", "blog.bar.com"]

public = "~/webapps/path_of_second_website_in_webfaction"
name = "second_website_cert_name_you_choose"

You have flexibility to do it in a number of ways. Each [[certificate]] section represents one certificate with one or more domains (a SAN certificate).
One common way to organize things is to issue one certificate per website. In this case, you'd likely have one or more entries in the domains key and only one entry in the public key. This would look like @ericdorsey's splendid example in the previous comment.
Another way to organize things would be to issue one certificate to cover a number of different websites. In this case, you'd have several public values. This would like as follows, taking the previous example and modifying it:
[[certificate]]
domains = [ "foo.com", "www.foo.com", "blog.foo.com", "bar.com", "www.bar.com", "blog.bar.com" ]

public = [
  "~/webapps/path_of_first_website_in_webfaction",
  "~/webapps/path_of_second_website_in_webfaction"
]

name = "combined_cert_name"

The main reason I've seen to use one over the other involves the "Certificates per Registered Domain" limit of 20 per week. In the above examples, it is using foo.com and bar.com which are two "registered domains". But if both examples involved different subdomains of foo.com, you might start running into the limit if you constantly issued certs for foo.com subdomains. Then it would be recommended to start combining them into the same certificate.
Beyond that use case, it's mostly a matter of personal preference. My practice is to group the domains logically. For example if I have one website running www.foo.com and foo.com and another website running blog.foo.com, I would probably combine these, assuming both websites were closely related. If I had two different websites running www.foo.com and mypersonalfilestorage.foo.com, I might try to keep these as different certificates.
Does this make sense?


fix: Simplified types and exports

BREAKING CHANGE: Removed findOneIana() and getAllIanaWindowsMap(), and renamed findAlias()
BREAKING CHANGE: findWindows() returns an array for consistency
BREAKING CHANGE: None of the functions return undefined anymore

:tada: This PR is included in version 5.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


fix win

@sbalandi just as an idea, maybe we can move notebooks in different directories? notebooks/sam2-image-segmentation and notebooks/sam2-video-segmentation? I think it should simplify many things (it does not required these proposed workarounds) and also aligned with our desire to have one notebook per one directory


LFS access token authentication

Gitea version (or commit ref): 1.5.0-rc1
Git version: 2.18.0
Operating system: Ubuntu 16.04
Database (use [x]):

[x] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[x] Not relevant

Log gist:

Description
The current LFS implementation lacks support for access token based authentication.
Due to this users with 2FA enabled is unable to interact with LFS repos.
As far as i can tell it should be pretty straightforward to implement this in the parseToken function in the LFS server module.
Let me know if i should look further into correcting this  :+1:

Feel free to create a PR for this.


Mt record endless recursion

Codecov Report

Merging #843 into master will not change coverage.
The diff coverage is 80.00%.

@@            Coverage Diff            @@
##             master     #843   +/-   ##
=========================================
  Coverage     82.63%   82.63%
  Complexity     1007     1007
=========================================
  Files            70       70
  Lines          4456     4456
  Branches        536      536
=========================================
  Hits           3682     3682
  Misses          510      510
  Partials        264      264

Impacted Files
Coverage Δ
Complexity Δ

...rs/sharedtable/impl/HashPartitioningKeyMapper.java
88.07% <0.00%> (ø)
12.00 <0.00> (ø)

...haredtable/impl/MtAmazonDynamoDbBySharedTable.java
78.54% <ø> (ø)
69.00 <0.00> (ø)

...lesforce/dynamodbv2/mt/mappers/MtAmazonDynamoDb.kt
78.57% <85.71%> (ø)
0.00 <0.00> (ø)

...s/sharedtable/impl/AbstractQueryAndScanMapper.java
96.63% <100.00%> (ø)
26.00 <0.00> (ø)

...s/sharedtable/impl/SharedTableMappingDelegate.java
99.12% <100.00%> (ø)
22.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27e9bde...8b58a53. Read the comment docs.


[WIP] Use azurite in CI

Use azurite in CI
Motivation

Fixes #502

Checklist
Before you create the PR, please make sure you have:

[x] Picked a concise, informative and complete title
[x] Clearly explained the motivation behind the PR
[x] Linked to any existing issues that your PR will be solving
[ ] Included tests for any new functionality
[ ] Checked that all unit tests pass

Azurite sucks and is very incomplete

Is there any positive feedback we can leave for https://github.com/azure/azurite, or are things really that bad?

I think I was too harsh but I believe since Azure released new breaking API changes this year Azurite is trying to play catch up and many features are missing in the new APIs. I think this is directly related to what we saw with the old Azure pypi package having breaking API changes. I would have to do more digging to figure out exactly what features are missing but if you look at the Travis logs for the unmocked job you will see a ton of Azurite not implemented errors

The project itself looks good and of high quality (not a node.js fan but I am trying to stay objective). I think things are just still in development for the new API version and that is totally understandable

Is there anything we can point out for them to fix, by opening a new ticket? Or is it already a known problem?


Additionnal game support

Would it be possible to add support for the following game:

Ark Survival Ascended

Thanks

Would it be possible to add support for the following game:

Ark Survival Ascended

Thanks

At the moment they utilize epic and there isn't an open means to query it. Unless its on steam query but don't see it happening anytime soon

In the old version of DiscordGSM I added a way to add a custom server.
The objective was to as least show the server in the Discord channel.
The State of the server could be linked to a simple ping response on the IP of the server.
The Server information like map, max player, game, etc, were all manually entered.
I understand that represent a bit of work but it would be useful to many I think.

Would it be possible to add support for the following game:

Ark Survival Ascended

Thanks

I've managed to add support for ASA on Discord GSM. I'll make a pull request when I'm done.

Ark Survival Ascended (asa) is supported since 2.15.0. 👍


[AutoDiff] Handle materializing adjoints with non-differentiable fields

This PR fixes a compiler crasher in AutoDiff.
The compiler used to crash while generating a pullback for differentiable functions that take a differentiable input, whose tangent vector contains non-differentiable fields.
Changes in this PR fix the issue by specializing adjoint materialization logic for user-defined tangent vectors containing non-differentiable fields.
Fixes #66522

@asl @BradLarson could one of you invoke swift-ci tests?

I am confused. I think the discussion was that we need to fix adjoint generation for struct_extract instruction. Are there other usecases besides struct_extract that should be covered?
What is the test coverage for newly added code?

@swift-ci please test

I am confused. I thought the discussion was that we need to fix adjoint generation for struct_extract instruction.

I think I may have misunderstood how to go about fixing the issue. Are you suggesting that we never add non-differentiable fields to the adjoint for struct_extract?

Are there other usecases besides struct_extract that should be covered?

If we do make the fix in the adjoint generation code, then I believe we'd have to at least do the same for tuple_extract. There may be other instructions as well, but I'd need to take a closer look.

What is the test coverage for newly added code?

In unit tests none.
In the end to end tests, I added a tests for the path where we need to materialize an adjoint with non-differentiable fields. I should perhaps also have added a test for materializing normal adjoints with all differentiable fields.

I think I may have misunderstood how to go about fixing the issue. Are you suggesting that we never add non-differentiable fields to the adjoint for struct_extract?

There are multiple questions that need to be answered:

Are there any other instructions that are affected here. Besides struct_extract? For example, you've added handling of adjoint buffers in addition to values. Are there any testcases for this code?

Next, the current adjoint generation for struct_extract is essentially:

  ///   Original: y = struct_extract x, #field
  ///    Adjoint: adj[x] += struct (0, ..., #field': adj[y], ..., 0)
  ///                                       ^~~~~~~
  ///                     field in tangent space corresponding to #field

So, instead of materalizing that bunch of zeros, why we simply cannot do the following:
adj[x].field += adj[y]

After all, we know that things should conform to AdditiveArithmetic and therefore a += 0 should be equal to a.

In unit tests, none.

Please ensure that all codepaths are covered by tests.

1. Are there any other instructions that are affected here. Besides `struct_extract`? For example, you've added handling of adjoint buffers in addition to values. Are there any testcases for this code?

Since the adjoint materialization code is shared by all instructions, we shouldn't need to handle things separately for different instructions. As for the handling of adjoint buffer handling, it looks like that code isn't actually called anywhere.

2. Next, the current adjoint generation for `struct_extract` is essentially:

  ///   Original: y = struct_extract x, #field
  ///    Adjoint: adj[x] += struct (0, ..., #field': adj[y], ..., 0)
  ///                                       ^~~~~~~
  ///                     field in tangent space corresponding to #field

So, instead of materalizing that bunch of zeros, why we simply cannot do the following:
adj[x].field += adj[y]

After all, we know that things should conform to AdditiveArithmetic and therefore a += 0 should be equal to a.

Ah, indeed this is what I tried to do first and I ran into an issue with TangentBuilder's emitInPlaceAdd method, which takes the lhs as a buffer. This was causing SIL verification failures for '+='-ing trivial types such as Double, with the following error message -
SIL verification failed: operand of 'apply' doesn't match function input type
  $Double
  $*Double
Verifying instruction:
   %0 = argument of bb0 : $Double                 // user: %8
     %5 = struct_element_addr %1 : $*P<Double>, #P.value // user: %8
     %6 = witness_method $Double, #AdditiveArithmetic."+=" : <Self where Self : AdditiveArithmetic> (Self.Type) -> (inout Self, Self) -> () : $@convention(witness_method: AdditiveArithmetic) <τ_0_0 where τ_0_0 : AdditiveArithmetic> (@inout τ_0_0, @in_guaranteed τ_0_0, @thick τ_0_0.Type) -> () // user: %8
     %7 = metatype $@thick Double.Type            // user: %8
->   %8 = apply %6<Double>(%5, %0, %7) : $@convention(witness_method: AdditiveArithmetic) <τ_0_0 where τ_0_0 : AdditiveArithmetic> (@inout τ_0_0, @in_guaranteed τ_0_0, @thick τ_0_0.Type) -> ()

It looks like I may have misinterpreted the meaning of the error, however. And I think I simply needed to change the type of the input arguments.
I can definitely try this out.

In unit tests, none.

Please ensure that all codepaths are covered by tests.

Would you know of an existing unit test that tests auto diff code? I wasn't quite sure how I can setup PullbackCloner for unit testing.

1. Are there any other instructions that are affected here. Besides `struct_extract`? For example, you've added handling of adjoint buffers in addition to values. Are there any testcases for this code?

Since the adjoint materialization code is shared by all instructions, we shouldn't need to handle things separately for different instructions. As for the handling of adjoint buffer handling, it looks like that code isn't actually called anywhere.

Allright, but what are those "different instruction"? Besides tuple_extract? Looks like you're adding whole bunch of generic code without any tests just to fix one particular corner case. Differential paths are known to have less-than-optimal test coverage and this usually cause all kinds of issues when the code is actually called in some obscure case. So we try to improve the test coverage, not to reduce it.

Would you know of an existing unit test that tests auto diff code? I wasn't quite sure how I can setup PullbackCloner for unit testing.

test/AutoDiff contain the tests for different aspects of autodiff code.

This was causing SIL verification failures for '+='-ing trivial types such as Double, with the following error message -

The lhs is a struct somewhere. So you should be able to take the address of its field and pass to += method.

Would you know of an existing unit test that tests auto diff code? I wasn't quite sure how I can setup PullbackCloner for unit testing.

test/AutoDiff contain the tests for different aspects of autodiff code.

Oh, I think I was mistaking unit tests for the ones that occur in the unittests directory in the Swift repo.

This was causing SIL verification failures for '+='-ing trivial types such as Double, with the following error message -

The lhs is a struct somewhere. So you should be able to take the address of its field and pass to += method. However, in your particular case it was rhs that caused the assertion as the argument is passed indirect.

Yeah it was the right one. I was a little bit surprised because I was thinking that only the left one needs to be taken as a pointer (or by reference). But I'm guessing this more generalized SIL definition exists to accomodate for different types that may want to conform to AdditiveArithmetic?

1. Are there any other instructions that are affected here. Besides `struct_extract`? For example, you've added handling of adjoint buffers in addition to values. Are there any testcases for this code?

Since the adjoint materialization code is shared by all instructions, we shouldn't need to handle things separately for different instructions. As for the handling of adjoint buffer handling, it looks like that code isn't actually called anywhere.

Allright, but what are those "different instruction"? Besides tuple_extract? Looks like you're adding whole bunch of generic code without any tests just to fix one particular corner case. Differential paths are known to have less-than-optimal test coverage and this usually cause all kinds of issues when the code is actually called in some obscure case. So we try to improve the test coverage, not to reduce it.

I see the point. Indeed, I missed a number of tests that could be added with this change. I was thinking we can start by covering pullbacks for instructions listed here.
Can you think of any others that I should also test?

2. Next, the current adjoint generation for `struct_extract` is essentially:

  ///   Original: y = struct_extract x, #field
  ///    Adjoint: adj[x] += struct (0, ..., #field': adj[y], ..., 0)
  ///                                       ^~~~~~~
  ///                     field in tangent space corresponding to #field

So, instead of materalizing that bunch of zeros, why we simply cannot do the following:
adj[x].field += adj[y]

After all, we know that things should conform to AdditiveArithmetic and therefore a += 0 should be equal to a.

@asl I've incorporated this feedback about adjoint materialization and am creating a new PR revision for some early feedback.
I'm still working on adding test coverage but am struggling a bit to understand what exactly the tests should be doing. I'll post my questions about tests in another comment shortly.

@rxwei Could you also please take a look at the approach and comment whether it makes sense and whether I am making changes at the right place?

@asl I've incorporated this feedback about adjoint materialization and am creating a new PR revision for some early feedback.

Have you forgotten to push your new changes?

@asl I've incorporated this feedback about adjoint materialization and am creating a new PR revision for some early feedback.

Have you forgotten to push your new changes?

I don't think so. I amended the existing commit, however, instead of creating a new one.

@asl or @BradLarson could one of you kick of swift-ci?

@swift-ci please test

@BradLarson @asl could one of you kick off swift-ci please?

@swift-ci please test

@rxwei @asl Thanks a lot for reviewing this PR so far.
I have fixed the nits. Could you please take a look?

@rxwei @asl Thanks a lot for reviewing this PR so far!
I have fixed the nits. Could you please take a look?

See new comment.

See new comment.
Did you forget to publish the comment 😅?

Also, will you please squash the commits?
Sure.

@swift-ci please test

@asl Had to make some changes after I discovered some small bugs while compiling and running internal test suite. I have added tests for these cases as well.

@swift-ci please test

@asl looks like the "Swift Test Linux Platform" failed due to some network issues. Can we restart them?

@swift-ci Please test Linux platform

@swift-ci please test linux

Huh I see the following error due to which the Linux tests failed -

ERROR: Error fetching remote repo 'origin'

Seems like something may be up with the Linux build hosts?

@shahmishal Is CI down?


Add build_id to AnalysisRequest. [phab:D2062]

Created by zarko at 2017-11-28 20:14:19:

shahms accepted this revision at 2017-11-28 20:18:48

zarko closed via commit df0f06e2b389fb52733d6dec69bd285331fbb0fb at 2017-11-28 20:22:06


1.1.73 Update , translating dictionaries stuck at 1%

Describe the Bug
Translating dictionaries stuck at 1.1.73
To Reproduce
Updated game. dictionaries stopped at 1%
Save file & Username
https://we.tl/t-A0IQsP5NfR
XnagitoX

This is due to a change in Factorio's locale system. I have determined the issue and filed a bug report with the Factorio devs. Once it is fixed, I will release an update to flib which will solve this problem.
It's not just affecting this mod, it's affecting any mod that uses flib's dictionary system.
https://forums.factorio.com/104297

Can confirm the problem still exists in 1.1.74

Like I said, the fix requires updated to both Factorio and flib. I have pushed the flib 0.12 update, which fixes this.

Thanks. its fixed.


refactor(typescript): navigation components and laws

Description
Comments/issues/screenshots:
#243

Added types for warning
Removed redux in Navigation.tsx, which means that it will check if it's any warnings on every render of new page. Will fix this soon with context/hooks (maybe useSWR()).
The bug where the warning-snack not disappearing when clicking it has been fixed
Added default font-sizes for h1, h2, h3 and set shape.borderRadius to 8 and applied that everywhere.
Renamed footer in Navigation.tsx to noFooter which means that pages now have to tell if they don't want the footer, insead of telling that they want it.
Migrated Laws.tsx as well so I could delete Link.js and LinkButton.js as they were unnecessary and not accecibility compliant.

Pull request checklist
Please check if your PR fulfills the following requirements:

[x] The PR includes a closes #issueID
[ ] The PR includes a picture showing visual changes

Typescript convert checklist

[x] All relative imports has been converted to absolute imports
[x] All HOC's have been replaced with Hooks
[x] Uses theme.spacing() instead of custom padding
[x] Uses theme.breakpoints instead of custom media queries

closes #229


Getting rid of deprecation warnings thrown by the latest PyTorch 1.2

My PyTorch version is the latest 1.2
when I am running examples/gcn.py to test whether I have configured PyG correctly, deprecations warnings keep throwing though the model works as expected. (see the screenshot below)
I guess it might be that some part of the framework is using the deprecated feature. If so, I think it is necessary to get rid of the warnings in the next version.
and I will appreciate it if someone could tell me which part of the framework I should make some changes to in order to mute those warnings temporarily.

Thanks for reporting. I am aware of this issue and am working to fix it. In the meantime, it is sufficient to convert the train_mask, val_mask and test_mask to dtype torch.bool in order to mute the warnings.

Fixed in master! I will craft a new minor release soon.


Extension issue

Hi, your library and the :
[UIApplication sharedApplication]
is not compatible with extension ... it would be nice to have been modified for extension :-)
cheers

Thanks so much for the PR! :)


Scrivener support

It should in principle be possible to compile straight from Scrivener through pandoc to what-have-you, citations and all. For this appropriate scripts would be placed in ~/Library/Application Support/MultiMarkdown for OSX -- yet to figure out what to do for Linux & Windows.

http://literatureandlatte.com/forum/viewtopic.php?f=21&t=30956&p=196747&hilit=pandoc#p196747 points at a way to point to the bibliography

http://literatureandlatte.com/forum/viewtopic.php?f=21&t=30956&p=196747&hilit=pandoc#p196747

proposal for workflow that will need to be supported
1. Writing in Scrivener, add citations
1.1. In Scrivener when you call for citation (cmd+Y) to open a plugin in which you can search zotero and select a paper. When you press on the paper to automatically copy the citation on the clipboard and then by cmd+v to add it in the text in Scrivener
1.2. Options for citations for example (author date) or (date) or author (date)
1.3. Options for multiple citations
1.4. The citation to be short for example something like {@santa, 2015} and be able to add page or some text before after the citation. The citation should look simple
2. In Scrivener compile for markdown (pandoc-markdown)
3. In BetterBibtex or new addon scan the generated markdown file and create word file that contains the citations and the bibliography
4. BetterBibtex or the new addon to give you an option to choose style
5. Get a fully expanded Word document including bibliography, with which you can do whatever you need to

http://tempsend.com/19E1A1797C

http://tempsend.com/26BC40D33B, url is now localhost:23119/better-bibtex/cayw

http://tempsend.com/94F11B21C2
So if your Cmd-Y command calls a script that does
#!/bin/bash
curl -o /dev/null localhost:23119/better-bibtex/cayw?keyprefix=@

the Zotero citation dialog will pop up and will leave the selected citations on the clipboard. If Scrivener wants to go for real integration (they haven't answered my emails yet -- disappointing) they could call the same URL and the body text will give the citations.
This handles 1.1/1.3/1.4. I don't know if pandoc handles 1.2.
2 would be handled by zotxt from @egh.
3 would no longer be necessary

would be specified in the front matter which will be picked up by pandoc

profit

So we're nearly there, really.

This is excellent. The development is going very fast. I will raise the issues with Scrivener and push the integration.
Is it possible to create a script that can be added in the field for Bibliography manager in Scrivener? This will help non-technical users to more easily use the integration.

#!/bin/bash
curl -o /dev/null localhost:23119/better-bibtex/cayw?keyprefix=@

is the entire script.

This was all release under 1.1.1 BTW.
If Scrivener were interested in a proper integration without having to paste in the results, this can also be done with 1.1.1, using https://zotplus.github.io/better-bibtex/cayw.html . But that would be up to them.

Sorry, that should be
#!/bin/bash
curl -o /dev/null localhost:23119/better-bibtex/cayw?keyprefix=@&clipboard=yes

You will want to read up on the pandoc manual. What's here can already be used with markdown, but it won't be anywhere near as fluid. The new picker + current zotxt + pandoc will do one-click full compile without any of the rtf scan malarkey.

Try to minimize the Zotero window(s) after the picker is closed.

Fixed: zotpick.app added to cayw page, plus instructions
Todo: review of installation of pandoc-zotxt
Todo: minimize Zotero after picker closes

Fixed: zotpick.app added to cayw page, plus instructions
Todo: review of installation of pandoc-zotxt
Fixed: minimize Zotero after picker closes
Todo: figure out why Scrivener prevents the clipboard from being filled
I will contact L&L; if I start zotpick manually, the clipboard gets filled, if Scrivener calls it, it is not.

now, by using zotpick2.app, the clipboard is filled in. The citation output is @Kane.etal.2014.whats not<EMAIL_ADDRESS>

Where does zotpick2.app come from? Did you appify that yourself? And this works consistently? Because mine does not, as in, it consistently doesn't work. Could you drop yours into https://dbinbox.com/retorquere ? I'm starting to deeply dislike OSX. Nothing just works, dammit.
BTW, the best I can do is to minimise Zotero after the pick is done; the picker expects the app that wants the pick to activate itself after the pick is done. Scrivener could easily do this (just raise itself after it calls the URL), but given that Scrivener is closed source and has no extension API, there's really not anything else I can do myself. Zotero will be out of the way, but it's unlikely that Scrivener will have input focus.

(oh and if you want<EMAIL_ADDRESS>the url you want is http://localhost:23119/better-bibtex/cayw?keyprefix=%40&citeprefix=%5B&citepostfix=%5D&clipboard=yes rather than http://localhost:23119/better-bibtex/cayw?keyprefix=@&clipboard=yes)

hrng I hate those apps trying to be smart on your behalf -- dbinbox also unzips whatever you put in it. even obliterating the directory structure in the process. Is there another way I can directly download the zip?

I used the download from the CAYW web site. The rename might have been made automatically by my computer. I placed the zotpick.app in Applications (in Finder) and then I placed it in Scrivener's Bibliography manager.
For now the workaround, when you open Zotero and then minimize it, makes it possible when you call the picker and select the bibliography you to "still be" on Scrivener.

Regarding the pick output format in Setting up Pandoc (https://zotplus.github.io/better-bibtex/cayw.html)
I changed it —to parametar to doc and docx but it failed to compile. It gives this error "pandoc: Unknown writer: doc”. Thus now it works only with html output. I test it it works.
Question: Can we use something that I found on pandoc site (https://github.com/mb21/Pandoc-Mac-OS-X-Services) to create doc output?

The link to the pandoc page lists supported formats (you can use docx, but doc is not supported), and caveat link does say some output formats are problematic: "though output to stdout is disabled for the odt, docx, epub, and epub3 output formats". We could perhaps fiddle around with putting the result in a temp file and then outputting that to stdout -- that might satisfy Scrivener.
Pandoc can create all these things just fine, it's just that Scrivener demands that it can run a script that reads standard input and outputs to standard output.

I get these errors when I use compile to MultiMarkdown ->Web (html)
With this script
#!/bin/bash
pandoc -F pandoc-zotxt -F pandoc-citeproc --from markdown --to docx
Error 1
There was a problem writing to the path '/Users/mijalce/Downloads/file.html'. You may not have the necessary permissions to save to this location - please try saving to a different place.
Error 2
Error parsing MultiMarkdown File
pandoc: Cannot write docx output to stdout.
Specify an output file using the -o option.
When I tried compile to MultiMarkdown ->Web (html)
With this script
#!/bin/bash
pandoc -F pandoc-zotxt -F pandoc-citeproc --from markdown_mmd+citations --to html
I get the same error 1 and
This error
Error parsing MultiMarkdown File
pandoc: Unknown writer: html
with this script
#!/bin/bash
pandoc -F pandoc-zotxt -F pandoc-citeproc --from markdown_mmd+citations --to docx
I get the same error 1 and this error
Error parsing MultiMarkdown File
pandoc: Unknown writer: docx

You should be able to work around this issue using a script like this:
#/bin/bash
t=$(tempfile) || exit
pandoc -f markdown_mmd+citations -t docx -F pandoc-zotxt -F pandoc-citeproc -s -o "$t" < foo.mmd
cat "$t" && rm "$t"

in the case of Scrivener it should probably be:
#/bin/bash
t=$(tempfile) || exit
pandoc -f markdown_mmd+citations -t docx -F pandoc-zotxt -F pandoc-citeproc -s -o "$t"
cat "$t" && rm "$t"

As Scrivener will pipe in the markdown through stdin.
Mind that I have no idea what Scrivener does with the output -- I haven't heard back from L&L how the pipeline works in detail.

(but what really should be happening is that L&L puts in place proper pandoc support of course)

I tried with both scripts
now there is no error, but the compile freezes in the middle and the output is not generated.
I compile Multimarkdown -> Webpage (.html)
One error I noticed, might not have any influence, but it might be connected - when it asks me where to save the output it automatically assigns the .html extension. When I try to change the extension and save the output as .docx it does not allow me.

As far as I can tell, only the html and LaTeX pathways allow interception the way we've documented. The other way would be to replace the binaries that Scrivener itself calls rather than pretending to be the convenience scripts, but for that we need input from L&L -- who have gone silent again.

Is Scrivener using Multimarkdown to convert to XHTML and then post-processing with an XSLT as described here: http://fletcherpenney.net/multimarkdown/use/
Because if that is the case I don't think this approach will work. You want Scrivener to output markdown with citations that can then be processed by pandoc, not HTML with citations - that cannot currently be processed by pandoc. (It can handle HTML input, but not with citations.)

They are and they aren't -- there's two ways MMD can generate XHTML, one using a one-go transformation using the mmd binary, and one that does post-processing using XSLT. If you choose post-processing, Scrivener will call a script that calls mmd to call the xslt, and what I do in the CAYW explanation is simply replace the script that calls mmd wholesale.
We will really need assistance fro L&L on this one. We're flying blind, Erik even moreso than me, trying to work around limitations that only L&L really knows. What we'd really need from L&L (if they're not going to do proper pandoc support) is an instruction on how we can achieve what we want given these limitations only they know in sufficient detail. What we want is simple: allow piping the generated markdown to a script and outputting in any of the supported pandoc formats.

I sent an email to L&L and waiting for their response.

Response from L&L on the paste issue:
I'm not set up to test the script precisely as you wrote it, but I did a dirt simple check to make sure nothing strange was happening with the integration between pipes and Cocoa process execution, and the building blocks all seem to be functioning. I created a simple .app with a Bash script that did nothing other than:
#!/bin/bash
echo "Test" | /usr/bin/pbcopy

This placed the word "Test" on the clipboard, and I was able to successfully do so from Scrivener's Cmd-Y hotkey. My guess: wrap your URL in quotes, that question mark might be tripping up execution.

Todo: add quotes around URL in zotpick.app
Done: review of installation of pandoc-zotxt
Fixed: minimize Zotero after picker closes

In the existing workflow solution, I have not identified how we can select the style (APA, Cell etc)?
Also, regarding 1.2 can we use the guidelines by pandoc-citeproc?
"Citations go inside square brackets and are separated by semicolons. Each citation must have a key,
composed of ‘@’ + the citation identifier from the database, and may optionally have a prefix, a locator,
and a suffix. The citation key must begin with a letter, digit, or _, and may contain alphanumerics, _,
and internal punctuation characters (:.#$%&-+?<>~/). Here are some examples:
Blah blah [see @doe99, pp. 33-35; also @smith04, ch. 1].
Blah blah [@doe99, pp. 33-35, 38-39 and passim].
Blah blah [@smith04; @doe99].
A minus sign (-) before the @ will suppress mention of the author in the citation. This can be useful
when the author is already mentioned in the text:
Smith says blah [-@smith04].
You can also write an in-text citation, as follows:
@smith04 says blah.
@smith04 [p. 33] says blah."
(source: http://pandoc.org/README.pdf, p. 49)

can I ask you when you add quotes around url in zotpick.app to use this url
http://localhost:23119/better-bibtex/cayw?keyprefix=%40&citeprefix=[&citepostfix=]&clipboard=yes
This url creates a citation (that I think) is more common in writing papers (author, date)

answer from L&L
"> The optimum solution from the user point of view would be to have Scrivener call the Zotero CAYW url itself and do its own pasting -- but this can of course only be done by L&L.

This should already be possible. For instance, if you test out Bookends with Scrivener, you will see that if you hit Cmd-Y in Bookends once you have the citation, it will return focus to Scrivener and paste the text in. This is done by sending Scrivener an Apple Event, which Scrivener accepts. I believe you just need to send Scrivener the kAEMiscStandards (‘misc’) event class with the kAEPaste (‘past’) event ID.

This will work (if someone finds out the requisite applescript incantation), but this is a workaround (and a hackish one at that), not a solution. This will paste the references in the first scrivener window found, whichever that may be - indeterminate if you have more than one open. The proper solution is still for scrivener to either call the picker themselves, or to add an api that allows me to do it.

Done: add quotes around URL in zotpick.app
Done: review of installation of pandoc-zotxt
Done: minimize Zotero after picker closes
Todo: add AppleScript code to zotpick.sh to give a random scrivener window the focus and paste the clipboard, in lieu of proper support by Scrivener.
The applescript code will probably look something like
tell application "Scrivener"
	activate
	tell application "System Events"
		keystroke "v" using {command down}
	end tell
end tell

Not tested yet.

zotpick.app has been updated BTW.

I made some tests of compiling in MMD and then using the terminal to us the terminal with the command > pandoc -F pandoc-zotxt -F pandoc-citeproc -s -o file.docx file.md < and it successfully makes the transformation.
Because Pandoc support will be enabled in Scrivener next year I would like to suggest the following solution for discussion
- The author writes in Scrivener, adds citations through zotpick.app
- It compiles in MultiMarkdown
- We create GUI app (part of BetterBibTex) that can be invoked through Zotero by using a keyboard shortcut or general shortcut.  This app based on the selections will compile the pandoc command we now use in the terminal. There are some existing wrappers and interfaces that can be use it (https://github.com/jgm/pandoc/wiki/Pandoc-Extras#pandoc-wrappers-and-interfaces), but they do not include the filters of pandoc-zotxt and pandoc-citeproc
- The author is asked to select the input file (file.md)
- The author is asked to select the Style (s/he can choose from the styles s/he has in Zotero)
- The author is asked for the format of the output file (docx, odt, pdf, rtf etc). With an option to check an option for default output
- The author clicks on convert button
The  new file is placed in the folder where the input file is. I do not know is it possible to have an option when the output file is created the input file to be deleted?
By having the GUI app we actually (independently from scrivener) make successful integration

What is interesting when I run the command we have in the script for compile MMD-HTML (pandoc -F pandoc-zotxt -F pandoc-citeproc --from markdown --to docx) in terminal it does not make the transformation. Thus there might be a thing that we need to change the command in the script.

the updated zotpick.app works perfectly.
One question regarding it:
In the zotero citation picker when I add the page or add text as suffix/ prefix that is not reflected in the citation key placed on the clipboard. I can use the syntax from pandoc to make this adjustments in the Scrivener document, but it will have more "like-word" feeling if the values I put in the picker regarding the page/ suffix/ prefix are added to the bibtex citation key (using the pandoc-citeproc syntax) and placed in the clipboard so I can add it in the document.

GUI app
I'm sorry, but the GUI app as described isn't going to come from me. This has a few reasons, but the main ones are:

BBT aims to make Zotero a usable BibTeX reference manager. Part of this is that I facilitate integration with programs that use BibTeX bibliography and/or BibTeX keys, but BBT does not aim to become a program that uses BibTeX(/keys).
I suck at building GUIs and I only build them when I absolutely must
Such a GUI built into BBT would mean I'd have to invoke external executables, which automatically means BBT would have to pass the continental-drift speed Mozilla review process once extension signing becomes mandatory. Right now, BBT passes the checks for automatic validation, and I intend to do whatever it takes to keep it that way.

There is an embarrassment of riches when it comes to markdown editors that either support pandoc, or are open source or have a plugin architecture that allows adding pandoc support cleanly; a much better way to address this problem. I can yield the selection via the http endpoint.
Selecting the citation style
I honestly don't know how to do this for pandoc.
CAYW picker reference details
How do you call up this interface? I only see the reference picker, not any place to enter details.

Yes, I completely understand you about the GUI app. It will take a lot of time. I am working to make it very clear how can it be done through terminal so that could ease the workflow for the other people that will use the integration
Regarding the CAYW picker reference details. When you have selected the citation, and the citation is in the picker, you just press on the citation and it will show dropdown options for page, prefix, suffix and suppress authors.

changing style in the workflow
When you compile the .md file through Scrivener you can use the following command to convert the file from .md to .docx by using zotxt and pandoc-citeproc and changing the citation style
In the Terminal you go to the folder where inputfile.md and citation-style-name.csl is (for example cd ~\Documents\Paper)
Then you put the following command

pandoc inputfile.md -o outputfile.docx -F pandoc-zotxt -F pandoc-citeproc --csl=citation-style-name.csl

Important: You put the both files: inputfile.md and citation-style-name.csl, in the same folder (for example cd ~\Documents\Paper)
With this I think that we have completed the whole workflow we have set in the beginning. Now we can work on making it more nicer :)

I've updated https://zotplus.github.io/better-bibtex/cayw.html; http://tempsend.com/55AA42F9D8 goes with that for now until I publish a new release. The Pandoc format is too much for simple prefixes etc, so I've just made it so format=pandoc or format=mmd does the right thing given the selection.
WRT the GUI app, it's not just that it will take a lot of time (it may or may not), it simply falls outside the scope of what BBT wants to be. This really is work that Scrivener ought to be doing themselves, and I'm quite willing to adapt BBT to facilitate integration, but it can't be the case that I'm building glue apps to make up for deficiencies in an app that is hostile to integration builders; not open source, and zero extension facilities. Even MS Word is more open than Scrivener. If I'm going to build GUI apps, I'd much sooner work on http://leoeditor.com to bring that up to par with Scrivener.

1.2.20 has the new caw endpoint

some update on the workflow, that make settings one time activity (even when you change styles) and simplified process how to convert.
use the following content for the new mmd-xslt (see https://zotplus.github.io/better-bibtex/cayw.html how you can do that)
#!/bin/sh
pandoc --from markdown --to html -F pandoc-zotxt -F pandoc-citeproc --csl=/Applications/citation.csl
In order to be able to change the citations as you like I made a decision to give a generic name to the csl file (citation.csl) and to place it in a location where not much changes are done (Applications). Also it is the same path for all the users and no need for adjustments of the content.
In order to change the citation style, you download the wanted citation style from Zotero Style Repository and when you save it, you save it in Applications with the name citation.csl. So you overwrite the previous citation and now pandoc-citeproc uses the new citation style.
When you have done that, you run Compile Markdown -> Web Page (.html)
Save this html file where you want.
To convert the html to docx you need to use the terminal (the existing GUI complicate the issue more). However, because very often the html is located somewhere in a folder that is in a folder that is in a folder (i.e. the folder where you have all your documents for the paper) and has empty spaces in the title it can be problematic to enter everything correctly in the terminal. Thus I propose the following procedure:
Open Terminal and write
Pandoc (empty space)
than go in Finder and drag your html file in the terminal. You will get something like this /Users/mijalce/Desktop/My\ Beautiful\ paper.html
than leave empty space and write -o (leave empty space)
than again drag and drop the same html file in the terminal. However now you delete .html (at the end) and write .docx and press enter. Then in the same folder you should have the My Beautiful paper.docx
The content in the terminal should look like this:
pandoc /Users/mijalce/Desktop/My\ Beautiful\ paper.html -o /Users/mijalce/Desktop/My\ Beautiful\ paper.docx
(in bold is what you write)

I've put up an editable version at https://github.com/ZotPlus/ZotPlus.github.io/wiki; can you update the page with your contributions? When you're done, let me know, and I can publish it to the site.

Thanks. I will update it tomorrow.
One thing that I found out. With the proposed script If my scrivener file has image, I can not create one html, it creates a folder.  However I need standalone html file. To do this I need to use --standalone command. I tried several timse to make the order of the commands appropriate so I can get standalone html, but with no success. Do you know how we can do this?
Thanks

I don't really know -- it shouldn't matter what order they're in, but your best bet is to ask on http://stackoverflow.com/questions/tagged/pandoc

for standalone html the content of the script should be
#!/bin/sh' 'pandoc --from markdown --to html --standalone -F pandoc-zotxt -F pandoc-citeproc --csl=/Applications/citation.csl
However I suggest not to use it because of the following:

you need to put the image in scriv document like this ![This image won't be a figure](/url/of/image.png)\ or like this ![This is the caption](/url/of/image.png). If you make mistake in location pandoc will show error
if the image is too big (MB) it will create problem in the conversion process
if the image is too large (size W/H) it will take large space of your html and latter docx file.

Thus you can use the --standalone command, but with care

I updated the Wiki. Please inform me when you publish it so we can spread the word for this (in my opinion) very good integration.

OK, the page is up.

Information shared at
Zotero - https://forums.zotero.org/discussion/51168/scrivener-and-zotero-integration/#Item_1
Scrivener - http://www.literatureandlatte.com/forum/viewtopic.php?f=5&t=32047&p=201800&hilit=zotero#p201800
Marked2 - http://support.markedapp.com/discussions/suggestions/44981-scrivener-zotero-markded-integration

I appreciate the accolades, but I assure you Erik's work is at least as
important as what BBT provides!

One question:
is it possible to have a downloadable script (or an app that) when executed will make all the things that we propose in Setting up pandoc section?

It is technically possible, but I don't feel comfortable writing programs that root around in directories of other applications. I'd be blind in making changes to other people's systems without any way to reproduce any errors. I'm not opposed to adding a link to such a script or app, but it's not something I feel comfortable doing myself, sorry, As the instructions are, the user is in control, and will be required to apply good judgement; a script cannot do this.

you have a good point. Also, what I noticed is that on Scrivener forum the post already has 90 views and still no questions about issued how to do the setup. Thus it might be that the complete process is already explained well and not very demanding. As a result development of such a script or app is not a necessity.

OK, I'm going to consider this fixed, barring any problems with the cayw endpoints; please open a new issue if problems arise with the endpoint.

Hey, if you are interested in doing the scrivener integration with pasting happening automatically, I can confirm the following as working and stable. Can easily be packaged as an app:
do shell script "/usr/bin/curl 'http://localhost:23119/better-bibtex/cayw?format=pandoc' | /usr/bin/pbcopy"
tell application "Scrivener"
	activate
	tell application "System Events"
		keystroke "v" using {command down}
	end tell
end tell

I have pasting into scrivener working and tested if you're interested. Applescript which runs bash and bash script which runs osascript below. The bash script running embedded applescript is faster even though it is mostly applescript because... Apple.
Bash Script
#!/bin/bash

/usr/bin/curl "http://localhost:23119/better-bibtex/cayw?format=pandoc" | /usr/bin/pbcopy
/usr/bin/osascript -e 'tell application "Scrivener"
	activate
	repeat until application "Scrivener" is frontmost
	end repeat
	tell application "System Events"
		keystroke "v" using command down
	end tell
end tell'

Applescript
do shell script "/usr/bin/curl 'http://localhost:23119/better-bibtex/cayw?format=pandoc' | /usr/bin/pbcopy"
tell application "Scrivener"
	activate
	repeat until application "Scrivener" is frontmost
	end repeat
	tell application "System Events"
		keystroke "v" using command down
	end tell
end tell

I've updated https://github.com/ZotPlus/ZotPlus.github.io/blob/master/better-bibtex/cayw.md; should appear on https://zotplus.github.io/better-bibtex/cayw.html soon (github pages have been slow to update lately).

@davepwsmith thanks for doing this. I used the Bash script and it works perfectly. Even with multiple Scrivener files opened it pastes in the one from which you called the picker.
@retorquere , can I ask you on the CAYW page to bring back the downloadable app only with the changed content from the Bash script. I think that in this way people that do not know how to make-script-change-to-app will be able to download the app and add it in Scrivener.
Thanks!

Sorry, should probably also note that presumably GUI scripting must be enabled for this to work (although I haven't verified or tested this!).
App here:
https://www.dropbox.com/sh/htybak3pb3uhfp0/AAAvozL5etMu7V9WSH-gx9Csa?dl=1

"What really should be happening is that L&L takes the 30 minutes it would take to add this to Scrivener and make this conversation irrelevant."
I wouldn't hold your breath. Scripting with external applications has been a long-requested feature and absolutely no interest has been shown.

Not holding my breath, but also not taking a potentially risky responsibility because they have no interest in their users' needs.

OK, sorry, back on topic:
Bug with the search box when called with the script:
If you do the following, then it is impossible to get rid of the box without quitting zotero:

Open search box with URL call piped to pbcopy.
Press [Esc]
Attempt to enter reference followed by [Enter] fails.

It's also problematic that you can't dismiss the box using [Esc].
Presume that this has to do with the pipe in Bash -- You can escape out if the box is from a url called from the browser.

Can you open a new issue for this?

I think that we have two very good solutions for the picker. Both have their weak and strong side. I think that the win-win situation is when we offer both of them to potential users, with appropriate explanation for the possible issues with the auto-paster app. When the bug is fixed in the auto-paster I can hosted on my dropbox or use Dave's dropbox and link it to CAYW.

Tests on #334 are running.

new version with fix is out