Adding polarization support

I wanted to simulate a system with birefringent optics when I looked into OpticSim.jl code to see the polarization is not yet supported. I am unsure if I can volunteer on it, maybe I can do it by myself or delegate to some of friends/colleagues. But prior to doing so I wish to cross-check if there is already an effort on this.
W.r.t. @caseykneale's quote below, I believe that Jones is not enough as it handles only polarized light, but not unpolarized. IMHO the correst formalism is Stokes, or in other words Poincare sphere plus power. To me the easiest way how to enhance the current processintersection is to keep power variable untouched and add 3 Poincare coordinates definind the SoP (state of polarization). What is still missing is to define a coordinate system for polarizations. In real world one of popular choices is H/V, but -- correct me if I'm wrond -- this dies of singularity in case of purely vertically propagating ray. Therefore what I have in mind (and not yet on paper/code editor) is, for each particular ray to define a non-ambiguous coord system derived from its world-coordinate direction of propagation.
To my view, the changes needed are:

altering the number of return values in processintersection() (adding SoP), and adding support for full Fresnel eqns there

keeping in mind that in case of birefringent material instead of having <=2 outcomes (t and r), we now may have <=3 (r, o, e)

propagate new component of Ray (SoP) throughout the code
adding support for anisotropic materials, defined as indicatrix tensor matrix in place of simple index of refraction scalar

plus few convenience idealized components, such as ideal polarizer, ideal waveplate

Any feedback welcome.

Originally posted by @caseykneale in https://github.com/microsoft/OpticSim.jl/issues/33#issuecomment-804431676 As for representing polarization - it's tricky. I don't know if anyone has a good way of doing that beyond field equations. The Jones/Mueller matrices are a little niche but they can be broadened out to handle other systems.  At one point I was eyeing up adding polarization to local plane interface approximation (LPIA) for a system somewhat similar in spirit to what you all are working on. Just ran out of time. Well - I can do a code dump at some point soon and you all can decide if any of it meshes with your interests or not.

I've just found OpticSim so I don't know much for the underlying infrastructure. One resource to different options for polarization raytracing is "Polarized Light and Optical Systems" by Chipman, Lam and Young. There is enough detail there to guide good decisions about how to implement the support needed for efficient and accurate polarization raytracing. The textbook approach means that it almost provides cookbook guidance for writing code.
In particular, it describes coordinate systems and issues with phase pretty well.

(..) The textbook approach means that it almost provides cookbook guidance for writing code.
In particular, it describes coordinate systems and issues with phase pretty well.

Thanks for the pointer, I will definitely have a look.

@hrobotron, @mattderstine I've wanted to support polarization for a long time but don't (currently) have the background to design the api. I ordered the book you recommended and start thinking about an api. I can't work on it full time because we are focused on improving OpticSim support for repeated structures, but can definitely help.

@BrianGun I had been working on my own raytracing code in Julia before I saw this project. It wasn't around when I started. I've just started looking at it and will be thinking about this more as it is much better than what I did in many regards and seems to have the same basic approach, even to using Makie for graphics.
With just the quickest of looks, there are some design decisions being made that may have a major impact on how easy it will be to add polarization later on. In particular, I have noticed that there is some stochastic decisions sometimes made about r & t and the way info is managed with that process. Doing polarization is just like keeping track of r & t except there is more bookkeeping along the way. This means that decisions about detectors and what data to keep when doing a raytrace are very important.  Most importantly, I think is that a potentially significant advantage for OpticSim and polarization raytracing is the ability to easily include nanostructured interfaces and meta materials. This is what I would keep in mind while developing the raytracing planning for adding polarization.  I'd look at chapters 9,10, 19,20 for how to do polarization raytracing and backtrack to the other chapters to understand the details and applications.  I look forward to helping where I can.

@mattderstine Reading the polarization book is a workout in the physical as well as mental sense. It is massive. But it looks excellent so far.

With just the quickest of looks, there are some design decisions being made that may have a major impact on how easy it will be to add polarization later on. In particular, I have noticed that there is some stochastic decisions sometimes made about r & t and the way info is managed with that process. Doing polarization is just like keeping track of r & t except there is more bookkeeping along the way. This means that decisions about detectors and what data to keep when doing a raytrace are very important.

I think that the stochastic dealing with t/r is quite OK for polarization as well. At the interface, we have a reflection, and one or two transmitions depending on birefingence. The Monte Carlo approach is good in dealing with that in a sense that we do not suffer explosion in number of rays (doubling or tripling at each interface).
The question to decide is whether to support sull SoP including partially polarized light (Poincare/Stokes) in each ray, OR to delegate the partially polarized light to Monte Carlo as well and do it stochastically (Jones/Mueller). I think it is better to cover it in the SoP for each ray.
Does the nanostructure you talk about differ in any way compared to naturally birefringent materials? I'd think the basic formalism for a material w.r.t. polarization is changing from scalar refractive index to indicatrix tensor, do the materials you talk about require something better?

The issue with polarization ray tracing through birefringent materials is that rays do explode. The issue I am concerned about is not the pruning of rays to those of interest, but that the bookkeeping of the polarization characteristics along the ray. What I'm really suggesting is that when a ray is pruned, the r or t of what remains should be tracked (and by this I'm assuming these are complex number with amplitude and phase, not R or T for power). The polarization that needs to be tracked includes these properties in multiple dimensions. This is important when the rays are to be combined at some point when all of the amplitudes, phases and k vectors will determine the properties. This even more important if a material or surface is depolarizing. Ensuring the bookkeeping is present enables this to handled efficiently rather than having to trace ever more of an exploding set of rays.
My preference about tracing polarization is to separate the polarization from the idea of refractive index. If one were to do RCWA on these structures the polarization properties come out more naturally than forcing the answer to be in terms of index, whereas the polarization properties can be easily found from the index of more natural materials.  In the end it's a matter of bookkeeping and understanding. Keeping things in terms of polarization aberrations that can be aggregated across the system (compare to wavefront aberration for example) make it easier to understand what is happening rather than having to pretend there is an element that represents the system that has a certain refractive index (complex, of course) tensor .
Naturally the choice that works best is problem dependent. I'm just advocating for more general structure of the raytrace so that the interesting problems of our day (AR headsets say) can have the robust foundation needed to generate results that are trusted even when it is difficult to understand them.
I attach a white paper from AiryOptics that shows off their raytrace program.  This is just the simplest kinds of operations that I'd like to to be able to use. I'd also like to be able to get these same kinds of plots after going through birefringent-like coatings and materials. For example Example 20.2 of Chipman, Lam & Young.
CassegrainWhitePaper.pdf

Hello,

(..) The issue I am concerned about is not the pruning of rays to those of interest, but that the bookkeeping of the polarization characteristics along the ray. What I'm really suggesting is that when a ray is pruned, the r or t of what remains should be tracked (and by this I'm assuming these are complex number with amplitude and phase, not R or T for power). The polarization that needs to be tracked includes these properties in multiple dimensions. This is important when the rays are to be combined at some point when all of the amplitudes, phases and k vectors will determine the properties. This even more important if a material or surface is depolarizing. Ensuring the bookkeeping is present enables this to handled efficiently rather than having to trace ever more of an exploding set of rays.

yes, indeed. This is exactly what I am talking about since the first post in this enhancement, nothing more, nothing less.

My preference about tracing polarization is to separate the polarization from the idea of refractive index. (..)

The ideas are separate. Even the fully isotropic media (scalar index) do have polarization effects via Fresnel equations. Even current OpticSim code (in processintersection()) does talk about them, although taking only approximate result just because that the SoP is not propagated further.
I mentioned indicator matrix, as this is needed and AFAIK sufficient measure how to properly handle anisotropic materials once the SoP is introduced into the code.

I attach a white paper from AiryOptics that shows off their raytrace program. This is just the simplest kinds of operations that I'd like to to be able to use. (..)

Sure, I see no problem with that and the Cassegrain example you provided shall indeed work without any problems.
I am afraid we are are at the point where I or someone else shall show some code. My time allocation is less than poor, but let's see, if I will be struck by procrastination in this direction, I will contribute for your review :-)

I've looked at the polarization textbook. Doesn't look like it would be that difficult to add. But how will it be tested? Do you all have polarization systems with known responses that the OpticSim implementation could be compared to?

But how will it be tested? Do you all have polarization systems with known responses that the OpticSim implementation could be compared to?

I think that a plan-parallel plate of BK7 with non-perpendicular angle of incidence is a good starting point :-) After introducing indicator matrix, next valuable example could be beam displacer using a single piece of prism, demonstrating how the beam splits into three.

I agree with @hrobotron that simple examples could be compared to results calculated directly.
For what I had been doing, I was testing against the textbook examples and problems. This was an easy way to check the work.
The white paper has examples that could be modeled, at least qualitatively. If you have access to Zemax and CodeV they have some capabilities and side by side testing would be interesting if not definitive. I would also look to the papers and dissertations of Chipman's students. I expect good examples might be found there.

Just confirming I also got the book, starting to look.

I have briefly looked into the formalism of "Polarized Light and Optical Systems" by Chipman, Lam and Young. I welcome the WCS of P matrix. However, the formalism seems to deal only with fully polarised light, which I think is a pity: the common case of unpolarized light would need a stochastic mixture, which I think is a wasteful approach.
To me at this point seems the best way to go along Zhang et al: Three-dimensional polarization ray tracing calculus for partially polarized light DOI:10.1364/OE.25.026973. This is my suggestion and if I will have some spare time (which I do not promise) I'd like to contribute either some initial code, or at least test examples.
It seems there is even a more general approach described in later Li et al: Three-dimensional polarization algebra for all polarization sensitive optical systems DOI:10.1364/OE.26.014109, but here IMHO the added value (i.e. the new cases this is able to deal with compared to previous one) is arguable, given much larger matrices and therefore complexity in this case. (Change my mind -- then please say which practical examples covered by Li&al are not covered by Zhang&al.)
So my suggestion is clear: DOI:10.1364/OE.25.026973. Which is just a well described and presumably proven formalism capturing the spirit of my initial post (i.e. to support also partial polarizations).

I'm going to go silent while I dig into this. My current understanding of the approach by Chipman is that the results from characterizing the optical system can then be used with unpolarized and partially polarized input light to determine the output. However, this is not something that I've spent much time with and so need to look carefully at the details to make sure that I understand the limitations of one approach over the other. Thanks for the detailed references.
One question that I know little about his how depolarizing optics are modeled. It would seem to me from basic physical principals that the k-vectors are also changed in this process and so modeling this kind of element becomes stochastic as well. Or is this outside the scope of what we can expect a "general" optical modeling program to cover?

I'm going to go silent while I dig into this. My current understanding of the approach by Chipman is that the results from characterizing the optical system can then be used with unpolarized and partially polarized input light to determine the output. (..)

According to my (very brief) skim of Chipman&al and according to words by which Zhang&al and Li&al refer to Chipman&al, which is consistent with my view, the "ray property" in Chipman&al is polarization matrix which is equivalent to some polarized light, with totally arbitraty polarization, BUT this means that the unpolarized light can not be modelled by a single ray. The unpolarized light is not a single polarization -- it is a stochastic mixture. This and only this reason is why I suggest to advance to Zhang&al. The simplest approach to understand the matter is Stokes vs. Jones (this is without 3D).

One question that I know little about his how depolarizing optics are modeled. It would seem to me from basic physical principals that the k-vectors are also changed in this process and so modeling this kind of element becomes stochastic as well. Or is this outside the scope of what we can expect a "general" optical modeling program to cover?

I am not competent to answer. AFAIK depolarizing optics in real life always change optical mode of the light (geometry of the beam), if made strictly off passive components. But I have no strong opinion and zero knowledge of this. Everywhere I talk about unpolarized light is where the ray has random polarization already at the source. (And along the way through "lens and prism assembly" it can only split into a larger number of more-or-less polarized rays.)

Still working but some first impressions. Either method works fine for partially polarized light as long as there is no depolarizing elements in the system. It's just a matter of converting the Jones matrix to the Mueller matrix and then using the proper stokes vector for the input light. From my quick read of Zhang et al, their comments about the limitations with regard to partial polarization of Chipman's approach can be attributed to the details of polarization raytracing being complicated and the pressure put on authors to emphasize novelty beyond what might really be there.
On the other hand, I take seriously the comparison of the double gauss lens where the value of the 3D coherence method is described as an improvement over an linearization assumption made by Chipman's group. At the same time, this means that I need to dig into the details of the calculations on both sides to see where different approximations are made.  This statement is rather surprising to me as when I implemented the Chipman method, it did not appear that there were any approximations, but I need to get to the bottom of this.
One limitation that is not discussed by Zhang et al, is how this would model a depolarizing or scattering element. Here I reference the discussions of approaches described in Chipman sections 27.2.8-27.2.10. These explain strategies to address the depolarizing with raytraces. It is clear that unless the raytrace is a Mueller matrix based approach then depolarization is not well handled. My limited understanding of Zhang suggests that it is not. On the other hand, the scattering the comes with depolarization is challenging to model and so just using a Mueller matrix would not really make a more useful tool.
As I said, this isn't a final conclusion, just a quick update along the way to help me organize my thinking and perhaps get pointers where I might look.

@mattderstine @hrobotron
I created a new branch (#276) for this issue and have started modifying the code to add polarization information to OpticalRay. In the new system you create an optical ray with polarization = nothing, or to a subtype of AbstractPolarization. Currently the only subtype is Chipman defined as an empty stub. As soon as I get the current code working with polarization = nothing then I'll start on the Chipman polarization tracing.
It doesn't look like the polarization tracing will take long to code, unless I'm misreading the book. Once I finish could one or both of you write tests to make sure the code works?
It will simplify life if you make all your modifications to this branch.

@mattderstine @hrobotron
I'm going to rewrite processintersection for FresnelInterface to handle polarization using the Chipman method because this looks like something I can handle with my beginner's understanding of polarization. I have no idea how to do this for the holographic elements. Do either of you know how to do this?
The new code is written so that we can easily add a new polarization type without rewriting any of the trace code. We should only have to rewrite the processinintersection functions. I'll work on abstracting the polarization code out of the processinintersection functions as much as possible so that only a little code has to be rewritten to handle different polarization models.

@hrobotron I had a more careful look at the beginning of the Zhang article and the double Gauss lens. Unlike the their analysis of the first part of the article, they didn't actually implement the polarization approach of Chipman for the double Gauss. I looked carefully at Chapter 9 and there is no paraxial approximation made. All of the analysis is valid over all angles. This realization combined with other unwarranted claims and mistakes in the paper lead me to the conclusion that for nearly all cases, the approach in Chipman is valid and, given the more complete description of the pitfalls of implementation, is what I would choose for a robust polarization raytrace for unpolarized, partially polarized and polarized sources.
This approach does not work well for optic systems that have depolarizers. In these very few cases, I think that the approach outlined in 27.2.10 would be appropriate. I have never seen such elements used in an optical design that was complicated enough that the performance wouldn't be better modeled by just coding the Mueller matrices directly. With the proper combination of rays at the end (and with that statement I hide lots of complexity), the Chipman approach would enable modeling of polarimeters, fiber optic isolators for unpolarized light and all manner of polarization performance of birefringent and normal optical elements.

@BrianGun I have not really had time to look at OpticSim, I've been deep in the papers and book. I like your approach although I do suggest you read and understand chapter 8 & 9 before actually implementing the stuff in chapter 10.
Starting with the Fresnel equations is a great choice. Holograms are an advanced topic that I suggest is on the back burner for a while; unless someone else is really pushing for them and can contribute. A more important next step is coatings. I have no idea what is in the package right now for coatings (I really have only ever run the first example, read some of the examples and looked at the issues on GitHubs.   There are references to polarization changes in holograms and Chipman has a chapter on RCWA that all provide background.
The key point is that your choice of placing the polarization interaction into the processinteraction methods means that nothing breaks as new calculations of the P matrix are added for new surface types and there is no need to implement all of these things at one time.
A bigger issue for the long term is handling the ray traces with strongly birefringent materials. This will lead to bifurcating of the rays at each of these elements. Again, I haven't looked at the OpticSim raytrace, but these materials will require different raytrace calculations.This is the point of chapters 17-20.
I'd recommend a plan something like this (with the expectation that whomever is doing the work should push the functions they need):

Add the basic code to permit calculation of the P matrix at each surface and manipulation at the end of raytrace
Add the the code to calculate the P matrix from the fresnel equations for normal surfaces and optical materials, including metals with complex refractive indices.
Write functions to calculate Jones Pupils and other visualizations of Chapter 11 and 12
Write functions to handle unpolarized/partially polarized sources and analyze their performance
Add polarization effects of coatings (this is project that will never really be done)
Add a real birefringent raytrace
Add beam combination functions and visualizations
Plan for other experts to find the package, make improvements to what has been done and add the fancy optics for their specific problems.

I need to figure out how much I can really help, but I certainly would like to write functions that test all of the examples and problems sets from the book. I see two advantages to this approach. The first is that there is a very good chance the answers in the book are correct and the second is that the book then helps explain what the code does to make it easier for people to use.
Thoughts?

@BrianGun @mattderstine

I created a new branch (#276) for this issue and have started modifying the code to add polarization information to OpticalRay. In the new system you create an optical ray with polarization = nothing, or to a subtype of AbstractPolarization. Currently the only subtype is Chipman defined as an empty stub.

Dear Brian, thanks a lot for the quick action. I believe it is very wise to introduce SoP as an abstract class, as our preference and understanding may eventually evolve (although better to choose the "best" one now in order to avoid future massive code rewrites for different classes of interfaces).

It doesn't look like the polarization tracing will take long to code, unless I'm misreading the book. Once I finish could one or both of you write tests to make sure the code works?
It will simplify life if you make all your modifications to this branch.

I will happily help. I will start with taking your branch and refreshing my mind on how to write a proper test.

Dear @mattderstine

Still working but some first impressions. Either method works fine for partially polarized light as long as there is no depolarizing elements in the system. It's just a matter of converting the Jones matrix to the Mueller matrix and then using the proper stokes vector for the input light.

the Mueller/Stokes approach is what I am heading towards, in contrast to Jones, which is simply incapable to support the most ordinary, unpolarized light. Which I believe still shall be the default source and default input for a generic lens assembly for a person who does not care about polarization much.
While you can easily convert Jones matrix to Mueller matrix, you can not do the opposite. And Chipman's P-matrix is just a tool how to handle Jones in 3D in WCS (world or lab coordinate frame). Therefore for this reason I think it's not what we want. Maybe we can draw an inspiration and test cases there, yes. <= Correct me if I am wrong in understanding what P matrix does in Chipman.
I need to read the Zhang (which I have not yet) to see if it really provides something like marriage of Chipman and Mueller, OR if it is more into the direction of your impression.
FYI: @BrianGun

Dear @hrobotron,
The Jones approach is actually the more general approach for a raytrace. It supports both polarized and unpolarized light because it can be reduced to Mueller matrices in the end. The claims by Zhang that Jones cannot be used as the basis for modeling systems with unpolarized light is absolutely incorrect.
Let me be clear and explain the process of how this would work. A traditional optical system is modeled (let's just say the telescope in the Airy Optics white paper). The polarization changes for each surface are computed and represented in the P matrix. Because of the formulation of the polarization calculus, it is relatively simple to computer a P matrix for the entire system. This can then be used to generate Jones matrices as a function of pupil position (what Chipman calls the Jones pupil). It is also straightforward to compute the Mueller matrix for each position in the pupil. From this, a source with an arbitrary set of Stokes parameter can be used to find the Stokes parameters at all coordinates in the pupil and in the image plane. If you think about the problem, this telescope is almost certainly for imaging unpolarized light. By using the Jone approach, it clearly shows where the issues are in the design and how the imaging properties will be effected by the polarization changes. And then, when the design is finished, the stokes parameters can be calculated to understand what the polarizing at the detector might be. However, this is probably less important than how the resolution is impacted by the polarization changes.
The Mueller approach does not support the phase information that is tracked in the Jones approach. This is why you cannot go from Mueller to Jones. It is also why you could not model the changes to imaging performance with Mueller as shown in the example .There is less information about the optical system in the Mueller approach. As I said before the only place the Meuller approach does better in a raytrace is when there are depolarizing elements like opal glass or painted surfaces. If this is where your interest lies, then I agree that the Mueller approach is better. If you interest is seeing how systems made of more traditional optical elements work with unpolarized light, then I think the Jones matrix with proper interpretation would be a better investment of your time.  If your interest is in applications like polarimeter cameras, then the Jones approach is the clear choice. It will give much better insight into what is happening in the optical design and what is happening to the polarization than a Mueller raytrace, but at the same time easily provides the Mueller matrices needed to examine the Stokes parameters to link it to specific application problems.
Zhang is primarily an alternate formulation of tracking polarization with alternative ways of generating results that mirror the Mueller and Jones matrices. It's primary advantage is that if somehow have a depolarizer that doesn't also scatter it will better model the results than other methods (Chipman, Jones, Mueller, ...). However, such a depolarizer seems unphysical to me. I cannot imaging how to change the entropy of just this parameter without altering the entropy of other parameters. As far as I know, all true depolarizers either change the light in spatial or temporal dimensions as well. Pseudo depolarizers are the exception but these could be modeled with the Jones approach because there is a well defined polarization transformation at each place across the aperture.

Dear @mattderstine
Let me concentrate only to a small but fundamental subset of what you wrote:

The Jones approach is actually the more general approach for a raytrace. It supports both polarized and unpolarized light because it can be reduced to Mueller matrices in the end. The claims by Zhang that Jones cannot be used as the basis for modeling systems with unpolarized light is absolutely incorrect.

Here I can not agree with my current knowledge. You can see in my initial post where I was completely unaware of both Chipman as well as Zhang that I was heading in the direction which has the same basis which you call absolutely incorrect. So, either you have some manipulation of the rays in mind which I do not follow, or one of us is incorrect. I believe there is one simple test which may reveal the truth:
please, can you take one single ray, unit power, zero phase, and write me either Jones vector or Chipman P matrix to represent this ray? But please, really do write numbers, not human language sentences.
If you succeed, I hope I will learn a lot or at least see the direction.
If you fail, then I think our minds will be in tune.
Thank you for your effort.

jones matrix example.pdf
Attached is a pdf of a Mathematica Notebook that shows how this is done for a nontrivial example. Something is wrong with the Jones to Mueller matrix conversion, probably with Mathematica's rigid implementation of complex numbers, but I'm not going to spend the time to fix that.
I picked an example with a wire grid polarizer and a zero order waveplate. This kind of optic might be found in an old LCD projection TV.  This kind of analysis might have been done to compare the design to measured performance.
The example uses Jones matrices rather the the polarization ray trace matrix of Chipman because the rays directions never changes. Doing it with the P matrix would have just added lots of extra zero elements to the Jones matrices. It shows the basic process. Implicitly a ray is traced through the system. At each interface the Jones matrix is computed (or in my case, they can trivially be stated). Then the system Jones matrix is computed (or the P matrix if that was chosen). This is then converted to a Mueller matrix and then multiplied by the Stokes vector for unpolarized light computing the polarization state at the output.
Pardon the Mathematica, but I'm not yet proficient enough to do symbolic calculations in Julia.  Hopefully, in spite of my problems with the Jones to Mueller conversion, you see the process to simulate the systems response with any light that can be represented with Stokes parameters but analyzed with the Jones or P matrix formulation. Even if I did it badly, this shows that the Zhang statement is not correct.
I'm going to move on to learning OpticsSim and seeing if I can make some contributions.

Dear @BrianGun

I created a new branch (#276) for this issue and have started modifying the code to add polarization information to OpticalRay. In the new system you create an optical ray with polarization = nothing, or to a subtype of AbstractPolarization. Currently the only subtype is Chipman defined as an empty stub. As soon as I get the current code working with polarization = nothing then I'll start on the Chipman polarization tracing.
It doesn't look like the polarization tracing will take long to code, unless I'm misreading the book. Once I finish could one or both of you write tests to make sure the code works?
It will simplify life if you make all your modifications to this branch.

I tried to check out this branch, but I do not see any commit except that the branch exist. Is there any problem on my side (between kbd-chair), or do you have your commit with AbstractPolarization still to be pushed?
Thx.

Dear @mattderstine

[jones matrix example.pdf]

thank you very much for your quick response and for elaboration of the example. Please let me recap to make sure our understanding is the same. In Chipman's P-matrix formalism the 2D Jones 2x2 complex matrix is extended to 3D yielding complex 3x3 matrix, representing Jones action in WCS. During the propagation of the ray through N interfaces, at each (n-1,n), the matrix P (and not the 3D Jones vector E) is kept as a property of ray. At the detector, the ray's P matrix is a product of elemental P matrices of each interface intersection. How does a polarization of the source evolve to a polarization at detector is then solved by applying the P matrix, likely from-to respective pupil local coordinate systems (LCS). Is my understanding correct?
I have to repeat, that while I like the idea of P matrix as an end-to-end polarization transfer function in WCS, I still think that raytracing polarization this way is very impractical in OpticSim for the following reasons:

doubling/tripling of rays. During the propagation, the ray does not know its polarization. For this reason, it is necessary to cover all doubling (r & t) and tripling (r & t_o & t_e) of rays -- exactly as Chipman discusses. The problem is that there can be even infinite amount of these ray forks, which in deterministic case is hard to handle -- may be reduced by a heuristic to avoid cycles, but then it is less realistic. (Simple example: plan-parallel plate nearly perpendicular to the ray; or: a lens with long focus.) What I find especially suboptimal is that even in case that there will be zero power in specific ray fork, it is not known until the very end so the (zero-power) ray still consumes the resources.
Memory (and possibly also computing speed): the P-matrix is 3x3xComplex (18 numbers). On the other hand, 3D Jones vector is 3xComplex, enhanced with partial polarization adds one number (DoP, degree of polarization), therefore in total 7 numbers (degrees-of-freedom). I find this a bit of waste, especially as most users will be happy with as much rays as possible.
Compatibility with current OpticSim.jl: prior to starting this Issue thread, I looked into the code and I was actually quite happy with the Monte Carlo implementation of reflect/transmit forks, which maintains constant number of rays, only changes their pseudorandom distribution in a way they will cover more powerful beams with more rays and near zero power rays with nearly no power -- which fits the purpose very well.

Below, I would like to make a proposal how to handle the SoP of each ray. But prior to that, let me say:

I believe the Chimpan's P-matrix is a very nice idea and we can introduce it at least during development as a test matrix, which shall be able to prove if any other representation of SoP works well. The only thing which I would not like to do is to implement ray forking and explosion in number of rays.
I have finally looked in to Zhang&al and I share your opinion the added value is not much. I do not think the formalism is especially suited for us.

Let me now propose how to handle the polarization. First, let me express what components related to polarization, power, phase and direction we need to track:

theta orient (plane of polarization/ellipsoid major axis w.r.t. ray's LCS)
epsilon ellipticity
DoP degree of polarization (0..1)
phi phase
pow intensity (power)
direction or vector k (unit vector, i.e. 2 degrees-of-freedom)

The well-known formalisms do handle the following (sub)sets:

Jones has pow, phi, theta, epsilon, lacking DoP

Chipman's P matrix/E vector adds 2 DoF: direction

Poincare sphere covers theta, epsilon, DoP
Stokes vector covers Poincare sphere plus pow

In the framework I'd like to propose covers all 7 DoFs, i.e. the same as Chimpan's vector E plus DoP. (Please note that I do want to propagate the vector E and not matrix P, in order to be able to evaluate instantaneous ray's power and therefore Monte Carlo redistribution of rays. And to save some memory at the same time.)
Now, what exact representation of the above? Although I like a lot the idea of common WCS for all quantities without changing to LCS of each frame, after some doodling on a paper it seems to me that simplest and still very efficient is actually to switch to ray's LCS at each interface. In order to enable this, we need an un-ambiguous definition of LCS based solely on beam's direction:
k1 = normalize([k3[2]+k3[3], -k3[1]+k3[3], -k3[1]-k3[2]])
k2 = cross(k1, k3)
T = [k1 k2 k3]

This gives us an ortho coord transformation matrix T to go from-to the LCS of a ray at the interface. The T is thus fully defined by current OpticSim's ray property (direction). Then, the new elements to add are: theta, epsilon (perhaps find a better name not resembling eps()), DoP which constitutes the SoP, and phi (or phase) for phase. pow also already exists.
In the LCS, we can start to process the most basic existing interfaces such as Fresnel. I propose to add -- as a means of debug and development helper -- the Chipman's P-matrix in order to validate the results, to one of the AbstractPolarizations. For "production" I'd however advice to put it aside.
Memory-wise the current ray will need to carry 4 more Float values.
FYI: @BrianGun

@hrobotron @mattderstine
A Monte Carlo approach should handle the problem of ray splitting in birefringent materials without causing an explosion of rays. We currently do this to handle reflection/transmission and it seems effective.
A possible advantage of using the P matrix instead of the E vector is that one can compute the response for any type of input polarization without having to rerun the raytrace. This would only work for a single source, i.e., you'd need to compute one P matrix image for each source.
I still need to better understand how P matrices are added, as opposed to being composed by multiplication, to verify that this is true. If it is then you could do a ray trace, propagate the P matrices through the system and store the resulting P matrix at each pixel. Then using that P matrix image you could generate results for aribtrary input polarizations. This would be extremely fast even for large images.
If you have multiple sources then you'd need to create multiple P matrix images. Unless your images are very large or you have many sources it would still be faster to compute the output for a different polarization this way than to redo the raytrace.
Whereas it seems that if you propagate the E vector then your result is only useful for that polarization. If you change the polarization of the source then you have to rerun the entire raytrace.
The extra overhead from computing and propagating the P matrices should be small compared to everything else being done during the raytrace.
Don't know why the branch doesn't have the latest changes I've made locally. I'll look at it today.
Give me a few more days to add the P matrix computations before you start munging on the code so we don't get too out of sync. Then you'll have something to build on and create tests against.

Dear @hrobotron,
I have spent too much time on this already and I'm afraid this is going to be a bit blunt.
Forking at e and o is only necessary for birefringent material where the power actually is split. That said, for many problems, there is no other solution than to fork. It doesn't matter about the formalism used for tracking the polarization. I agree that if your problem only needs one of the forks, it would be nice if the code didn't do the extra calculations. It is more important that the code does the needed calculation and it is correct and understandable. (and as a side note, I am not convinced with the approach taken with the stochastic method used with r & t but that's for another thread.).  As a justification for forking, I will point to the problem of analyzing the impact of stress birefringence on the "PSF" of an objective lens. This cannot be analyzed unless the e and o rays are traced and recombined to understand the impact. This could be on two ray traces (one for e and one for o) or it could have forking. With a 30 element objective, I'd rather have the code fork for me for the 1000 rays that I need than have to set up 2^29 cases.
Assuming a SoP before the ray is traced has marginal benefit for most problems. It is more useful to understand what happens with variations to the source SoP when assessing the value of design. Requiring separate ray traces for each SoP is very wasteful in this instance.
For most applications, the intermediate P and Q matrices do not need to be saved. This is no different than only saving the ray data at a detector. This means the memory differences of one approach or that other are pretty much unimportant. If enough rays are to be traced that it does matter, the computation to the interesting quantities can be done at the end of each raytrace, for example at a polarization detector.  For cases where a few rays are traced to see the evolution, having the full polarization state is better (see last paragraph).
In spite of the difficulty in getting it correct, the polarization raytrace is not the hardest part of the problem. It the most critical and must be absolutely correct and it must provide the capability for all of the analysis that a user needs. Creating and perfecting the code to actually analyze the results is much harder than doing the raytrace. Having the complete information about the polarization properties of the system in one consistent representation makes this implementation much easier and facilitates breaking problems up so that multiple people can address them.
The approach is Chipman is complete, well documented, and addresses nearly all possible problems. It is in book form so that many people can refer to it, study it, and have a common understanding of how all the parts work. While it all may not be computationally optimal, it is correct and users can spend their time understanding their optics problem rather than trying to understand if the issue they have is from a quirk in the code.
I told @BrianGun that I would work on the examples and problems from the book. This is actually a huge undertaking since it will require developing or testing numerous analysis routines. The only way I can possibly accomplish this is to have a reference that at least gives me hints on how to approach these routines. I will simply be unable to do this if we choose some subset of this solution. The key output of this effort is actually not the testing of the code, but the development of the analysis routines that others can use for similar problems.
OpticsSim has the potential to have a huge impact on the way optical design is done.  We should be focusing on capabilities and rigor rather than on optimizing for Monte Carlo simulations. Basing code on well explained methods will only make it accessible to more users but also easier for those users to add and share capabilities.  CodeV and Zemax suffer from having polarization grafted on. The notebook interface and the structure of OpticSim are ideally suited for solving problems that are difficult or impossible with those codes. Let's push to get that capability in users hands rather than getting bogged down in understanding if the code actually works for their applications.
I can see that in version 2 or 3 there may be a need for computational optimization. Let's wait until then to know what the users need and will provide benefit before we start pruning our options.

@mattderstine @hrobotron
I appreciate the time and care you are both putting into this. Your comments and analysis have helped me to better understand how to incorporate polarization into OpticSim.
It seems to me now that the total amount of code that will need to be rewritten/modified is small, on the order of at most a hundred lines, probably far less. It's tricky primarily because we are doing surgery on an existing system and have to be careful to make sure all the appropriate dependent functions are updated properly when we make a change.
Because the code changes are small and localized I'm less concerned with trying Chipman first and then switching to something entirely new later if Chipman proves insufficient. An initial implementation using Chipman's formulation will be a good way to start because it is well documented, relatively easy to understand, and easy to verify against the worked examples in the book. It won't be that big a deal to switch to something entirely different if we decide to change later.
Modifying the code to incorporate P matrices will be much easier for me to do than for either of you because of my familiarity with the code. I hope to have the P matrix propagation done soon after which I can turn it over to you all to experiment with and verify. Once we learn the limitations/power of the result we can turn the crank again and make necessary improvements or extensions.

This is great. Thank you!  I'm going to devote my time to learning how the rest of the code works until you are ready with something to test.

@mattderstine @hrobotron
OpticSim currently doesn't support complex index of refraction. It's easy enough to change the processintersection function of resnel to allow complex arguments for index but this doesn't address the issue of how users specify and attach these to interfaces.
Do either of you have suggestions about how to incorporate this? What are good references for standard complex indices of refraction? Will we have to change GlassCat and the associated functions to support Complex instead of Real index of refraction?
If we need to modify GlassCat then this will be a bigger change. We will probably want to get the P matrix working first then address GlassCat.

@mattderstine @hrobotron
I don't know enough about the current code to suggest something that fits within the design approach. However, this might give you some ideas.
The surface interaction is not just a function of the refractive index. If the surface has a coating or pattern, then additional information is needed to compute r, t, P etc. What would it be like to have an AbstractClass for different types of surface properties that are then used by methods to calculate r,t, P etc. This would avoid performance penalties for simple stuff but enable individuals to write methods for the weirdest of situations and still use all of the existing code.
Then with this in place, write methods for fresnel calculations with real and complex refractive index.  The ones for real could use GlassCat as default and the complex relative indices could be just stored in the class.
I use refractiveindex.info for this kind of info.  For starters some helper functions to load the data from the website would be nice, but not necessary.
I wouldn't mess with GlassCat right now. The current setup doesn't work properly for new glasses unless you are running from a repo because sources.txt is write protected if you just load OpticSim as a package. (and I really don't want my loaded packages to be writable).  I prefer that data from refrativeindex.info or whereever is stored in my project rather than in the package and so this becomes an even bigger change to GlassCat.  I'll follow this up with a separate enhancement request and so let's separate it from polarization.

Dear @mattderstine
I'll comment just a subset in order to clarify where I feel you perceive something different than what I had in mind (which is most likely due to my language/expression).

Forking at e and o is only necessary for birefringent material where the power actually is split. That said, for many problems, there is no other solution than to fork. It doesn't matter about the formalism used for tracking the polarization.

It does matter in Monte Carlo in the sense, that in a beam with 0.1% of total power there will be proportionally lower number of rays than in a beam having 30% of total power, thus rendering the more powerfull beam with better quality, while still providing some insight into the other one, useful e.g. to assess ghosting.

I agree that if your problem only needs one of the forks, (..)

For some unknown reason you think that I did not want to care about the forks. Why? Since the first post I am talking about them (in a positive way). I only want to handle them the same way as current polarization-unaware OpticSim.jl does, i.e. via Monte Carlo. And for this I wish to have a decent value of pow ray property, in order to split correctly. It is sufficient to have one or more (non-ideal) polarizers in the system and it shall become evident why. I do not want to build full fork tree for each ray, because this gives potentially unlimited number of rays, not knowing which of them do have near zero power. Also, it means a huge impact on OpticSim code.

[..] (and as a side note, I am not convinced with the approach taken with the stochastic method used with r & t but that's for another thread.)

OK, this is probably the point where we diverge.

Assuming a SoP before the ray is traced has marginal benefit for most problems.

I can not generally agree. I believe there are application which benefit from the possibility to study SoP impact, and there are others (real life ones), which are perfectly fine e.g. with fully unpolarized light or fully linear polarized light (such as laser setup) at the input.

[..] Requiring separate ray traces for each SoP is very wasteful in this instance.

I agree.

For most applications, the intermediate P and Q matrices do not need to be saved. [..]

I have to admit that I have not yet looked into the OpticSim enough to know what visualisations are in place and how much the intermediate results are or are not examined. So yes, maybe you are right and the memory is a no-issue.

The approach is Chipman is complete, well documented, and addresses nearly all possible problems. It is in book form so that many people can refer to it, study it, and have a common understanding of how all the parts work. While it all may not be computationally optimal, it is correct and users can spend their time understanding their optics problem rather than trying to understand if the issue they have is from a quirk in the code.

I believe so. My only problem not addressed by Chimpan remains: Monte Carlo + pow at each interface. I'd like to recall that I also appreciate Chipman as well proven method with nice solved examples in the book, including numeric values in the matrices (!). For this reason despite my bias to propagate SoP instead of P-matrix, I'd wish to include the P-matrix as a verification option to propagate along. (But which ray wins the Monte Carlo Darwinism, will be controlled by the SoP and pow which it yields.)

I told @BrianGun that I would work on the examples and problems from the book. This is actually a huge undertaking since it will require developing or testing numerous analysis routines. The only way I can possibly accomplish this is to have a reference that at least gives me hints on how to approach these routines. I will simply be unable to do this if we choose some subset of this solution. The key output of this effort is actually not the testing of the code, but the development of the analysis routines that others can use for similar problems.

Well, I thought that in my previous post and what I repeat above should be the best for both of us. I will give you the P-matrix and meanwhile use the SoP to choose which ray to survive. Are you happy with that? For me this is win-win. Easiest modification of OpticSim (where with deterministic forking I am afraid is highly non-trivial, and I believe also not optimal), existence of P along the path (as an option).

OpticsSim has the potential to have a huge impact on the way optical design is done.

I have the same feeling. I do all engineering tasks in Julia and certainly the world of optics may strongly benefit from Julia's advantages.

We should be focusing on capabilities and rigor rather than on optimizing for Monte Carlo simulations.

I am trying to find an easy way, which still has a potential to be correct. This is not yet optimizing. This is how to survive and do not mess up totally the code which then no one can put together. My main points, repeated (especially FYI @BrianGun):

unknown pow means incorrect probability of ray splitting and a high likelihood that "important" beams will have few rays and vice versa
deterministic splitting will probably cause huge redesign (and at the same time I am not convinced we wish deterministic splitting)

Basing code on well explained methods will only make it accessible to more users but also easier for those users to add and share capabilities. CodeV and Zemax suffer from having polarization grafted on. The notebook interface and the structure of OpticSim are ideally suited for solving problems that are difficult or impossible with those codes. Let's push to get that capability in users hands rather than getting bogged down in understanding if the code actually works for their applications.

I can only agree.

I can see that in version 2 or 3 there may be a need for computational optimization. Let's wait until then to know what the users need and will provide benefit before we start pruning our options.

Dtto.
BTW, thank you again for the excellent book and for sharing your insight.

Dear @mattderstine

Because the code changes are small and localized I'm less concerned with trying Chipman first and then switching to something entirely new later if Chipman proves insufficient. An initial implementation using Chipman's formulation will be a good way to start because it is well documented, relatively easy to understand, and easy to verify against the worked examples in the book. It won't be that big a deal to switch to something entirely different if we decide to change later.
Modifying the code to incorporate P matrices will be much easier for me to do than for either of you because of my familiarity with the code. I hope to have the P matrix propagation done soon after which I can turn it over to you all to experiment with and verify. Once we learn the limitations/power of the result we can turn the crank again and make necessary improvements or extensions.

For me this is quite OK, assuming you will not substantially change the way how the ray behaves at reflection / transmission. Once you push your commit with this AbstractPolarization with one subclass being Chipman, I'd be happy to test and to try to subclass and show what I had in mind above. And we may compare how it behaves.

@mattderstine @hrobotron
OpticSim currently doesn't support complex index of refraction. It's easy enough to change the processintersection function of resnel to allow complex arguments for index but this doesn't address the issue of how users specify and attach these to interfaces.
Do either of you have suggestions about how to incorporate this? What are good references for standard complex indices of refraction? Will we have to change GlassCat and the associated functions to support Complex instead of Real index of refraction?
If we need to modify GlassCat then this will be a bigger change. We will probably want to get the P matrix working first then address GlassCat.

We may also need tensor indices, not only complex. I'd recommend for the start NOT to add new material types, since we can happily test lots of polarization effects just using tilted homogenous glasses with real refractive index. Later, yes indeed.
Also I believe we shall introduce some ideal components, like ideal Jones 2D/3D (having the ability to implement perfect waveplates, polarizers). We shall be clever here, as there shall be probably the same logic as with implementing ideal amplitude/phase plates (unsure is this element already exists, have to check the code). In fact an arbitrary "pupil funcion", which manipulates ray depending on u,v coords.

@hrobotron
Please give an example where calculating the SoP for each surface during the raytrace has an advantage rather than after the raytrace has been done. I cannot think of any. Choosing the branch of a stochastic fork is not enough reason, as this could be done with the P matrices just as easily.
Next, there needs to be two (or more) types of ray traces to address the issues of interferometer, scattering, polarization etc. I agree that a deterministic single ray through the system is required. To do birefringent materials properly it is a requirement that the ray trace fork or at least appear to fork so that the different branches of rays can be combined after the raytrace in ways that include phase and path difference. Thus the single ray is the place to start (ie. Chipman Chapter 10) and leave the forking and birefringent materials for some later time (Chapters 19 & 20) when some of the analysis infrastructure is finished.
I will create an issue on why stochastically choosing the fork does not seem to provide any value. Let's move that discussion to a different place.

Hello,

@hrobotron
Please give an example where calculating the SoP for each surface during the raytrace has an advantage rather than after the raytrace has been done. I cannot think of any.

ray encounters a tilted glass plate. Depending of ray's SoP, the power will be distributed into s / p polarizations unequally. For some SoP, even one of those will be zero. For other SoP it may in contrast become the main working beam of the system. (Believe it or not, we have some interferometer where this is the case, a reference surface being the glass/air uncoated interface.)
Based on the result, one or two resulting rays will be populated with stochastic ensemble, and number of rays in each ensemble will be proportional to the power -- determined by the SoP.

Choosing the branch of a stochastic fork is not enough reason, as this could be done with the P matrices just as easily.

How?

Next, there needs to be two (or more) types of ray traces to address the issues of interferometer, scattering, polarization etc. I agree that a deterministic single ray through the system is required. To do birefringent materials properly it is a requirement that the ray trace fork or at least appear to fork so that the different branches of rays can be combined after the raytrace in ways that include phase and path difference. Thus the single ray is the place to start (ie. Chipman Chapter 10) and leave the forking and birefringent materials for some later time (Chapters 19 & 20) when some of the analysis infrastructure is finished.

Just FYI: interferometers are my main subject I wish to later solve with OpticSim, not the lens assemblies, so I can just assure you that I understand the need. I however do not understand why do you think it is impossible to mix rays resulting from Monte Carlo without keeping track which of these are siblings. I think it is fairly well possible. Imagine a set of rays, which fork many times, some of them taking different paths, and at the end all of them meet at detector pupil. Why do you think it is now impossible to build an intereference pattern? It is just a linear superposition of the fields, which are here approximated by spatial samples -- the intersection of rays and the detector.

I will create an issue on why stochastically choosing the fork does not seem to provide any value. Let's move that discussion to a different place.

OK, I will try to follow.

I'm embarrassed that I couldn't get my Jones to Mueller example to work. After some research, I found the problem was in definition of functions that got scrambled from the original work to Chipman's example and then in choosing Mathematica's functions.
Here is a working version in Julia with reference to the algorithm used.
`
using LinearAlgebra
ϕs = 0 #phase shifts at the polarizer
ϕp = 0.1
ts = sqrt(0.8) #transmission of the s
tp = sqrt(0.1)  #not a great polarizer
ϕwp = π/2 #QWP
tss = sqrt(0.96) #normal fresnel loses at the final interface
tps = sqrt(0.96)
θ = π/4
jonespol = [ts * exp(im * ϕs) 0.; 0. tp * exp(im * ϕp)]
r(θ) = [cos(θ) -sin(θ); sin(θ) cos(θ) ]
joneswp = [exp(-im * ϕwp/2) 0 ; 0 exp(im * ϕwp/2)]
jonessurf = [ tss 0 ; 0 tps]
jonessys = jonessurf * r(-θ) * joneswp * r(θ) * jonespol
function jonesToMueller(jones)
"""
jonesToMueller(jonesMatrix)
Compute the Mueller matrix that corresponds to the jonesMatrix

Uses the method from the derivation of Appendix 4 of
Spectroscopic Ellipsometry: Principles and Applications
H. Fujiwara © 2007 John Wiley & Sons, Ltd. ISBN: 978-0-470-01608-4

Wiley has made this appendix available as:
https://onlinelibrary.wiley.com/doi/pdf/10.1002/9780470060193.app4

"""
u = 1/sqrt(2.) * [1 0 0 1; 1 0 0 -1; 0 1 1 0; 0 im -im 0]
m = real.(u * kron(jones, conj.(jones)) * adjoint(u))

end
sop = jonesToMueller(jonessys)* [1 ,0 ,0 ,0] #see how it modifies unpolarized light
`

@mattderstine @hrobotron
There is a somewhat working version of Chipman's Pmatrix algorithm on this branch. So far I've tested it against example 9.4 and the results match. There are undoubtedly errors, both programming and conceptual, and perhaps significant code refactoring left to be done but it seems to be in the right direction. The code changes were fairly minor.
You should be able to start testing against other worked examples to find the holes and bugs in the implementation. As you write these tests work please add them to the test set for Polarization.jl in the test directory, which doesn't exist yet.
Haven't added the example 9.4 as a test yet. Will do it soon.
This version will probably have terrible performance because none of the code is optimized and tracing polarization allocates. We carefully optimized the code for tracing existing primitives so they do not allocate anything. To get acceptable multithreaded performance for polarization we will have to do the same for it. Let's get it correct and complete, at least as complete as we have energy for, then worry about optimization.

@BrianGun
Thanks!  I'm new to Github as well as OpticSim so it may be slow going until I have something to report.  I'm going to assume that to use odd and even values to specify an asphere, I will have to add a new surface type. My intermediate goal is to replicate Chipman Fig 10.34

@BrianGun,
Thanks again. I'm just getting started and I'm able to run your example. I, however, am not able to run other simple examples like draw_cooketriplet()
I get the following error. I assume it's related to the changes that you made to fresnel.jl. I'm not going to look for a fix as I expect it's just a trivial oversight. Is there a better way for me to pass on bugs like this?

ERROR: LoadError: MethodError: no method matching mᵢandmₜ(::GlassID, ::GlassID, ::SVector{3, Float64}, ::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}})
Closest candidates are:
mᵢandmₜ(::Any, ::Any, ::SVector{N, T}, ::SVector{N, T}) where {T<:Real, N} at /Users/matt/Development/GitHub/OpticSim.jl/src/Optical/Fresnel.jl:5
Stacktrace:
[1] processintersection(opticalinterface::FresnelInterface{Float64}, point::SVector{3, Float64}, normal::SVector{3, Float64}, incidentray::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}}, temperature::Float64, pressure::Float64, test::Bool, firstray::Bool)
@ OpticSim ~/Development/GitHub/OpticSim.jl/src/Optical/Fresnel.jl:165
[2] trace(assembly::LensAssembly, r::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}}, temperature::Float64, pressure::Float64; trackrays::Vector{LensTrace{Float64, 3}}, test::Bool, recursion::Int64)
@ OpticSim ~/Development/GitHub/OpticSim.jl/src/Optical/LensAssembly.jl:344

@mattderstine
yeah, sorry about that, should have tested more thoroughly before having you use the code.
Last night I pushed a bunch of fixes for a variety of these errors that got introduced when I refactored the processintersection code. Most stuff works now, but trace fails on some examples. Try again and see if it works better now.

@BrianGun,
Thanks again. I'm just getting started and I'm able to run your example. I, however, am not able to run other simple examples like draw_cooketriplet()
I get the following error. I assume it's related to the changes that you made to fresnel.jl. I'm not going to look for a fix as I expect it's just a trivial oversight. Is there a better way for me to pass on bugs like this?

ERROR: LoadError: MethodError: no method matching mᵢandmₜ(::GlassID, ::GlassID, ::SVector{3, Float64}, ::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}})
Closest candidates are:
mᵢandmₜ(::Any, ::Any, ::SVector{N, T}, ::SVector{N, T}) where {T<:Real, N} at /Users/matt/Development/GitHub/OpticSim.jl/src/Optical/Fresnel.jl:5
Stacktrace:
[1] processintersection(opticalinterface::FresnelInterface{Float64}, point::SVector{3, Float64}, normal::SVector{3, Float64}, incidentray::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}}, temperature::Float64, pressure::Float64, test::Bool, firstray::Bool)
@ OpticSim ~/Development/GitHub/OpticSim.jl/src/Optical/Fresnel.jl:165
[2] trace(assembly::LensAssembly, r::OpticalRay{Float64, 3, OpticSim.Polarization.NoPolarization{Float64}}, temperature::Float64, pressure::Float64; trackrays::Vector{LensTrace{Float64, 3}}, test::Bool, recursion::Int64)
@ OpticSim ~/Development/GitHub/OpticSim.jl/src/Optical/LensAssembly.jl:344


VSCode and tsc very slow when using external tsplusTypes

Initializing language features in editor, as well as compiling, takes very very long on my projects since using external tsPlusTypes definitions.
I mean I do have a lot of errors atm as im converting, but the process is very painful with such slow load times

Additionally I noticed the code that scans for the tsplusTypes is called multiple times for me during build for example. Not sure if it maybe needs caching. Will see if I can find out why

Perhaps also related: https://github.com/ts-plus/typescript/issues/248
Using these https://github.com/tim-smart/effect-rpc/tree/main/packages/router/vendor by @tim-smart
coming out of https://github.com/tim-smart/tsplus-gen

I've fixed the performance issue with 3 caches, but the last one seems to have made the most difference.
(initialize caches outside the Parser definition)
const resolvedModuleCache = {}
const resolvedModule = key in resolvedModuleCache ? resolvedModuleCache[key] : (resolvedModuleCache[key] = resolveModuleName(moduleName, options.configFilePath, options, sys).resolvedModule);

The diff to my patch is here: https://github.com/effect-ts-app/libs/pull/21/files#diff-2d8242e8fed0e717f1e83478a154f8c6401df0ae6efd8e5e6b4103cadf318c0f

@0x706b this seems to work well now, thanks!


LiveLink is sending no data to UE5.1.1

Livelink in Unreal see correctly the server but won't receive any data from the livelink even the EditorActiveCamera

And the icon is turning red only when I move the object in maya but it's  not updating inside UE


[WIP] NO_ISSUE: Copying resources directly to target instead to src/main/resources to keep module sources clean.

This has been a bad config we've been spreading around for some time, with this change we copy static resources directly to target, so they'll be bundled in the .jar but keeping the sources clean.
Ensemble:
https://github.com/apache/incubator-kie-kogito-runtimes/pull/3691
https://github.com/apache/incubator-kie-kogito-apps/pull/2106

Many thanks for submitting your Pull Request :heart:!

Closes/Fixes/Resolves #ISSUE-NUMBER

Description:

Please make sure that your PR meets the following requirements:

[ ] You have read the contributors guide
[ ] Your code is properly formatted according to this configuration
[ ] Pull Request title is properly formatted: Issue-XYZ Subject
[ ] Pull Request title contains the target branch if not targeting main: [0.9.x] Issue-XYZ Subject
[ ] Pull Request contains link to the JIRA issue
[ ] Pull Request contains link to any dependent or related Pull Request
[ ] Pull Request contains description of the issue
[ ] Pull Request does not include fixes for issues other than the main ticket

How to replicate CI configuration locally?

Build Chain tool does "simple" maven build(s), the builds are just Maven commands, but because the repositories relates and depends on each other and any change in API or class method could affect several of those repositories there is a need to use build-chain tool to handle cross repository builds and be sure that we always use latest version of the code for each repository.
build-chain tool is a build tool which can be used on command line locally or in Github Actions workflow(s), in case you need to change multiple repositories and send multiple dependent pull requests related with a change you can easily reproduce the same build by executing it on Github hosted environment or locally in your development environment. See local execution details to get more information about it.

@pefernan
Thanks for the PR, I really appreciate.
Here and in the other one I found minor details to be fixed, IINW.
Anyway, reading it, I just realized that maybe a much simpler way, instead of copy files here and there, to simply override the resources configuration as in
<resources>
                <resource>
                  <directory>${project.parent.basedir}/integration-tests-data-index-service-common/src/main/resources</directory>
                  ```


Add a function to return all the keys of the entries that hold the same value.

Hi!
I just added a new function to the API which searches for a given value in the whole map and returns all the keys that holds that value.
// get all enteries that have the same values returned as an array of keys // it actually iterates the whole map as hashmap_iterate does. void ** hashmap_get_same_values(hashmap * m, uintptr_t value);

I understand.
I can't figure out what is the resulting keys ahead of time because that's the point the function i added should loop through the map to know, and for the key length you are right! , i didn't manage that.
I will consider making a callback function as well.
And to be honest i thiugh about using hashmap_iterate in my function but it dodn't seem to help because even the call back function provides no way to return an array of keys because the callback function is inernal in hashmap_iterate so no outout from the function itself that's why i rewrote it again.
I will try to improve my solution as you noted and you can decide to merge or not later.
Thank you for the review!

I understand.
I can't figure out what is the resulting keys ahead of time because that's the point the function i added should loop through the map to know, and for the key length you are right! , i didn't manage that.
I will consider making a callback function as well.
And to be honest i thiugh about using hashmap_iterate in my function but it dodn't seem to help because even the call back function provides no way to return an array of keys because the callback function is inernal in hashmap_iterate so no outout from the function itself that's why i rewrote it again.
I will try to improve my solution as you noted and you can decide to merge or not later.
Thank you for the review!

Yeah, you definitely cannot know the number of matches ahead of time, I just worded that poorly. I was trying to say that the user would have to know the number of matches ahead of time in order for the function to be useful in its current state.
< And to be honest i thiugh about using hashmap_iterate in my function but it dodn't seem to help because even the call back function provides no way to return an array of keys because the callback function is inernal in hashmap_iterate so no outout from the function itself that's why i rewrote it again.
I also wasn't trying to say that you have to use hashmap_iterate() in the function you are trying to make, but that it could be used as an alternative to the function you wrote. Also, you can get output from hashmap_iterate() by using the user pointer:
struct iter_input
{
	// the value we are searching for
	uintptr_t search_value;
	int count;		// the size of the array
	int capacity;	// amount of space allocated in the array
	void **key_arr; // an array of keys
};

void iter_callback(void *key, size_t ksize, uintptr_t value, void *usr)
{
	struct iter_input *input = (struct iter_input *)usr;

	if (value != input->search_value)
		return;

	++input->count;

	if (input->count > input->capacity)
	{
		input->capacity *= 2;
		input->key_arr = realloc(input->key_arr, sizeof(void *) * input->capacity);
	}

	input->key_arr[input->count - 1] = key;
}

/*
...
*/

int main(int argc, const char* argv[])
{

	hashmap* m = hashmap_create();

	// matching keys
	hashmap_set(m, hashmap_str_lit("hello"), 400);
	hashmap_set(m, hashmap_str_lit("world"), 400);
	hashmap_set(m, hashmap_str_lit("foo"), 400);
	hashmap_set(m, hashmap_str_lit("bar"), 400);
	// non-matching keys
	hashmap_set(m, hashmap_str_lit("oof"), 600);
	hashmap_set(m, hashmap_str_lit("baz"), 500);

	// search for the value `400`, set count to initial value of `0`, allocate initial space of 1 array item
	struct iter_input input = {400, 0, 1, malloc(sizeof(void*))};

        // use the user pointer to get output from `hashmap_iterate()`
	hashmap_iterate(m, iter_callback, &input);

	// print out the resulting array items
	for (int i = 0; i < input.count; ++i)
	{
		printf("key %i: %s\n", i, input.key_arr[i]);
	}

	// free the array now that we're done with it:
	free(input.key_arr);
}

This should print the following result:
key 0: hello
key 1: world
key 2: foo
key 3: bar

Obviously this is a bit clunky, and if you wanted to add this to the library itself you you would have to define a new struct and give the user access to it. This also doesn't solve the key length problem. This is why I think it should be up to the user to implement their own version of this to suit their individual needs, since adding this feature would require adding a new type to map.h and it would take a decent amount of documentation to properly explain how to use it.
If you can think of a better solution I would be happy to look at it though :)

Yeah, you definitely cannot know the number of matches ahead of time, I just worded that poorly. I was trying to say that the user would have to know the number of matches ahead of time in order for the function to be useful in its current state.
< And to be honest i thiugh about using hashmap_iterate in my function but it dodn't seem to help because even the call back function provides no way to return an array of keys because the callback function is inernal in hashmap_iterate so no outout from the function itself that's why i rewrote it again.
I also wasn't trying to say that you have to use hashmap_iterate() in the function you are trying to make, but that it could be used as an alternative to the function you wrote. Also, you can get output from hashmap_iterate() by using the user pointer:
struct iter_input
{
	// the value we are searching for
	uintptr_t search_value;
	int count;		// the size of the array
	int capacity;	// amount of space allocated in the array
	void **key_arr; // an array of keys
};

void iter_callback(void *key, size_t ksize, uintptr_t value, void *usr)
{
	struct iter_input *input = (struct iter_input *)usr;

	if (value != input->search_value)
		return;

	++input->count;

	if (input->count > input->capacity)
	{
		input->capacity *= 2;
		input->key_arr = realloc(input->key_arr, sizeof(void *) * input->capacity);
	}

	input->key_arr[input->count - 1] = key;
}

/*
...
*/

int main(int argc, const char* argv[])
{

	hashmap* m = hashmap_create();

	// matching keys
	hashmap_set(m, hashmap_str_lit("hello"), 400);
	hashmap_set(m, hashmap_str_lit("world"), 400);
	hashmap_set(m, hashmap_str_lit("foo"), 400);
	hashmap_set(m, hashmap_str_lit("bar"), 400);
	// non-matching keys
	hashmap_set(m, hashmap_str_lit("oof"), 600);
	hashmap_set(m, hashmap_str_lit("baz"), 500);

	// search for the value `400`, set count to initial value of `0`, allocate initial space of 1 array item
	struct iter_input input = {400, 0, 1, malloc(sizeof(void*))};

        // use the user pointer to get output from `hashmap_iterate()`
	hashmap_iterate(m, iter_callback, &input);

	// print out the resulting array items
	for (int i = 0; i < input.count; ++i)
	{
		printf("key %i: %s\n", i, input.key_arr[i]);
	}

	// free the array now that we're done with it:
	free(input.key_arr);
}

This should print the following result:
key 0: hello
key 1: world
key 2: foo
key 3: bar

Obviously this is a bit clunky, and if you wanted to add this to the library itself you you would have to define a new struct and give the user access to it. This also doesn't solve the key length problem. This is why I think it should be up to the user to implement their own version of this to suit their individual needs, since adding this feature would require adding a new type to map.h and it would take a decent amount of documentation to properly explain how to use it.
If you can think of a better solution I would be happy to look at it though :)

I understand.
You are right about how to use hashmap_iterate callbacks to fulfill the needs of the function that i want to add. I didn't know it is possible but you gave a great example though! also i though adding functions like mine would really make the library huge but it will make a good rich API that is so easy to use just by function calls.

I understand.
You are right about how to use hashmap_iterate callbacks to fulfill the needs of the function that i want to add. I didn't know it is possible but you gave a great example though! also i though adding functions like mine would really make the library huge but it will make a good rich API that is so easy to use just by function calls.


touchend will trigger twice and the tap event in vue-touch is the same

when I bind touchend instead of click in the elements, it will trigger twice.
And the vue-touch with hammer.js is the same situation.
How can I fixed it?
I don't want to bind click because of its 300 millisecond delay.

@Hedgehogking I tried to reproduce it with no luck (Vue 1 and 2): https://jsfiddle.net/posva/y90Loons/1/
If the issue is with vue-touch you should open an issue there
Aren't you binding click and touchend?

I was in some special situation such as :https://jsfiddle.net/4e8skt3h/
When I click to next box, the next box's items will show. Then the mouse will click on the item which is just right under the mouse too.
My English is poor! Forgive me.😂
原始邮件
发件人:Eduardo San Martin<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>发送时间:2016年8月10日(周三) 23:04
主题:Re: [vuejs/vue] touchend will trigger twice and the tap event invue-touch is the same (#3422)
@Hedgehogking I tried to reproduce it with no luck (Vue 1 and 2): https://jsfiddle.net/posva/y90Loons/1/
If the issue is with vue-touch you should open an issue there
Aren't you binding click and touchend?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

You may be looking for event.stopPropagation 😄

It’s useless… I’m crazy.
http://hedgehogking.com/works/jiuqu/#!/evaluateIt may be slowly...
The event trigger twice at the same position
And the normal.⬇
Because the label or input?
原始邮件
发件人:Eduardo San Martin<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>发送时间:2016年8月11日(周四) 03:44
主题:Re: [vuejs/vue] touchend will trigger twice and the tap event invue-touch is the same (#3422)
You may be looking for event.stopPropagation 😄
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@Hedgehogking You should create a thread at forums. People are more likely to look at an question on the forums that on a closed issue 😉

OK. Thanks.


Upstream PRs  #854 #852 #857 #858 #860 #845 #862 #875 #878 #874 #877 #880 #864 #882 #894 #891 #901

[upstream PR #854]: Rename msg32 to msghash32 in ecdsa_sign/verify and add explanation
[upstream PR #852]: Add sage script for generating scalar_split_lambda constants
[upstream PR #857]: docs: fix simple typo, dependecy -> dependency
[upstream PR #858]: Fix insecure links
[upstream PR #860]: fixed trivial typo
[upstream PR #845]: Extract the secret key from a keypair
[upstream PR #862]: Autoconf improvements
[upstream PR #875]: Avoid casting (void**) values.
[upstream PR #878]: Remove unused secp256k1_fe_inv_all_var
[upstream PR #874]: Remove underscores from header defs.
[upstream PR #877]: Add missing secp256k1_ge_set_gej_var decl.
[upstream PR #880]: Add parens around ROUND_TO_ALIGN's parameter.
[upstream PR #864]: Add support for Cirrus CI
[upstream PR #882]: Use bit ops instead of int mult for constant-time logic in gej_add_ge
[upstream PR #894]: ctime_test: move context randomization test to the end
[upstream PR #891]: build: Add workaround for automake 1.13 and older
[upstream PR #901]: ci: Switch all Linux builds to Debian and more improvements
This PR was automatically created with ./contrib/sync-upstream.sh .
Also added -zkp modules to .cirrus.yml - with some small simplifications, but I didn't spend to much time on it because we may refactor this anyway upstream.

Still failing:
https://cirrus-ci.com/task/6253397255913472?command=cat_tests_log#L3

ACK 4091e619248b4723a2a4ed5dd7289628ee0320a5 merge commit picks the right parents, merge resolution and  additional commit look good


DNS Permit Override applies to all clients

I have set-up a DNS permit override for a specific client, but it seems like this entry is also permitted for at least another client - let me know if you want me to turn on specific logging or whatnot.
Cheers

Okay I think this is my fault, my bad 😅

for debugging this, do you have any suggestions on UI to make it easier ?


Grid header checkbox now clears on clear selection

Github Issue (if applicable): #735
Trello Link (if applicable):
Forum Link (if applicable):
Explanation of Bugfix/Feature/Enhancement:
Fixes grid header checkbox not unchecking after user performs "Clear Selection" from an Account's Edit dropdown.
Recommended Release Notes:

Thank you!


Timer timing issue

temporal-ruby Timer timing issue
Scenario:

Timer started (30 minutes)
Sleep started by workflow code (15 minute)
Worker goes down
Sleep ends
Timer fires.
Worker comes back up
Worker receives a history containing the sleep finishing followed by timer firing.

The workflow thus receives two TIMER_FIRED events in its history at once.  Now what should happen?
What’s happening in temporal-ruby today
It processes the timers in order.  Sleep first, then the other timer.  The workflow completes first, and it’s not allowed to do anything after the workflow has completed.  Thus, if the timer issues a command, the task will fail with Temporal::WorkflowAlreadyCompletingError.
Attempted solution
My first attempt was to early-out in state_manager.rb
        history_window.events.each do |event|
          apply_event(event)
          break if if workflow_finished?
        end

This fixed my problem and is logical; the workflow will execute the same as it would have if the worker had never gone down.  But, it will silently drop events.
I asked Maxim what other SDKs do…
Other SDKs’ behaviors
According to Maxim:

Temporal processes new events in batches. One batch per workflow task. It applies all the events before running workflow code. So in case of Java or Go, for example the Futures that correspond to timer will be set to ready before running any workflow threads. This way workflow can check if any of these Futures are ready to decide what to do next.
Signals have similar properties, but they use callbacks in Java and some other SDKs. Temporal SDK schedules callback threads before running workflow threads. This way workflow cannot complete before the callback threads were called.

Then in case of Java SDK it is going to make the main workflow thread eligible to run. But it still will run only after all other threads executed and all other Futures resolved.
Without this functionality you guaranteed to lose signals if they are delivered as callbacks.

Rust core does the same
Java implementation - we can a see that callbacks (timer in this case) have a higher priority than the main thread (which is awoken by the sleep in this case).
What now?
Changing temporal-ruby's behavior is obviously somewhat tricky and would cause version changes if not done in a version-safe way.  We might take this fix on but want to work with the community.

Should we fix this?
Implementation suggestions for the fix itself?
Version safety.  I'm guessing we should add a configuration that changes this behavior that folks can opt into.

Thanks for bringing this up!
I think there might be a relatively painless fix to this. All the concurrent execution threads in Ruby are represented by fibers, so if we can distinguish the callback fibers from the main one in which the workflow was executed and skip resuming it until the very end of the history window. We already do something very similar with event handlers ordering.
A simpler fix that you can do straight away — is to check yourself whether a workflow has finished or not from your time callback using workflow.completed?. Not pretty, but at least will save you from failing workflow tasks.
In term of version safety — I don't think it should affect anyone. At least I don't think there's a non-failure scenario where anyone would depend on this kind of behaviour. It they did they would already be seeing failures


Move assists to a separate crate

I am slowly coming to conclusion that ide_api_light does not make a lot of sense after all :D
This PR moves assists to a separate crate, so that assists can use database (so, inspect types, do name-resolution, etc)

bors r+


Route with params do not work after refresh

Hello there. I'm currently having an issue where my page do not work after refresh. It is clear that in Vuex inspector the event: {...} still has content. I have also tried asyncData() instead of fetch(), am I missing something here?
After refresh, it will says event is undefined.
According to official tutorials, I also like to know if axios in store actions is good practice or not. Thank you very much.
events/_id.vue
<template>
  <div>{{ event.attributes.name }}</div>
</template>

<script>
import { mapActions, mapGetters } from 'vuex'

export default {
  validate ({ params }) {
    return !isNaN(+params.id)
  },
  fetch ({ store, params }) {
    if (store.state.events.event.id !== params.id) {
      store.dispatch('events/getEvent', {
        id: params.id
      })
    }
    return store.state.events.event
  },
  computed: {
    ...mapGetters({
      event: 'events/event',
      error: 'events/error'
    })
  },
  methods: {
    ...mapActions({
      getEvent: 'events/getEvent'
    })
  }
}
</script>

store/events.js
// import { getUserFromLocalStorage } from '~/utils/auth'

export const state = () => ({
  event: {},
  events: [],
  error: ''
})

export const getters = {
  event: state => state.event,
  events: state => state.events,
  error: state => state.error
}

export const actions = {

  async getEvent ({ commit, dispatch }, { id }) {
    try {
      await this.$axios.$get(`events/${id}`)
      .then((res) => {
        console.log(res.data)
        commit('SET_EVENT', res.data)
        dispatch('clearError')
      })
    } catch (err) {
      dispatch('setError', err.response.data.error)
    }
  },

  async getEvents ({ commit, dispatch }) {
    try {
      await this.$axios.$get('events')
      .then((res) => {
        commit('SET_EVENTS', res.data)
        dispatch('clearError')
      })
    } catch (err) {
      dispatch('setError', err.response.data.error)
    }
  },

  setError ({ commit }, error) {
    commit('SET_ERROR', error)
    setTimeout(() => {
      commit('CLEAR_ERROR')
    }, 10000)
  },

  clearError ({ commit }) {
    commit('CLEAR_ERROR')
  }
}

export const mutations = {
  SET_EVENT (state, payload) {
    state.event = payload
  },
  SET_EVENTS (state, payload) {
    state.events = payload
  },
  SET_ERROR (state, payload) {
    state.error = payload
  },
  CLEAR_ERROR (state) {
    state.error = null
  }
}

In your case, it shuold be store.state.events.events.event.id

@clarkdo sorry if I'm unclear. I'm having issue only after I refresh the page, where it will shown as below:

I had tried with/without async/await, both fetch() and asyncData().

@bin4ryio Do you try v-if="event && event.attributes" ?
Your attributes field seems undefined :/
Do you try to print your event object to be sure it's correct ?

@jleveugle Hello. I have checked Vuex state in Chrome (F12). With your code I'm seeing that after a REFRESH the event state became null. Looks like it have to do with params is empty after refresh.
Before refresh:

After refresh:

I've used to and see no console logging of params.id after refresh.
  validate ({ params }) {
    return !isNaN(+params.id)
  },
  fetch ({ store, params }) {
    console.log(`params :: ${params}`)
    store.dispatch('events/getEvent', {
      id: params.id
    })
  },

event is a reference to state.event ?
Do you try to use $store.state.event in your template instead of event ?

Do you try to use $store.state.event in your template ?

That way it will surely work, but still does not solve the state loss after refresh.

Actually the fetch method is not called on client-side since it's called on derver-side after refresh. Are you sure the event displayed in console log is correct on server-side? Also, could you look at the source code and write to us what's inside window.__NUXT__?

Thanks for the prompt reply. I have successfully resolve it by using async fetch.... { await dispatch... }

I had the same problem when I tries to create my own router with asyncData(). The fix was to use the generate hook/property in the nuxt config and give it your prebuilt aliases as stated here.
For my case it was:
 generate: {     routes: function () {       return importData.map( (content) => {         return '/person/' + content.slug       })     }   }
Also, you need to use extendRoutes example:
module.exports = {   router: {     extendRoutes (routes, resolve) {       routes.push({         name: 'personne',         path: '/person/:slug?',         component: 'pages/person/_slug.vue'       })     }   } . .


Move hasOwnProperty to the last export

@TylorS this should fix #85!

Thank you for this PR @moneydance !! I'll deploy it later today and update everyone here

Hey @moneydance I deployed these changes this morning - Thanks again for this PR!
It's been a while since I really had to do any maintenance on this package (its been working well!) so I also took some time this morning to update all of the dependencies including TypeScript 3.4.5, so it should be very safe to use newer versions of TS. Every package got a major bump due to the breaking changes regarding the TS requirement

I realize how heavy handed that may have been. Let me know if you need me to do a patch release with only this change

No issues on my end with the bump. Thanks for getting this out so quickly!


Add ability do not wait deployment to complete. Just trigger master

This fixes #2757

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@jnummelin , updated PR according to comments. I've done --no-wait by analogy from upgrade command. Is it ok?


v2.1.0 maxes out CPU and doesn't finish

I am using readdirp with a little library called baconize and it fails consistently with certain combinations of files. It always gets to the last file and then just maxes out the CPU and doesn't finish. If I downgrade readdirp to 2.0.1 then it succeeds every time.
This only seems to occur on sufficiently complicated directories and it wasn't caught by the tests on baconize.
I can delve in and try to get more info if you like but considering the small delta between 2.0.1 and 2.1.0, I thought it might be possible for someone to figure out what might be causing this.

Thank you for this bug report. I had a similar case which was caused by the stream not ending. Downgrading to 2.0.1 helped fix the problem.

Interesting, main change that happened in between those versions is this commit.
@toddself do you have an idea how this might be related?

my initial guess is that it's somehow stuck in a paused state -- it intentionally loops asynchronously when paused.
@davej any other information on how to repo this would be super helpful!  i can try to dive into this later today

I've since switched to a different library and this was a long time ago so I'm not able to provide any insight. Sorry.


Make sure we actually try to get nonce from options first

Need to think about a better fix for this issue.