[MFEM example] Wrong result when change preconditioner in example MFEM using Ginkgo

Hi!
When I run ex1 (https://github.com/mfem/mfem/blob/master/examples/ginkgo/ex1.cpp) by reference excutor and cuda excutor, it's ok. But when I change preconditioner (Jacobi preconditioner), I ran by reference is ok. But with cuda excutor the result is wrong, it does not converge.
I think it's a error. You can try it and see.
Thank you!

It also occurs, when I try to modify the example 2 (https://github.com/mfem/mfem/blob/master/examples/ex2.cpp) using Ginkgo solver. It occurs with all preconditioners  using Cuda excutor. I try to use reference excutor and it's right, but with Cuda excutor it has wrong the result.

The examples with MFEM need to be modified to use the cuda executor. Unfortunately, they do not work out of the box. We are working on integrating it completely. You can have a look at @nbeams PR on the MFEM side with this branch.
But, you should still be able to use the cuda executor from Ginkgo's side, by the following modifications:
diff --git a/examples/ginkgo/ex1.cpp b/examples/ginkgo/ex1.cpp
index 2d9f565..ca8d9cb 100644
--- a/examples/ginkgo/ex1.cpp
+++ b/examples/ginkgo/ex1.cpp
@@ -195,8 +195,8 @@ int main(int argc, char *argv[])
       {
 #ifdef MFEM_USE_GINKGO
          // Solve the linear system with CG + ILU from Ginkgo.
-         std::string executor = "reference";
-         auto exec = gko::ReferenceExecutor::create();
+         std::string executor = "cuda";
+         auto exec = gko::CudaExecutor::create(0, gko::OmpExecutor::create());
          auto ilu_precond =
             gko::preconditioner::Ilu<gko::solver::LowerTrs<>,
             gko::solver::UpperTrs<>, false>::build()

But be aware that this copies the data from the host to the gpu, performs the solve and copies back the results to the cpu and hence it can be slow. For the version without the copies, you can have a look at the PR from @nbeams.

The examples with MFEM need to be modified to use the cuda executor. Unfortunately, they do not work out of the box. We are working on integrating it completely. You can have a look at @nbeams PR on the MFEM side with this branch.
But, you should still be able to use the cuda executor from Ginkgo's side, by the following modifications:
diff --git a/examples/ginkgo/ex1.cpp b/examples/ginkgo/ex1.cpp
index 2d9f565..ca8d9cb 100644
--- a/examples/ginkgo/ex1.cpp
+++ b/examples/ginkgo/ex1.cpp
@@ -195,8 +195,8 @@ int main(int argc, char *argv[])
       {
 #ifdef MFEM_USE_GINKGO
          // Solve the linear system with CG + ILU from Ginkgo.
-         std::string executor = "reference";
-         auto exec = gko::ReferenceExecutor::create();
+         std::string executor = "cuda";
+         auto exec = gko::CudaExecutor::create(0, gko::OmpExecutor::create());
          auto ilu_precond =
             gko::preconditioner::Ilu<gko::solver::LowerTrs<>,
             gko::solver::UpperTrs<>, false>::build()

But be aware that this copies the data from the host to the gpu, performs the solve and copies back the results to the cpu and hence it can be slow. For the version without the copies, you can have a look at the PR from @nbeams.

Thank you for your replying. I mean I made the same modification as above, but the result is wrong. You can try to modify Jacobi preconditioner and build: ./ex1 -o 3 insteaf of ILU preconditioner and ./ex1

I am sorry but I cannot reproduce it. This seems to work for me:
diff --git a/examples/ginkgo/ex1.cpp b/examples/ginkgo/ex1.cpp
index 2d9f565..5539530 100644
--- a/examples/ginkgo/ex1.cpp
+++ b/examples/ginkgo/ex1.cpp
@@ -195,14 +195,16 @@ int main(int argc, char *argv[])
       {
 #ifdef MFEM_USE_GINKGO
          // Solve the linear system with CG + ILU from Ginkgo.
-         std::string executor = "reference";
-         auto exec = gko::ReferenceExecutor::create();
-         auto ilu_precond =
-            gko::preconditioner::Ilu<gko::solver::LowerTrs<>,
-            gko::solver::UpperTrs<>, false>::build()
-            .on(exec);
-         GinkgoWrappers::CGSolver ginkgo_solver(executor, 1, 2000, 1e-12, 0.0,
-                                                ilu_precond.release() );
+         using bj = gko::preconditioner::Jacobi<double, int>;
+         std::string executor = "cuda";
+         auto exec = gko::CudaExecutor::create(0, gko::OmpExecutor::create());
+	   auto bj_precond = bj::build()
+                                     .with_max_block_size(16u)
+                                     .with_storage_optimization(
+                                         gko::precision_reduction::autodetect())
+                                     .on(exec);
+	  GinkgoWrappers::CGSolver ginkgo_solver(executor, 1, 2000, 1e-12, 0.0,
+                                                bj_precond.release() );
          ginkgo_solver.solve(&((SparseMatrix&)(*A)), X, B);
 #endif
       }

I see that you mean for --order 3 . For that it seems that the solution does stagnate for cuda , but not for reference. I would suggest using ILU for now as that seems to converge.

I see that you mean for --order 3 . For that it seems that the solution does stagnate for cuda , but not for reference. I would suggest using ILU for now as that seems to converge.

Yes, thank you for your support.

Hi @pratikvn. I'm sorry to bother you. I try to follow the code of PR you mentioned but it doesn't work when I build MFEM. I use ginkgo-1.2.0. It returns:
linalg/../fem/../mesh/../fem/../linalg/ginkgo.hpp: In constructor ‘mfem::GinkgoWrappers::GinkgoPreconditionerBase::GinkgoPreconditionerBase(std::shared_ptr, mfem::SparseMatrix&, mfem::Array&, bool)’:
linalg/../fem/../mesh/../fem/../linalg/ginkgo.hpp:675:111: warning: narrowing conversion of ‘(& inv_permutation_indices)->mfem::Array::Size()’ from ‘int’ to ‘gko::dim<2>::dimension_type {aka long unsigned int}’ inside { } [-Wnarrowing]
vec_permute_ = gko::matrix::Permutation::create(
^
linalg/../fem/../mesh/../fem/../linalg/ginkgo.hpp:679:115: warning: narrowing conversion of ‘(& inv_permutation_indices)->mfem::Array::Size()’ from ‘int’ to ‘gko::dim<2>::dimension_type {aka long unsigned int}’ inside { } [-Wnarrowing]
vec_inv_permute_ = gko::matrix::Permutation::create(
^
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/matrix.cpp -o linalg/matrix.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/hypre.cpp -o linalg/hypre.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/operator.cpp -o linalg/operator.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/blockmatrix.cpp -o linalg/blockmatrix.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/sparsemat.cpp -o linalg/sparsemat.o
linalg/ginkgo.hpp: In constructor ‘mfem::GinkgoWrappers::GinkgoPreconditionerBase::GinkgoPreconditionerBase(std::shared_ptr, mfem::SparseMatrix&, mfem::Array&, bool)’:
linalg/ginkgo.hpp:675:111: warning: narrowing conversion of ‘(& inv_permutation_indices)->mfem::Array::Size()’ from ‘int’ to ‘gko::dim<2>::dimension_type {aka long unsigned int}’ inside { } [-Wnarrowing]
vec_permute_ = gko::matrix::Permutation::create(
^
linalg/ginkgo.hpp:679:115: warning: narrowing conversion of ‘(& inv_permutation_indices)->mfem::Array::Size()’ from ‘int’ to ‘gko::dim<2>::dimension_type {aka long unsigned int}’ inside { } [-Wnarrowing]
vec_inv_permute_ = gko::matrix::Permutation::create(
^
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/handle.cpp -o linalg/handle.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/blockoperator.cpp -o linalg/blockoperator.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/complex_operator.cpp -o linalg/complex_operator.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/blockvector.cpp -o linalg/blockvector.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/petsc.cpp -o linalg/petsc.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/sparsesmoothers.cpp -o linalg/sparsesmoothers.o
nvcc   -O3 -std=c++11 -x=cu --expt-extended-lambda -arch=sm_60 -ccbin g++ -isystem ./../ginkgo/install/include  -c linalg/ginkgo.cpp -o linalg/ginkgo.o
At end of source: error: expected a "}"
1 error detected in the compilation of "/tmp/tmpxft_00002ad4_00000000-6_ginkgo.cpp1.ii".
makefile:422: recipe for target 'linalg/ginkgo.o' failed
make: *** [linalg/ginkgo.o] Error 1

Also it doesn't work when I use ginkgo-1.3.0.

@nbeams, maybe you can help out here ?

Hi @nghiakvnvsd,
Let me make sure I understand exactly what you are testing here.
The first error you reported was with the current master branches of MFEM and Ginkgo, correct?  And you saw a problem with the block Jacobi preconditioner for ex1 with CUDA.  Did you also try the OpenMP executor?  Also, what happens if you set the max_block_size to 1, i.e., making it scalar Jacobi instead of block Jacobi? This should work.
For the issue with ex2, I think I might be of more use if I could see exactly what modifications you made to the existing code.  Do you have the code available somewhere?
Finally, the build issue with my branch of MFEM.  Did you completely check out the branch, or add its commits on top of the current MFEM master?  Also, I should say, I have been using that branch with my own branch of Ginkgo, as well (https://github.com/ginkgo-project/ginkgo/tree/mfem-wrappers).  I have not seen that particular build issue before, but it wouldn't necessarily surprise me that there are build issues when not using my branch of Ginkgo.  There have been changes to both Ginkgo and MFEM since I last did updates on my branches.
So, I wouldn't actually recommend using my branch of MFEM as it currently stands.  I think what @partikvn meant was to look at the code in the PR for accessing an MFEM SparseMatrix on host or device, without copying, to wrap it as a Ginkgo matrix.  E.g., with std::shared_ptr<const gko::Executor> exec and SparseMatrix a

      bool on_device = false;
      if (exec->get_master() != exec)
      {
         on_device = true;
      }

      using mtx = gko::matrix::Csr<double, int>;
      const int nnz =  a.GetMemoryData().Capacity();
      auto gko_sparse = mtx::create(
                           exec, gko::dim<2>(a.Height(), a.Width()),
                           gko::Array<double>::view(exec,
                                                    nnz,
                                                    a.ReadWriteData(on_device)),
                           gko::Array<int>::view(exec,
                                                 nnz,
                                                 a.ReadWriteJ(on_device)),
                           gko::Array<int>::view(exec, a.Height() + 1,
                                                 a.ReadWriteI(on_device)));

Hi @nbeams
The first error I reported was with the download of website mfem.org and ginkgo-1.2.0.
The modified ex2 I mentioned, it was just combined between ex1 (ginkgo) and ex2 (serial).
I have tried to use the mfem-wrappers and tried to merge ginkgo-precond and master mfem but it didn't work when I tried to run the example 1 (ginkgo) using it reports:
Options used:
   --mesh ../../data/star.mesh
   --order 1
   --no-static-condensation
   --no-partial-assembly
   --device cpu
   --visualization
   --use_gko_solver
Device configuration: cpu
Memory configuration: host-std
Number of finite element unknowns: 20801
Size of linear system: 20801
Segmentation fault (core dumped)

Sorry, I should have changed "I wouldn't actually recommend using my branch of MFEM as it currently stands" to "do not use my branch of MFEM as it currently stands."  I forgot I had commented out some lines related to building Ginkgo solvers.  They were causing conflicts with an update in Ginkgo that was related to fixing a CUDA compiler issue.  I wasn't using the Ginkgo solvers in the tests in my branch, just the preconditioners, so it didn't matter to me at the time.
(But thanks for the reminder that I need to check again with the latest version of Ginkgo, and fix the solver integration in MFEM if needed.)
It looks like ex2 isn't configured to work with CUDA on the MFEM side.  You would also have to add the Device configuration steps.

Sorry, I should have changed "I wouldn't actually recommend using my branch of MFEM as it currently stands" to "do not use my branch of MFEM as it currently stands." I forgot I had commented out some lines related to building Ginkgo solvers. They were causing conflicts with an update in Ginkgo that was related to fixing a CUDA compiler issue. I wasn't using the Ginkgo solvers in the tests in my branch, just the preconditioners, so it didn't matter to me at the time.
(But thanks for the reminder that I need to check again with the latest version of Ginkgo, and fix the solver integration in MFEM if needed.)
~It looks like ex2 isn't configured to work with CUDA on the MFEM side. You would also have to add the Device configuration steps.~ I take it back. I think it should currently build the matrix on the CPU and copy to GPU, so it would actually expect MFEM to be using the CPU, anyway.

I think I will wait for the next version MFEM 4.2. Thank you very much for your support.

Could you post the code of the modified ex2 to use Ginkgo together with it so that we can try to look into it? Thanks!

Could you post the code of the modified ex2 to use Ginkgo together with it so that we can try to look into it? Thanks!

Yes. Thank you!!!
https://drive.google.com/drive/folders/1RNyy8UIcndXxzseSU4pO1mpcg_O83o3F?usp=sharing

As you see, The ex2.cpp and ex2-cuda-executor.cpp just are different between "cuda" and "reference" executor. But I don't understand why ex2 converge while ex2-cuda-executor doesn't converge

@nghiakvnvsd, I have built your examples ex2.cpp and ex2-cuda-executor.cpp locally with the latest MFEM master branch and Ginkgo 1.3.  I will investigate the issue.

@nghiakvnvsd: Unfortunately, I do not have an answer for you yet, but I haven't forgotten and I'm still looking into it.

@nghiakvnvsd Update:
The problem for this specific matrix is the underlying generation of the ILU preconditioner. By default, we use the ParILU factorization to generate the incomplete approximates L and U. L and U have (combined) the same sparsity pattern as the original matrix (sparsity ILU(0)), however, we only approximate L and U (it is not computed directly by default).
The reference implementation for the ParILU actually does compute the ILU(0) and not the approximation, which is why it works fine (and OpenMP produces a close enough result).
Here, for CUDA, I would actually recommend to use the proper ILU(0) factorication method, which you can do with:
    factorization = share(
        gko::factorization::Ilu<value_type, index_type>::build().on(exec));
    auto solver_gen =
        gko::solver::Cg<value_type>::build()
            .with_criteria(
                gko::stop::Iteration::build().with_max_iters(2000u).on(exec),
                gko::stop::ResidualNormReduction<value_type>::build()
                    .with_reduction_factor(reduction_factor)
                    .on(exec))
            .with_preconditioner(precond_fact)
            .on(exec);

For OpenMP and Reference, we did not implement this factorization method yet, but ParILU does work fine there.
I am now looking into the Jacobi preconditioner and why it sometimes prevents conversion.
I will post an update on that here as soon as I have something notable.


Risk Profile for Vaccinated people in Calculator

I and my pod are interacting with several people who are fully vaccinated, but who we cannot get a reliable activity log from. It would be extremely helpful to have an "average vaccinated person in your area" option.

We've had a lot of discussions on this topic and unfortunately concluded there isn't a epistemically sound way to get a number for "average vaccinated person" with the data we have.
Consider:

If 1% of the population of your area is vaccinated, you might think that the vaccinated people are 80% safer than average. That doesn't necessarily hold, because they know they are vaccinated, which may change their behavior (for instance, eating indoors in a crowded restaurant once or twice per week would negate the benefit from being vaccinated).

Worse:

if 99% of the population of your area is vaccinated, the "average" person is vaccinated, so there is nearly no bonus for being vaccinated. In fact, there may be a penalty for being unvaccinated, since the average unvaccinated person is more susceptible to getting covid than the average person (who is vaccinated)

Our suggestion is to use the Risk Tracker, which has been updated to support vaccines for Custom People. If you have a vague sense of the person's weekly behaviors, you can likely build a better model than "average."
If you truly know nothing about the person's behaviors except that they're vaccinated, we acknowledge that they are perhaps, on average, safer, but couldn't put a number on it. If your pod wants to be less conservative, you can certainly do that, but we don't feel comfortable putting a number on this given the uncertainty.

While I can see that there's uncertainty, surely it's possible to have a best estimate?
It seems odd to have microcovid as the singular outlier among the health guidance in saying that one needs to be exactly as cautious around vaccinated people as unvaccinated people when sources such as the CDC are condoning unmasked indoor gatherings with them.

@hmrm  It's an understandable question. For any paritcular person you're seeing, YES, you absolutely can adjust your behavior with them. That's what the Risk Tracker allows you to do: model their activities over the last 10 days, and mark whether they are vaccinated.
Would that help address your situation?
If not, can you tell us more about the "average vaccinated person in your area" scenario? Who specifically are you seeing? And can you tell us how you know they are "average" and "vaccinated" but don't know anything else about them?

Unfortunately I cannot accurately (or adequately conservatively) model their activities over the past 10 days. With this particular individual, an elderly relative, there's a lot of shame around any flaunting of pandemic guidance, and therefore particularly risky activities are often omitted from discussion (only to be discovered later). This is not unique; I've had the same problem before with other members of my community.
This interacts poorly with the fact that a single indoors meal at a restaurant has an immense impact on one's risk profile. If one accepted at face value the activities they've told us about doing as representative, they would have a very low number of microcovids, but there has already been evidence that these reports are likely underrepresenting their microcovids, but it's very hard to say how much. Putting a number on the chance that they have gone to a restaurant seems less likely to be accurate than treating it as fully unknown and putting them at some average risk level given the information that I do have, which is that they have fully received one of the mRNA vaccines.
I think this tendency to under-report risky activities may be extremely common in micro-cultures such as mine where there's immense social pressure to take all possible COVID mitigations, as I've seen it in multiple people throughout the pandemic. Because of that, I'm very skeptical of any plan that involves guessing the activities of people who are not keeping a log.

I definitely hear your frustration around trying to figure out how to model
this relative.
IMO, knowing that they are prone to doing risky things and not telling you
about them makes using "average" to describe them kind of sketchy. The
"average" category needs to be amortized over many people about whom you
know the same amount. For instance, when I go to the grocery store than any
individual may be 10x riskier than average, but I'll be doing the same
activity (passing briefly in a store) with hundreds of people over the
course of the pandemic so in expectation it ends up regressing to the mean.
Here's what I'd do:
Use the risk tracker to build two models for this person. One is what they
say they are doing, the other is an intentional overestimate of what they
might potentially be doing (i.e. "they're probably not eating out MORE than
X times per week, probably not going to more than Y indoor gatherings with
Z people). The truth will probably be in between. Then you figure out how
much of the possible range is ok for you (maybe you're ok with the average
of these two scenarios, and that's your estimate). Perhaps you're willing
to take the risk that the relative is all the way at the high end. Possibly
low case rates + a good vaccine means that you'd feel safe even if your
relative were doing things that would have been reckless a few months ago.
On Fri, Mar 19, 2021 at 8:15 PM Haney Maxwell @.***>
wrote:

Unfortunately I cannot accurately (or adequately conservatively) model
their activities over the past 10 days. With this particular individual, an
elderly relative, there's a lot of shame around any flaunting of pandemic
guidance, and therefore particularly risky activities are often omitted
from discussion (only to be discovered later). This is not unique; I've had
the same problem before with other members of my community.
This interacts poorly with the fact that a single indoors meal at a
restaurant has an immense impact on one's risk profile. If one accepted at
face value the activities they've told us about doing as representative,
they would have a very low number of microcovids, but there has already
been evidence that these reports are likely underrepresenting their
microcovids, but it's very hard to say how much. Putting a number on the
chance that they have gone to a restaurant seems less likely to be accurate
than treating it as fully unknown and putting them at some average risk
level given the information that I do have, which is that they have fully
received one of the mRNA vaccines.
I think this tendency to under-report risky activities may be extremely
common in micro-cultures such as mine where there's immense social pressure
to take all possible COVID mitigations, as I've seen it in multiple people
throughout the pandemic. Because of that, I'm very skeptical of any plan
that involves guessing the activities of people who are not keeping a log.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/microcovid/microcovid/issues/808#issuecomment-803235068,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABM25LLPXNS53AXINANMNE3TEQHN3ANCNFSM4ZNHH7HQ
.

I deeply appreciate your helping me out with this (and all the work you all continue to do on the microcovid project, which has made my life immeasurably better over the past six months). Apologies if I'm being obtuse, but I feel like we're talking past each other, so I'm going to go into a bit more depth in the hope that that makes things a bit clearer. I think our core confusion is one of principle rather than this concrete example, specifically:
I believe that any estimate or upper bound I (or any other similarly situated person) make of other people's activities will be a product of my own biases and assumptions, and will bear little relation to any reality of their actual activities or risk.
Let's look at the process I would use for making such an estimate or upper bound: my baseline is going to be the activities of either me or someone else I know who has been logging. Alternatively, I could start with the activities that they've mentioned taking part in, and guess how often they occur per week, which would lead to a remarkably similar number of baseline microcovids compared to the former method. Then I can add a healthy cushion on top of that of activities they haven't mentioned doing, but seem to me within the realm of conceivability, amounting to maybe 10x the microcovids I had at the baseline.
The problem is that I, and the rest of my pod, have been immensely blessed with the ability to maintain a yearly risk level of <.1%. My realm of "reasonable conceivability" doesn't extend to the microcovids of an average person, which is >100x my baseline. If my estimates were accurate enough to be worth using, we wouldn't have a pandemic. I think this likely extends to almost anyone operating at the .1% risk threshold, as we will all have a fairly useless frame of reference.
To pull it back to the concrete example: my intuition is that there's a ~10 percent chance that they ate indoors at a restaurant in the last 10 days, but, when it comes down to it, that's based on literally nothing. I don't know anyone who has admitted to having a meal indoors in a restaurant in the last year, or even discussed doing so out loud, and have no basis for any estimate of how likely anyone would be to do so. With the vaccine, even if they did eat indoors at a restaurant once in the week, that only accrues ~300 microcovids, so they still wind up looking extremely safe compared with the average person, but why am I assuming they didn't do so twice? I think the better approach is to say "I have no meaningful idea what their activities are, so the center of the probability distribution is that they're an average person-who-has-been-vaccinated".
To be clear though, this is not a case of just a single person. There are members of my pod who are now vaccinated, and they are getting increasingly many invitations for various social gatherings with other vaccinated people, most of whom we have even less context for than for my relative.
Hopefully this makes my use-case more clear, and elucidates why estimating (or upper-bounding) activities is not a solution. Let me know if I'm missing something and there's a sound methodology I could use to solve this issue. Thanks again for all the help with this, and apologies for the extended back-and-forth.

Perhaps this might help - the lower bound on an average person is the
regional prevalence times the vaccine multiplier. So in SF, the prevalence
is .05% = 500microCOVID. The vaccine multiplier is 0.2, so the average
vaccinated person is probably between 100 and 500 microCOVID.
You might split the difference and apply the square-root of the vaccine
multiplier, calling them 223.
We're looking at data from Pfizer that suggests revising to a .05
multiplier, so that methodology would yield 111.
I'm kind of curious why your pod uses 0.1% - do you have particularly
vulnerable people you are trying to protect?
On Sat, Mar 20, 2021 at 9:23 PM Haney Maxwell @.***>
wrote:

I deeply appreciate your helping me out with this (and all the work you
all continue to do on the microcovid project, which has made my life
immeasurably better over the past six months). Apologies if I'm being
obtuse, but I feel like we're talking past each other, so I'm going to go
into a bit more depth in the hope that that makes things a bit clearer. I
think our core confusion is one of principle rather than this concrete
example, specifically:
I believe that any estimate or upper bound I (or any other similarly
situated person) make of other people's activities will be a product of my
own biases and assumptions, and will bear little relation to any reality of
their actual activities or risk.
Let's look at the process I would use for making such an estimate or upper
bound: my baseline is going to be the activities of either me or someone
else I know who has been logging. Alternatively, I could start with the
activities that they've mentioned taking part in, and guess how often they
occur per week, which would lead to a remarkably similar number of baseline
microcovids compared to the former method. Then I can add a healthy cushion
on top of that of activities they haven't mentioned doing, but seem to me
within the realm of conceivability, amounting to maybe 10x the microcovids
I had at the baseline.
The problem is that I, and the rest of my pod, have been immensely blessed
with the ability to maintain a yearly risk level of <.1%. My realm of
"reasonable conceivability" doesn't extend to the microcovids of an average
person, which is >100x my baseline. If my estimates were accurate enough to
be worth using, we wouldn't have a pandemic. I think this likely extends to
almost anyone operating at the .1% risk threshold, as we will all have a
fairly useless frame of reference.
To pull it back to the concrete example: my intuition is that there's a
~10 percent chance that they ate indoors at a restaurant in the last 10
days, but, when it comes down to it, that's based on literally nothing. I
don't know anyone who has admitted to having a meal indoors in a restaurant
in the last year, or even discussed doing so out loud, and have no basis
for any estimate of how likely anyone would be to do so. With the vaccine,
even if they did eat indoors at a restaurant once in the week, that only
accrues ~300 microcovids, so they still wind up looking extremely safe
compared with the average person, but why am I assuming they didn't do so
twice? I think the better approach is to say "I have no meaningful idea
what their activities are, so the center of the probability distribution is
that they're an average person-who-has-been-vaccinated".
To be clear though, this is not a case of just a single person. There are
members of my pod who are now vaccinated, and they are getting increasingly
many invitations for various social gatherings with other vaccinated
people, most of whom we have even less context for than for my relative.
Hopefully this makes my use-case more clear, and elucidates why estimating
(or upper-bounding) activities is not a solution. Let me know if I'm
missing something and there's a sound methodology I could use to solve this
issue. Thanks again for all the help with this, and apologies for the
extended back-and-forth.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/microcovid/microcovid/issues/808#issuecomment-803510054,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABM25LL2O76BQ5L36VREDKTTEVYD7ANCNFSM4ZNHH7HQ
.

I'm kind of curious why your pod uses 0.1% - do you have particularly
vulnerable people you are trying to protect?

Yes, we have a member who is extremely elderly and one with a medical condition that would interacts poorly with COVID. They should both be fully vaccinated within a month or so, at which point I anticipate we may re-evaluate (although the course of the pandemic may be very different at that point and it may be unnecessary). However, even were it not for those members, when I was setting up the pod I found that almost all the members (even those without complicating factors) were uncomfortable with a 1% risk level, and it took a bit of convincing to build comfort around a .1% risk level.

In Israel there are now many activities that are open basically only to vaccinated people. For example, indoor dining is only allowed for people who have been vaccinated, while unvaccinated people can sit outdoors. Since probably 75% of the adult population has been fully vaccinated, I can see that an average vaccinated person is very similar to the average person in general, but I was wondering if it's safer to sit indoors (since then everyone is vaccinated) or outdoors (since the risk of infection is much less outdoors).

Haney:

Yes, we have a member who is extremely elderly and one with a medical
condition that would interact poorly with COVID. They should both be fully
vaccinated within a month or so, at which point I anticipate we may
re-evaluate

That makes sense. If I were living with someone that matched that
description I would probably hold off on indoor activities with anyone I
didn't have a good model for for at least a little longer; some combination
of your podmates getting vaccinated and prevalence continuing to drop
should get you there. It would be a shame for one of them to get COVID so
close to when they would be vaccinated.
This actually lines up pretty closely the CDC's guidance:

Visit with unvaccinated people from a single household who are at low
risk for severe COVID-19 disease indoors without wearing masks or physical
distancing

Adam:

I was wondering if it's safer to sit indoors (since then everyone is
vaccinated) or outdoors (since the risk of infection is much less outdoors).

Outside is still probably safer; Israel's data is showing about a 20x
reduction in cases among vaccinated people. Outdoors is probably 20x safer
than indoors, perhaps even more. So if you assume indoors and outdoors are
similar levels of crowded and every single person indoors is vaccinated and
every single person indoors is unvaccinated, the numbers work out about the
same. But consider:
a) some vaccinated people will probably sit outdoors, so the average risk
of a person who is outdoors is a mix of the two groups.
b) If vaccinated people are allowed to do more things / see more people,
they get exposed to more risk and that potentially reduces how much safer
they are than an unvaccinated person (who is presumed to continue being
safe, not eating indoors, etc)
On Sun, Mar 21, 2021 at 11:39 AM Haney Maxwell @.***>
wrote:

I'm kind of curious why your pod uses 0.1% - do you have particularly
vulnerable people you are trying to protect?
Yes, we have a member who is extremely elderly and one with a medical
condition that would interacts poorly with COVID. They should both be fully
vaccinated within a month or so, at which point I anticipate we may
re-evaluate (although the course of the pandemic may be very different at
that point and it may be unnecessary). However, even were it not for those
members, when I was setting up the pod I found that almost all the members
(even those without complicating factors) were uncomfortable with a 1% risk
level, and it took a bit of convincing to build comfort around a .1% risk
level.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/microcovid/microcovid/issues/808#issuecomment-803638899,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABM25LOZPUPEID6KDPRJIKDTEY4NZANCNFSM4ZNHH7HQ
.

This is an old thread but I want to +1 the request for a risk profile for "person about whom all I know is that they're vaccinated".
Population vaccination levels can and should be accounted for in the math for this, of course. If you know the local vaccination levels and local covid prevalence levels you can calculate an approximate covid prevalence level for each of "local unvaccinated people", "local 2-shot vaccinated people", etc. This does ignore the possibility of risk compensation, but I think it's much better than nothing. (Perhaps there could be a warning saying something like "this assumes vaccinated and unvaccinated people have roughly the same behaviors; you may want to adjust accordingly".)
My main use case is modeling the risk from attending a medium-to-large gathering of vaccinated people; this seems obviously safer than an equivalent gathering of unvaccinated people, or people not filtered by vaccination status; also for a largeish gathering it doesn't really make sense for me to try to model people individually since I don't really know much about them other than that one fact. As a rough estimate I've been manually doing the kind of math I mentioned above based on local vaccination levels, but this is very annoying and it'd be nice to have it automated!
It also would be useful for modeling smaller gatherings with vaccinated people I'm not close enough to to ask a lot of questions about their lifestyle. I realize that's necessarily riskier than meeting with people whose exposure levels I do know things about, but it's still gonna be less risk in expectation than meeting with unvaccinated people, and I'd rather have some way to model this.

I agree that this would be a good feature. I don't have time/energy to make this change, but could help someone who is motivated. Here's what would need to be done:
1 Update update_prevalence.py to pull in vaccination data from covidactnow and jhu.
2 Include this data in the js and csv exports
3 Update calculate.ts to compute the vaccine-adjusted prevalence for average vaccinated people
3 Add fields in the manual prevalence entry to support adding vaccinated %
4 Add a vaccine selector to the "Risk Profile" section in the calculator
5 Add the requisite field to CalculatorData
6 Add logic to calculate.ts to calculate the person's risk (need 3 different formulas depending on if the person is average, fixed value, or calculated by activities)
7 Add tests to calculate_test.ts to demonstrate proper functionality.
8 Figure out how to do all this logic in the Risk Tracker as well.

For the US, ignoring J&J, I think the equations roughly look like:
RA = avg. risk
RV = vaccinated risk
RH = half-vaccinated risk
RU = unvaccinated risk
PV = proportion vaccinated
PH, PU ....
RA = RV * PV + RH * PH + RU * PU
RV = .56 * RH = .1 * RU
->
RA = RV * PV + (RV/.56) * PH ...
RA = .1 * RU * PV + (.1/.56) * RU * PH ...
->
RV = RA / (PV + PH/.56 + PU/.1)
RU = RA / (.1 * PV + (.1/.56) * PH + PU)
Plugging in my local numbers I get multipliers of roughly .16 for vaccinated otherwise-unknowns and 1.6 for unvaccinated otherwise-unknowns.

Hi, new to this -
If we assume the vaccine multiplier is .2, would choosing a risk profile that's .2 as high yield the same results?
For example, if I'm calculating the risk of a dinner party with vaccinated guests who are average people in my area (3000) but who I know are vaccinated (so 3000 * .2), the risk profile would be .2 of its original number (600)?  Or am I totally missing something?
I respect the hesitancy to put out a sound number, just seeing if there's an easy way to calculate a workaround in the meantime.

JHU has vaccine breakdowns by type for US & worldwide, but not counties. It al: https://github.com/govex/COVID-19/blob/master/data_tables/vaccine_data/us_data/hourly/vaccine_data_us.csv
covidactnow has vaccine data for US counties, but without type breakdowns: https://apidocs.covidactnow.org/data-definitions/

@beshaya could you combine those two data sets with the assumption that any individual county matches the national breakdown of vaccine types? Not a perfect assumption, but probably "close enough"?

Sure, yeah, just makes the task larger.
On Fri, May 28, 2021, 17:45 Jeremy Blanchard @.***>
wrote:

@beshaya https://github.com/beshaya could you combine those two data
sets with the assumption that any individual county matches the national
breakdown of vaccine types? Not a perfect assumption, but probably "close
enough"?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microcovid/microcovid/issues/808#issuecomment-850738177,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABM25LJFNBPWZAGV7KLNNVLTQA2LXANCNFSM4ZNHH7HQ
.

I mean I'm happy with not doing it that way. I guess I inferred from your last post that you were saying you weren't sure which data set to use.

No, we need both. I'm putting things here as notes to myself.
On Sat, May 29, 2021, 10:26 Jeremy Blanchard @.***>
wrote:

I mean I'm happy with not doing it that way. I guess I inferred from your
last post that you were saying you weren't sure which data set to use.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microcovid/microcovid/issues/808#issuecomment-850868874,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABM25LLPVEG5E2TNCXAH24DTQEPUTANCNFSM4ZNHH7HQ
.


Morbo does not ignore .swp files anymore

I have just updated Mojolicious and now Morbo does not ignore the .swp files created by Vim anymore. Is this a bug?

Is this the wrong place to ask?

I just upgraded 6.41 -> 6.55 and found the same issue. Morbo is restarting whenever I type anything. I have reverted back to 6.41 for now.

We know about the issue. It can be fixed with this patch:
RawEditGraphNewPowered by Mojopaste
diff --git a/lib/Mojo/Server/Morbo.pm b/lib/Mojo/Server/Morbo.pm
index 9217ea9..873e86e 100644
--- a/lib/Mojo/Server/Morbo.pm
+++ b/lib/Mojo/Server/Morbo.pm
@@ -4,6 +4,7 @@ use Mojo::Base -base;
 # "Linda: With Haley's Comet out of ice, Earth is experiencing the devastating
 #         effects of sudden, intense global warming.
 #  Morbo: Morbo is pleased but sticky."
+use File::Basename 'basename';
 use Mojo::Server::Daemon;
 use Mojo::Util qw(deprecated files);
 use POSIX 'WNOHANG';
@@ -24,6 +25,7 @@ sub modified_files {
   my $cache = $self->{cache} ||= {};
   my @files;
   for my $file (map { -f $_ && -r _ ? $_ : files $_ } @{$self->watch}) {
+    next if basename($file) =~ /^\./;
     my ($size, $mtime) = (stat $file)[7, 9];
     my $stats = $cache->{$file} ||= [$^T, $size];
     next if $mtime <= $stats->[0] && $size == $stats->[1];
diff --git a/t/mojo/morbo.t b/t/mojo/morbo.t
index ed53db8..c213afa 100644
--- a/t/mojo/morbo.t
+++ b/t/mojo/morbo.t
@@ -116,9 +116,10 @@ is $tx->res->body, 'Hello!', 'right content';

 # New file(s)
 is_deeply $morbo->modified_files, [], 'directory has not changed';
-my @new = map { catfile $subdir, "$_.txt" } qw/test testing/;
+my @new = map { catfile $subdir, "$_.txt" } qw(.skip.swp test testing);
 spurt 'whatever', $_ for @new;
-is_deeply $morbo->modified_files, \@new, 'two files have changed';
+is_deeply $morbo->modified_files, [grep { $_ !~ /skip\.swp/ } @new],
+  'two files have changed';
 is_deeply $morbo->modified_files, [], 'directory has not changed again';

Need to decide if this is the right thing to do.

The patch is working, thanks.
It looks like a nice solution to me. I don't think anybody puts application code in dotfiles. But if you want to be conservative you could enable it via a --ignore-dotfiles switch.

Yes, the patch works.

:+1: for @jhthorsen 's patch as long as it includes some documentation somewhere

:+1: from me too

You don't think it might be problematic that this can't handle directories like .git? Is it better for performance to check the prefix before or after mtime/size?

hmmmm, yeah, handling .git would probably be wise though it is less important than swp since you have to save and commit before the .git directory is changed

I supposed stat() is a lot slower than calling a perl function.
What do you mean with "can't handle directories like .git" @kraih ? Do you mean --watch .git or --watch . ? If so, I imagine that would be slow either way. At least this patch avoids watching temp files while editing.

As @jberger pointed out, here is what plack does: https://metacpan.org/source/MIYAGAWA/Plack-1.0039/lib/Plack/Loader/Restarter.pm#L55
I think something like this is a middle way:
next if basename($file) =~ m!\.(?:bak|swp|swpx|swx)$|~$|_flymake\.p[lm]$|\.#!;

@jhthorsen I mean it doesn't ignore files inside a directory with dot prefix, like .git. That regex looks horrible to me.

@jhthorsen makes a point about .git, you have to explicitly watch that directory for changes (or else watch .). Regarding the Plack regex, I said at the time that that didn't look very Mojo to me :smile:

@jhthorsen What are .swpx and .swx files anyway?

Yeah, I think the regex is a mess as well. Let's not make it more complicated than we have to, and currently users are complaining about dotfiles.
I'm +1 on the patch, since it's simple and makes morbo behave like it used before https://github.com/kraih/mojo/commit/ea8dc338c66c504d53b28dd1c4ba34783a6fc8a7#diff-52d3ad944c9afec616d8ccbcc2f1ce2eL48

I would be interested some performance metric if possible.

@jhthorsen I don't think that is true, it looks like the old code used to ignore directories with a dot prefix too.

Note that i'm not saying it should ignore directories like .git, just asking the question if it might be problematic not to do so. If users actually do -w ., then that's possibly thousands of additional files to check every second.

Quick benchmark for the Mojolicious repo on OS X with SSD.
~/repo/mojo $ perl -MMojo::Server::Morbo -Mojo -E 'my $m = Mojo::Server::Morbo->new(watch => ["."]); n { $m->modified_files } 10'
1.66039 wallclock secs ( 0.48 usr +  1.18 sys =  1.66 CPU) @  6.02/s (n=10)
~/repo/mojo $ find . | wc -l
    5048

Almost forgot, these are the results with @jhthorsen's patch applied. So there's a very noticeable performance cost.
~/repo/mojo $ perl -Ilib -MMojo::Server::Morbo -Mojo -E 'my $m = Mojo::Server::Morbo->new(watch => ["."]); n { $m->modified_files } 10'
2.49952 wallclock secs ( 1.30 usr +  1.20 sys =  2.50 CPU) @  4.00/s (n=10)

And if the dot prefix check gets moved behind the mtime/size check it jumps back to 6.02/s. I strongly recommend you optimize that part.

That's awful! :disappointed: Guess I've learned a lesson... I think that if speed is an issue, then we should't do so much stat()/for(). Here is a new idea:
diff --git a/lib/Mojo/Server/Morbo.pm b/lib/Mojo/Server/Morbo.pm
index 9217ea9..e6ed571 100644
--- a/lib/Mojo/Server/Morbo.pm
+++ b/lib/Mojo/Server/Morbo.pm
@@ -6,6 +6,7 @@ use Mojo::Base -base;
 #  Morbo: Morbo is pleased but sticky."
 use Mojo::Server::Daemon;
 use Mojo::Util qw(deprecated files);
+use File::Find ();
 use POSIX 'WNOHANG';

 has daemon => sub { Mojo::Server::Daemon->new };
@@ -20,16 +21,21 @@ sub check {

 sub modified_files {
   my $self = shift;
-
   my $cache = $self->{cache} ||= {};
   my @files;
-  for my $file (map { -f $_ && -r _ ? $_ : files $_ } @{$self->watch}) {
-    my ($size, $mtime) = (stat $file)[7, 9];
-    my $stats = $cache->{$file} ||= [$^T, $size];
-    next if $mtime <= $stats->[0] && $size == $stats->[1];
-    @$stats = ($mtime, $size);
-    push @files, $file;
-  }
+
+  no warnings;    # skip "Can't stat" warnings
+  File::Find::find(
+    sub {
+      return if /^\./ or -d _;
+      my ($size, $mtime) = (stat _)[7, 9];
+      my $prev = $cache->{$File::Find::name} ||= [$^T, $size];
+      return if $mtime <= $prev->[0] && $size == $prev->[1];
+      @$prev = ($mtime, $size);
+      push @files, $File::Find::name;
+    },
+    $_
+  ) for @{$self->watch};

   return \@files;
 }

Test case and benchmark:
$ for i in $(seq 0 5000); do touch "lib/x_$i"; done
$ find lib/ -type d | wc -l
      29
$ find lib/ -type f | wc -l
    5134
$ git checkout master
$ perl -Ilib -MMojo::Server::Morbo -Mojo -E 'my $m = Mojo::Server::Morbo->new(watch => ["lib"]); n { $m->modified_files } 50'
1.71442 wallclock secs ( 0.83 usr +  0.88 sys =  1.71 CPU) @ 29.24/s (n=50)
$ git checkout dotfiles
$ perl -Ilib -MMojo::Server::Morbo -Mojo -E 'my $m = Mojo::Server::Morbo->new(watch => ["lib"]); n { $m->modified_files } 50'
1.11585 wallclock secs ( 0.69 usr +  0.42 sys =  1.11 CPU) @ 45.05/s (n=50)

The difference is even bigger when watching ".", but that would be cheating since the patch above also skips directories starting with ., like .git.

New patch:
diff --git a/lib/Mojo/Server/Morbo.pm b/lib/Mojo/Server/Morbo.pm
index 9217ea9..879a925 100644
--- a/lib/Mojo/Server/Morbo.pm
+++ b/lib/Mojo/Server/Morbo.pm
@@ -5,7 +5,9 @@ use Mojo::Base -base;
 #         effects of sudden, intense global warming.
 #  Morbo: Morbo is pleased but sticky."
 use Mojo::Server::Daemon;
-use Mojo::Util qw(deprecated files);
+use Mojo::Util qw(deprecated);
+use File::Find ();
+use File::Spec::Functions 'splitpath';
 use POSIX 'WNOHANG';

 has daemon => sub { Mojo::Server::Daemon->new };
@@ -20,17 +22,21 @@ sub check {

 sub modified_files {
   my $self = shift;
-
   my $cache = $self->{cache} ||= {};
   my @files;
-  for my $file (map { -f $_ && -r _ ? $_ : files $_ } @{$self->watch}) {
-    my ($size, $mtime) = (stat $file)[7, 9];
-    my $stats = $cache->{$file} ||= [$^T, $size];
-    next if $mtime <= $stats->[0] && $size == $stats->[1];
-    @$stats = ($mtime, $size);
-    push @files, $file;
-  }

+  my $wanted = sub {
+    return if -d $_;
+    my ($size, $mtime) = (stat _)[7, 9];
+    my $prev = $cache->{$File::Find::name} ||= [$^T, $size];
+    return if $mtime <= $prev->[0] && $size == $prev->[1];
+    return if (splitpath $File::Find::name)[2] =~ /^\./;
+    @$prev = ($mtime, $size);
+    push @files, $File::Find::name;
+  };
+
+  no warnings;    # skip "Can't stat" warnings
+  File::Find::find({no_chdir => 1, wanted => $wanted}, $_) for @{$self->watch};
   return \@files;
 }

Benchmark on 5163 files with n=50:

with patch: 1.2915s
without patch: 1.74053s

Here is more information from the discussion on IRC: http://irclog.perlgeek.de/mojo/2016-03-14#i_12180034

Just wanted to mention that @jhthorsen's latest patch should also result in the deprecation of Mojo::Util::files if it gets chosen.

This should really be two issues, the first about ignoring dotfiles efficiently with the current code, and the second about optimizing Morbo by deprecating Mojo::Util::files.

Is there no way to ignore file via pattern?
like lib/Foo/Controller/Bar_flymake.pm
is this wrong place to ask?


crypto/internal/fips/aes/gcm: TestAllocations fails on PPC64

Go version
master
Output of go env in your module/workspace:
GOARCH=ppc64le

What did you do?
cd crypto/internal/fips/aes/gcm
go test
What did you see happen?
--- FAIL: TestAllocations (0.00s)
ctrkdf_test.go:31: expected zero allocations, got 6.0
What did you expect to see?
All tests pass

@golang/ppc64


store sd_hash in File table

If a claim file is uploaded outside of speech it will, when viewed on speech, be downloaded and stored with name, claimId, and outpoint. Speech needs re-download the file when it has changed. But outpoint will be updated when metadata is updated but not the file. So we should store the sd_hash in the File table. When it matches chainquery, serve it. When it does not match, re-download.
Internal Use
Acceptance Criteria

Definition of Done

[ ]  Tested against acceptance criteria
[ ] Tested against the assumptions of the user story
[ ] The project builds without errors
[ ] Unit tests are written and passing
[ ] Tests on devices/browsers listed in the issue have passed
[ ] QA performed & issues resolved
[ ] Refactoring completed
[ ] Any configuration or build changes documented
[ ] Documentation updated
[ ] Peer Code Review performed

Related to https://github.com/lbryio/spee.ch/issues/418

moving away from localdb toward chainquery and sdk, this is unnecessary.


Add Storage Policy Management Service to VMwareWebService Gem

In vSphere 5.5 VMware added a new SDK to the vSphere Management SDK to handle Storage Policy Based Management (SPBM/PBM).  This allows administrators to tag their datastores with capabilities to ensure that VM workloads match storage capabilities, and integrates with VSAN and VVOL.
Storage Policy Based Management Overview
[Storage Policy SDK Documentation] (http://pubs.vmware.com/vsphere-60/index.jsp?topic=/com.vmware.vspsdk.apiref.doc/index.html&single=true)
VMware Ruby Gem
This PR adds a PbmService to the VMwareWebService gem so that PbmProfiles can be retrieved and their associations to other Managed Objects queried.  It is using the RbVmomi gem which has bindings for the PBM SDK already.
Depends on: ~~https://github.com/ManageIQ/handsoap/pull/3~~

@miq-bot add-label providers/vmware, enhancement, wip

@jameswnl @roliveri can you review?

Wonder your opinions on these methods returning a RbVmomi::PBM class not a hash, e.g.:
ExtManagementSystem.first.connect.pbmProfiles
=> {"c0dfac92-5993-477f-89ed-f64865f259cc"=>#<RbVmomi::PBM::PbmCapabilityProfile:0x000000029bb2a0 ... >}

Think this is okay or should convert it to a hash before returning to the caller?

LGTM (except do we care about the rubocop's opinion about snake case naming?)

@miq-bot remove-label wip

@roliveri handsoap has been tagged so we can use the session cookie, can you take another look?


updating netcore and variables for AspNetCore.Tests

Fix Issue #1769

/AzurePipelines run

Hey Eddy, can you confirm that this compiles locally?
You're getting build errors:

Microsoft.ApplicationInsights.AspNetCore.Tests.csproj(0,0): Error NU1605: Detected package downgrade: Microsoft.Extensions.Configuration.Json from 2.1.0 to 1.0.2. Reference the package directly from the project to select a different version.

/AzurePipelines run

/AzurePipelines run


Use U2D2 to controll OM failed

As title
My environment is PC(Ubuntu16.04 + kinetic) + U2D2 + SMPS2DXL + OM
What I already do :

set servos ID 11 to 15 and baudrate set 1000000mbps.
USB Latency Timer Setting finish

When I run roslaunch open_manipulator_controller open_manipulator_controller.launch
Show this error

But when I use old USBtoDynamixel, it can success work

hi @zhl017
Thanks for using OM.
We need more information about your situation.
I think that problem may be due to U2D2.
Please check again that dynamixel successfully work in your hardware environments, by using the dynamixel wizard 2.0.
http://emanual.robotis.com/docs/en/software/dynamixel/dynamixel_wizard2/

yes, it can work at wizard 2.0.
I use two U2D2, both happened this problem.
by the way, I use XM430-W350-R instead XM430-W350-T

@zhl017
Would you test if this work ?
Regards,
Ryan

good to work ...

@zhl017
Please test the next one (5.1.2) to see if the motor controller works properly.
You need to set your motor ids to 1,2 (you need two motors for this example)
Regard,
Ryan

Can't work and I didn't see /dynamixel_command..
I clone wrong package ??

@zhl017
Please do the followings

Update your motor firmware with DXL-Wizard
Download the latest version of WB package

and let me know if it still doesn't work.
Regards,
Ryan

I use git clone https://github.com/ROBOTIS-GIT/dynamixel-workbench.git yesterday
I think it's the latest version.
and my motor firmware also the latest

@zhl017
Would you test if this tutorial works?
Regards,
Ryan

sorry for reply late.
I try sync_read_write C++ example
Motor can move one times and then show failed.

@zhl017

Would you connect your U2D2 to a USB hub and connect to it to your computer, and test it again?
Check if you really changed USB latency](http://emanual.robotis.com/docs/en/platform/openmanipulator_x/ros_setup/#usb-latency-timer-setting) and test it again.
If it still doesn't work, the problem might be on your U2D2, so just buy a new one.

Regards,
Ryan

So strange, I change another PC, it can work now !!
thanks.


Unable to hear audio in a mp4 file

I created an mp4 file with audio but when I start recording I only hear about 1 millisecond of sound and then silence.
The video is still playing but no audio can be heard from that initial blip.

Thanks for raising the issue.
The only workaround I currenly know of is to split the audio from the video, and play it using a separate Audio track.
Cause of the issue:
Out of the box Unity's VideoPlayer is not compatible with Unity Recorder (the video plays at incorrect speeds). As a fix I made a modification to use VideoPlayer.SetForward each frame rather than VideoPlayer.Play here. While this fixes the video playback speed, it breaks the audio.

Ok, that sounds good. Does seem in keeping with Unity's struggles with Video playback. I'll keep with the workaround for now.


egg/setup.py

I just had a setup problem :
when testing a site just deployed using a "pip install -r requirements.txt", I had an error stating (more or less) that
Choices has no REVERTED_CHOICES_CONST_DICT attribute.

In my requirements.txt file I have :
-e git://github.com/twidi/django-extended-choices.git#egg=extended_choices

I uninstalled the egg, and installed it again, but not as an egg :
pip uninstall django-extended-choices

and it worked again. So there must be a file missing when installed with "-e" option. Sorry I cannot investigate more right now, I'll search on the next install....

Could you provide the full stack trace (or at least the interesting parts ?)
Thanks for your report !

j'ai donc eu l'erreur à nouveau, fait une réinstallation depuis github avec pip, avec et sans option -e et là c'est bon
y'a du avoir un bug à la premiere installation...

Ok alors. J'avoue ne pas connaître très bien le packaging python


Remove the system limitation and check the generic solution

Then it is clear that @Forien was a dragon with this code because even without the item properties the basic functionalities of the module work with the "generic" case. So to fix the problem No longer works on PF1E (and also for backward compatibility) i removed the tag system from the module.json.
This was a miss from my side sorry @Varriount .
All the other bugs remain to be fixed ... and the update to fvtt10...
P.S. @Varriount  i didn't apply the new prettier configuration just to point out that I haven't touched any of the code other than removing some commented code,  If you like, you can apply it before the release

Feel free to make the Prettier PR at any time - since it doesn't actually affect functionality, it doesn't matter whether it's included in a release.


Support ctrl + n/k for navigating items

This should work even if the search input is focused

Fixed by 55dd7ad.


isuue with mcool2hic

Hello, I could not find the code and tutorials for mcool2hic. Could you please provide some more detailed information?
Any suggestion would be greatly appreciated!

Hi there--sorry for such a slow reply!
If you're still curious: mcool2hic was removed from the Hi-C pipeline in v43, which is why you're having a tough time learning more about it. If you would like to see more, you can view the file in branch v42.4 (or another earlier version). Just be aware that the current version of the pipeline will look quite different than before v43.
Thank you for asking! I should remove the file from the current documentation to avoid confusion.


As a  subscriber, I should can subscribe multiple type of messages from publisher

Describe the bug
As a  subscriber, I can subscribe to multiple types of messages from the publisher
Additional context
There are two examples of Port

https://github.com/slawlor/ractor/blob/main/ractor/examples/output_port.rs
https://docs.rs/ractor/latest/ractor/port/output/type.OutputPortSubscriber.html#example

I am not sure how to start with my implementation, I want to subscribe to multiple messages from publisher.
Many thanks, I'm a newbie, and maybe I didn't study your code thoroughly, but a better example would make it easier for everyone to get started because 2 different examples make me a little confused.

It seems that the subscribe message must impl From trait, but in my case, the publisher pubs the message to the subsribler directly, is it possible with the current code?

It seems that the subscribe message must impl From trait, but in my case, the publisher pubs the message to the subsribler directly, is it possible with the current code?

well, it can be done by filtering message enum type.


[Feature request] Validate Docker registry credentials before proceeding

Expected Behaviour

We can validate that the Docker registry credentials passed by a user, have an entry for the "registry" that they specified in the init.yaml
The other thing we can check is that they are not using a credStore
Current Behaviour

User will eventually work it out by troubleshooting error messages. i.e @doowb has this issue.
Possible Solution

Create a struct for the Docker config JSON
Parse it
Write a unit test for the invalid case
Write a unit test for the valid case
Integrate into the main CLI and show evidence of it passing/failing.

https://blog.alexellis.io/golang-writing-unit-tests/

Normally ~/.docker/config.json - there might be some code in openfaas/faas-cli that could help with parsing the file, if we can avoid adding a dependency to Docker it would be better.

I am working on this. @alexellis

No problem

@rajibmitra - Any updates on this? I got stung by this today (using macos keychain)

@Waterdrips please take this.
We should be able to do this without vendoring code from Docker, just with a simple struct to see if th registry as a value in base64.


Make clear asciidoctor-epub3 doesn't work under Ruby 2.5.0 (at least for now)

Greetings,
As issue #153 indicates, newcomers do install Ruby 2.5.0 (current version), but, asciidoctor-epub3 gem will not be installed under this newer version of Ruby. it'll be labor-saving for them to state clearly in the README.adoc  file that this is a known limitation for now.
(@mojavelinux Thank you for your great work! respect :bowing_man:)

This would be a nicer treatment. Thanks.

I think you'll find that Asciidoctor EPUB3 install just fine on Ruby 2.5. See https://travis-ci.org/asciidoctor/asciidoctor-epub3/builds/515662808
The nokogiri dependency is pulled in by gepub. So in order to upgrade it, we need to upgrade gepub. But for some reason, gepub is now 10MB!

Regardless, I'm going to say that this issue is about upgrade gepub. That should also eliminate warnings.

The point release of gepub (1.0.3) is now only 37 KB. If you reinstall your gems, you should get this one instead.


fix canvas tag syntax errors

Sometimes <canvas> has no closing angle bracket, sometimes it has two. Sometimes it is invalid just because there's no html5 doctype.

I made all the changes. Since I have to make them in a couple of places, I made them myself rather than use the pull.
Is there a browsers that was complaining about these errors?
Thanks again.

Ed Angel
Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico
1017 Sierra Pinon
Santa Fe, NM 87501
505-984-0136 (home<EMAIL_ADDRESS><EMAIL_ADDRESS>505-453-4944 (cell) 				http://www.cs.unm.edu/~angel http://www.cs.unm.edu/~angel

On Jul 18, 2015, at 12:09 PM, Anton Khorev<EMAIL_ADDRESS>wrote:
Sometimes  has no closing angle bracket, sometimes it has two. Sometimes it is invalid just because there's no html5 doctype.
You can view, comment on, or merge this pull request online at:
https://github.com/esangel/WebGL/pull/5 https://github.com/esangel/WebGL/pull/5
Commit Summary
fix canvas tag syntax errors
File Changes
M Chap10/mandelbrot1.html https://github.com/esangel/WebGL/pull/5/files#diff-0 (3)
M Chap10/mandelbrot2.html https://github.com/esangel/WebGL/pull/5/files#diff-1 (3)
M Chap3/cad1.html https://github.com/esangel/WebGL/pull/5/files#diff-2 (3)
M Chap3/cad2.html https://github.com/esangel/WebGL/pull/5/files#diff-3 (3)
M Chap3/square.html https://github.com/esangel/WebGL/pull/5/files#diff-4 (2)
M Chap3/squarem.html https://github.com/esangel/WebGL/pull/5/files#diff-5 (2)
M Chap3/triangle.html https://github.com/esangel/WebGL/pull/5/files#diff-6 (2)
M Chap4/cube.html https://github.com/esangel/WebGL/pull/5/files#diff-7 (3)
M Chap4/cubeq.html https://github.com/esangel/WebGL/pull/5/files#diff-8 (3)
M Chap4/cubev.html https://github.com/esangel/WebGL/pull/5/files#diff-9 (3)
M Chap4/trackball.html https://github.com/esangel/WebGL/pull/5/files#diff-10 (3)
M Chap4/trackballQuaterion.html https://github.com/esangel/WebGL/pull/5/files#diff-11 (3)
M Chap7/cubet.html https://github.com/esangel/WebGL/pull/5/files#diff-12 (3)
M Chap9/figure.html https://github.com/esangel/WebGL/pull/5/files#diff-13 (3)
M Chap9/robotArm.html https://github.com/esangel/WebGL/pull/5/files#diff-14 (3)
Patch Links:
https://github.com/esangel/WebGL/pull/5.patch https://github.com/esangel/WebGL/pull/5.patch
https://github.com/esangel/WebGL/pull/5.diff https://github.com/esangel/WebGL/pull/5.diff
—
Reply to this email directly or view it on GitHub https://github.com/esangel/WebGL/pull/5.

Browsers weren't complaining for those particular files, but they could start unexpectedly. In case of missing angle bracket, every word of "Oops" message is interpreted as a canvas attribute.

It’s good to get them fixed.
I also have a long list of other little things that someone else put in a pull request for that includes indentation changes, extra white space and the like. Those may have to wait until the course is over.

Ed Angel
Founding Director, Art, Research, Technology and Science Laboratory (ARTS Lab)
Professor Emeritus of Computer Science, University of New Mexico
1017 Sierra Pinon
Santa Fe, NM 87501
505-984-0136 (home<EMAIL_ADDRESS><EMAIL_ADDRESS>505-453-4944 (cell) 				http://www.cs.unm.edu/~angel http://www.cs.unm.edu/~angel

On Jul 18, 2015, at 6:26 PM, Anton Khorev<EMAIL_ADDRESS>wrote:
Browsers weren't complaining for those particular files, but they could start unexpectedly. In case of missing angle bracket, every word of "Oops" message is interpreted as a canvas attribute.
—
Reply to this email directly or view it on GitHub https://github.com/esangel/WebGL/pull/5#issuecomment-122608254.


[Bug] Import-DbaCsv prompts for confirmation and throws exception

Environmental information
#### PowerShell version:

Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Darwin 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

#### dbatools Module version:

Name    : dbatools
Path    : /Users/craig/.local/share/powershell/Modules/dbatools/1.0.104/dbatools.psd1
Version : 1.0.104

SQL Server:
Microsoft SQL Server 2017 (RTM-CU17) (KB4515579) - 14.0.3238.1 (X64) 	Sep 13 2019 15:49:57 	Copyright (C) 2017 Microsoft Corporation	Developer Edition (64-bit) on Windows Server 2016 Datacenter 10.0 <X64> (Build 14393: ) (Hypervisor)

Report
Host used

[ ] powershell.exe
[ ] ISE
[X] VS Code
[ ] Other (please specify)

If anything other than powershell.exe was used, please confirm that you can duplicate the issue with powershell.exe

[X] Still buggy in powershell.exe

Errors Received

PSMessageDetails      :
Exception             : Microsoft.PowerShell.Commands.WriteErrorException: Unable to find type [System.Data.SqlClient.SqlBulkCopyExtension].
TargetObject          :
CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Import-CsvFile.ps1
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, /Users/craig/.local/share/powershell/Scripts/LorenzETL/ETL/Import-CsvFile.ps1: line 144
                        at <ScriptBlock>, /Users/craig/.local/share/powershell/Scripts/LorenzETL/ETL/Import-CsvFile.ps1: line 131
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {0, 1}

Steps to Reproduce
Using this code:
$Table = 'EventLog'
$Columns = @{
    Computer = 'Computer'
    User = 'User'
    Source = 'Source'
    Timestamp = 'Created'
    LogLevel = 'LogLevel'
    Message = 'Message'
}

Get-ChildItem -Path $LogPath -File -Filter "*.log" | ForEach-Object {
  Import-DbaCsv -Path $_ -BatchSize 50000 -Table $Table -ColumnMap $Columns
}

Expected Behavior
The cmdlet imports the data without incident.
Actual Behavior
I was prompted for confirmation when Import-DbaCsv was reached:
Confirm
Continue with this operation?
[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"): y

In addition, the cmdlet threw a terminating error: Write-Error: Unable to find type [System.Data.SqlClient.SqlBulkCopyExtension]..
Remedies

Setting the Confirm parameter had no effect: Import-DbaCsv -Path $_ -BatchSize 50000 -Table $Table -ColumnMap $Columns -Confirm:$false
Setting the ConfirmPreference had no effect: $ConfirmPreference = 'Low'

I'm not able to replicate this issue.
Get-WinEvent -LogName Application -MaxEvents 5 | Export-Csv -Path c:\temp\testdata.csv -NoClobber -NoTypeInformation
Import-DbaCsv -Path C:\temp\testdata.csv -SqlInstance lab-sql-01 -SqlCredential $labAdminSqlCred -Database tempdb -Table whatever -AutoCreateTable

In VS Code as well:

This would be specific to the Confirm issue. We have the command set to Low so it won't actually pass that prompt to you.
On the issue around the SqlBulkCopy, that is already been reported here #6460 and is a known issue with our module in PS 7.

I'm calling Import-DbaCsv from within a script that has CmdletBinding.
CSV.ps1
[CmdletBinding()]
param ()

Import-Module dbatools

$Credential = Get-Credential

$Path = "~/Desktop/timezones.csv"
# create file
Get-TimeZone -ListAvailable | Export-Csv -Path $Path -NoTypeInformation

# import file
Import-DbaCsv -Path $Path -SqlInstance <IP_ADDRESS> -Database Custom -Table timezones -SqlCredential $Credential -AutoCreateTable

When I call the script,  no issues:
PS 7.0 [xxx] /Users/xxx/Desktop>./CSV.ps1 -debug
DEBUG: 38649 | [07:51:55][Import-DbaCsv] Started at 04/09/2020 07:51:55
WARNING: [07:51:55][Import-DbaCsv] Could not load a usable version of SqlBulkCopy. | error CS1702: Assuming assembly reference 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'System.Data.SqlClient' matches identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Runtime', you may need to supply runtime policy
DEBUG: [07:51:55][Import-DbaCsv] Could not load a usable version of SqlBulkCopy. | error CS1702: Assuming assembly reference 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'System.Data.SqlClient' matches identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Runtime', you may need to supply runtime policy

Confirm
Continue with this operation?
[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"):
PS 7.0 [xxx] /Users/xxx/Desktop>./CSV.ps1
WARNING: [07:53:34][Import-DbaCsv] Could not load a usable version of SqlBulkCopy. | error CS1702: Assuming assembly reference 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'System.Data.SqlClient' matches identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Runtime', you may need to supply runtime policy
InvalidOperation: /Users/craig/.local/share/powershell/Modules/dbatools/1.0.104/allcommands.ps1:38966
 Line |
38966 |  …    $rowscopied = [System.Data.SqlClient.SqlBulkCopyExtension]::RowsCo …
      |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      | Unable to find type [System.Data.SqlClient.SqlBulkCopyExtension].

ComputerName  : AZURE-xxx
InstanceName  : MSSQLSERVER
SqlInstance   : AZURE-xxx
Database      : Custom
Table         : timezones
Schema        : dbo
RowsCopied    :
Elapsed       : 1.03 s
RowsPerSecond : 0
Path          : /Users/craig/Desktop/timezones.csv

When I call the script with -Debug, however, the prompting occurs:
PS 7.0 [xxx] /Users/xxx/Desktop>./CSV.ps1 -Debug
DEBUG: 38649 | [07:53:45][Import-DbaCsv] Started at 04/09/2020 07:53:45
WARNING: [07:53:45][Import-DbaCsv] Could not load a usable version of SqlBulkCopy. | error CS1702: Assuming assembly reference 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'System.Data.SqlClient' matches identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Runtime', you may need to supply runtime policy
DEBUG: [07:53:45][Import-DbaCsv] Could not load a usable version of SqlBulkCopy. | error CS1702: Assuming assembly reference 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' used by 'System.Data.SqlClient' matches identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' of 'System.Runtime', you may need to supply runtime policy

Confirm
Continue with this operation?
[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"):

Maybe this is the expected behavior with -Debug.

I don't follow because both of your examples include the SqlBulkCopy error.....

The error isn't the issue, the prompting is:
Confirm
Continue with this operation?
[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"):

Prompting only occurs when -Debug flag is used when calling the script.

That is by design and how the Debug parameter works in PowerShell.

OK.  My mistake.


chapter 5, exercise 8, scaler.inverse_transform

hi
I don't understand how this line of code work and why -10 to 10 range is used. would you please explain it to me.
thanks.
line1 = scaler.inverse_transform([[-10, -10 * w1 + b1], [10, 10 * w1 + b1]])

Hi @ahmadkhalaj ,
Thanks for your question.
When we discuss petal length and width, we want to talk in centimeters. But for SVMs to work correctly, we need to scale the input features so that they have mean=0 and variance=1, so the SVC does not deal with centimeters, it deals with some weird units, let's call them weirdometers. The slopes and intercepts of the decision boundaries, w1, b1, w2, b2,  w3 and b3, are for inputs expressed in weirdometers.
If we want to display the decision boundaries in a graph expressed in centimeters, we need to apply the inverse transformation of the scaler (instead of going from centimeters to weirdometers, it does the reverse). To draw each line, we need two points, and I chose -10 and 10 (on the horizontal axis, expressed in weirdometers) because that's definitely going to be larger than the width of the diagram, so it will not draw a partial line that stops halfway through the diagram. If the mean petal length is (say) 3 cm, and the standard deviation is (say) 2cm, then 10 weirdometers corresponds to 3 + 10 * 2 = 23 cm, and -10 weirdometers corresponds to 3 - 10*2 = -17 cm. This will definitely be large enough.
Hope this helps!
The range -10 to 10 was just chosen to be larger than the range of the diagram (left to right).

Also forgot to mention that if an iris is on the decision boundary, and its petal length is 10 weirdometers, then its petal width must be 10 * w1 + b1.  w1 is the slope of the decision boundary, and b1 is the intercept, and both are expressed in weirdometers.

Hi @ahmadkhalaj ,
Thanks for your question.
When we discuss petal length and width, we want to talk in centimeters. But for SVMs to work correctly, we need to scale the input features so that they have mean=0 and variance=1, so the SVC does not deal with centimeters, it deals with some weird units, let's call them weirdometers. The slopes and intercepts of the decision boundaries, w1, b1, w2, b2, w3 and b3, are for inputs expressed in weirdometers.
If we want to display the decision boundaries in a graph expressed in centimeters, we need to apply the inverse transformation of the scaler (instead of going from centimeters to weirdometers, it does the reverse). To draw each line, we need two points, and I chose -10 and 10 (on the horizontal axis, expressed in weirdometers) because that's definitely going to be larger than the width of the diagram, so it will not draw a partial line that stops halfway through the diagram. If the mean petal length is (say) 3 cm, and the standard deviation is (say) 2cm, then 10 weirdometers corresponds to 3 + 10 * 2 = 23 cm, and -10 weirdometers corresponds to 3 - 10*2 = -17 cm. This will definitely be large enough.
Hope this helps!

Thanks a lot.  It was so helpful.


Integrate compilation and verification

Continues #27.
Combines code compilation and its verification. Add tests for the whole process

I don't like name for handlers.rs. It's not a handler. (in out code style hander is function that passes to actix)
maybe change filename and add word bruteforce to compile_and_verify_handler function? like compile_and_verify_bruteforce, or something else

I don't like name for handlers.rs. It's not a handler. (in out code style hander is function that passes to actix)
maybe change filename and add word bruteforce to compile_and_verify_handler function? like compile_and_verify_bruteforce, or something else


How could I find file "demographic_HH.csv"?

How could I find file "demographic_HH.csv"?

Hi there,
the file is generated by the data download script in this line:
https://github.com/DLTK/DLTK/blob/master/data/IXI_HH/download_IXI_HH.py#L133
Please refer to the readme file https://github.com/DLTK/DLTK/blob/master/README.md, "Start Playing" point 1. If you have questions, please ask them in our gitter chat.
Closing this, as it is not an issue.


Cause Detail Campaign Cell

Adds and displays a LDTCauseDetailCampaignCell - refs #712

Funky bug when a campaign doesn't have a cover image. (Never really happens on prod, but happens for a few campaigns on staging)


Using select inputs for filtering data does not include all expected values

Summary of problem or feature request
When using select inputs for filtering data, the select input shows only the unique items that are in the first screen, ignoring other values that can be in next pages
Code snippet of problem
Let's suppose I have 50 records, with a field "status" that can be "on" or "off".
The first 48 records have status "on", the last 2 have status "off".
I have 10 records per page.
The select box shows only the "on" status due to the LIMIT 10 feature of SQL query, ignoring that on the last page (LIMIT 10 OFFSET 40) there would be an "off" value to include in the select box.
Is there a workaround for this? (i.e. retrieving all data at once via SQL, thus removing LIMIT, and let javascript paginate)
Thank you :-)
System details

Operating System: Windows 10
PHP Version: 7.2.2
Laravel Version: 5.5
Laravel-Datatables Version: "yajra/laravel-datatables": "1.0"

The demo only works on client-side implementation. Similar to https://github.com/yajra/laravel-datatables/issues/1690, please see my propose workaround there. Thanks!


Authconfiguration Documentation outdated?

I tried to configure social logins
http://yawik.readthedocs.org/en/latest/installation.html#authentication
the mentioned file modules/Auth/config/module.auth.local.php does not exists

I've updated the documentation
http://yawik.readthedocs.org/en/latest/installation.html#authentication


SINGLY LINKED LIST IN C++

DESCRIPTION

What is a Linked list!
How can it be differentiated from array.
Its advantages..
Programs related to insertion, deletion ,searching and reversing a linked list.

Domain
C/CPP
Type of Contribution
Documentation, Others
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign

/assign

/assign


Back link should preserve the previous selected option when used

This issue is from May 2023 external accessibility audit report.
Usability issue
Issue ID:
DAC_Usability_Question_Pages_01
URLs:
https://signin.account.gov.uk/get-security-codes
Screen shots

After a user goes back to the ‘Choose how to get security codes’ page, the page is not in the state that they last saw it. The users selected radio button has been unselected, meaning the user must answer the question again in order to continue in the service.
Current code ref(s)
#mfaOptions
<input class="govuk-radios__input" id="mfaOptions" name="mfaOptions" type="radio" value="SMS">

User comments
No user comments submitted in the report
Solution
The GOV.UK Design System states that “Make sure it takes users to the previous page they were on; in the state they last saw it”.
When a user goes back to the previous page, their chosen option should be still selected.
Redundant entry mentions that; information previously entered by or provided to the user that is required to be entered again in the same process is either:

auto-populated, or
available for the user to select.

Except when:

re-entering the information is essential,
the information is required to ensure the security of the content, or
previously entered information is no longer valid.

It is currently unclear if this relates to a user navigating back pages to previously completed forms.
As WCAG 2.2 is still yet to be finalised it is currently unclear if this issue would fail this criterion.

Hey @CharlotteDowns I'm thinking there's not much to do on this issue, if we've contacted the One Login team.
We already have guidance that covers this, under the 'How it works' section of the 'back link' page. It seems decently well-placed in the guidance as well, in my opinion.
CC @claireashworth for a second content opinion perhaps?
Would it be reasonable to close this without further action?

We did have a live service implement the back link without the correct back-linking, which is how we got this audit result. However, it's not clear if that service team ever read our guidance on back links, or just used the code

The team who ran this service has solved the issue here: https://github.com/alphagov/di-authentication-frontend/pull/1075
Closing this ticket as completed!


Use autotune for MALA in RunManager Default

Use autotune for MALA in RunManager can allow the user not to input a mass matrix for the MALA local sampler, which could simplify the setup.

Currently, the autotune that is implemented only tunes the step size before the flowMC training loop is started and no tuning is done during training the NF. This can be problematic if the autotune has a goal local acceptance of 0.3 -- 0.5, since the local acceptance can drop during sampling, see for instance this example run. Also, the optimal acceptance rate of MALA seems to be 0.574 according to some sources such as [1] and [2] although these might be theoretically derived and therefore not of much relevance for GW PE.
It might be good, if desired, to also tune during the sampling stage as well, e.g. after each loop training loop iteration or after a few cycles. An attempt at this which seemed rather successful back then can be found (in an older flowMC version) here. The idea I had there was to multiply the MALA mass matrix with a factor gamma_T which is adapted based on the current mean local acceptance and the desired/target local acceptance. However, this is a bit of a hacky way to do it since the gamma_T multiplier has to be carried around over flowMC: simply making an attribute of e.g. Sampler did not work for me due to JAX being JAX, but perhaps others with more experience (@kazewong) could fix that.
[1] https://www.jstor.org/stable/2985986
[2] https://www.sciencedirect.com/science/article/pii/S0304414907002177


Add-ServiceNowAttachment not working on PowerShell 7.1.3

Hello,
I am seeing issues with using Add-ServiceNowAttachment on PowerShell 7.1.3. I get this error for seemingly any attachment on a TASK:
Add-ServiceNowAttachment -Number $Task -Table "sc_task" -File $TempFilePath

InvalidOperation: C:\Users\nscherer\Documents\PowerShell\Modules\ServiceNow\2.2.0\Public\Add-ServiceNowAttachment.ps1:118
Line |
 118 |                  $ContentType = [System.Web.MimeMapping]::GetMimeMappi …
     |                                 ~~~~~~~~~~~~~~~~~~~~~~~~
     | Unable to find type [System.Web.MimeMapping].

I have tested and this works in Windows PowerShell 5.1 on the same machine. I suspect this is because System.Web.MimeMapping is not available in .NET Core.
Environment
Operating System: Windows 10 1909
ServiceNow module version: 2.2.0
PowerShell version: 7.1.3

Steps to reproduce
Try to attach an attachment via Add-ServiceNowAttachment.
Expected behavior
Attachment attaches to TASK.
Actual behavior
Cmdlet errors out.

PR #132 has been created to fix this.


LUIS connectivity first attempt

Hi,
I'm getting the following exception when trying to use LuisDialog.

A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond <IP_ADDRESS>:443

Bot.Builder version 3.0.1
Emulator version <IP_ADDRESS>
any suggestion what can be done and what is going on?

Have you set the right subscription key and model id for your LuisDialog? The error looks like a timeout error because of networking issue. Can you check if you can query your luis.ai model directly from your computer?

Hi,
Thanks for your reply!
I've copied the exact key and model id from the LUIS configuration page, also verified against the rest URL that is generated when publish.
I'm not sure what you mean by directly from my computer. If you mean pasting the rest address to a browser with search key, then I did that and it works as expected.
Any additional information I should look for? Anything else that can help pinpointing the source of the issue?
Thanks again

Im not sure whether you tried that, but to rule out that your PC is the culprit:
Have you tried publishing the bot to azure and then trying out, whether its able to connect to luis?
If it can, then it has something to do with your local machine.

@vvvlad do you use a web proxy?

Thanks for your replies.
@NanashiGIT I don't have that option right now
@willportnoy Not sure what web proxy is. I do work inside enterprise network and browser has proxy configuration. Can the Emulator be configured to work with proxy?
I've also tried to configure proxy as a global system variable (for example it helps to work with Python packages), but that didn't help either.

Hi @vvvlad are you still having this issue?

I discovered it was not supposed to work :(
Thanks again for all the support!


Use LazyChoiceField to speed up loading of PageFields

Summary

Fixes #
Links to related discussion
Proposed changes in this pull request

This one wasn't meant to go here.


add write.as, writefreely and Matrix

Open source, federated blogging platforms.
These are almost the same services, with "write.as" being the commercial one and "writefreely" the open source equivalent.
See:

https://writefreely.org/
https://write.as/

Matrix is a messaging system.
See: https://matrix.org/


updating bug example

I have been reporting bugs using this page and it's working, fixed by Torre 3 of 3, this is my 4th bug to report.

New bug found and reported in about.


Refresh service quota limits (by 9/30/2022)

User Story
As an application developer, I want to be able to maximize the throughput of my log processing using Incite.

Therefore I want Incite to have the most service quota limits for the CloudWatch Logs Insights API.
Furthermore, if my service quota limit is "soft" (can be increased on request to AWS), I want to be able to consume all of my increased capacity without being limited by Incite.

Notes
The CloudWatch Logs Service Quota Limits page is here. Check if things have changed.
As of v1.3.2, Incite has:

Hard RPS limits of 5 requests per second (cloudwatchlogs_actions.go)
Hard concurrency limit (Config.Parallel) of QueryConcurrencyQuotaLimit = 10 (incite.go)

Note that as of 2022-07-26, Service Quota Limits page says the query quota limit can be INCREASED on request to AWS support. Therefore Incite's hard limit approach to concurrency is too limiting and should be fixed.

Query concurrency limit has been increased from 10 to 20 in the last few days.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html

Updated value of QueryConcurrencyQuotaLimit released as part of v1.3.3.
Next step is to fix the QueryManager construction so it allows you to exceed QueryConcurrencyQuotaLimit and if you do, it just logs an info message to the logger...


Missing function to set WPM (Morse::gpio_set_wpm)

Hi Tracey,
Thanks a lot for this nice piece of code, I like a lot!
I'm developing a simple ESP32 based CW beacon:

Your library just fits perfectly for this purpose. I've been missing only a function to change the WPM anytime I want. I added one that works for me:
Morse::gpio_set_wpm(uint8_t the_wpm) {
 			gpio_wpm = the_wpm;
 			gpio_unit_t = UNIT_T(gpio_wpm);
 			gpio_inited = 1;
 }

Thanks again also for RPI Pico version of it...
VY 73 AND HNY de Pavol OM0WT

Thank you. Makes sense to add this. Happy to add your code with a PR.

Code merged. Thanks!


install fails on Amazon Linux 2

Environment

Python:
Python 3.7.6
pyodbc:
Cheese Factory latest
OS:
Amazon Linux 2
DB:
NA
driver:
NA

Issue
pyodbc simply won't install on my Amazon Linux 2 server.
pip3 install pyodbc

Collecting pyodbc
  Using cached https://files.pythonhosted.org/packages/81/0d/bb08bb16c97765244791c73e49de9fd4c24bb3ef00313aed82e5640dee5d/pyodbc-4.0.30.tar.gz
Installing collected packages: pyodbc
  Running setup.py install for pyodbc ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-19y7mmge/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-k1aqu9nj-record/install-record.txt --single-version-externally-managed --compile:
    /usr/lib64/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)
    running install
    running build
    running build_ext
    building 'pyodbc' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPYODBC_VERSION=4.0.30 -I/usr/include/python3.7m -c src/buffer.cpp -o build/temp.linux-x86_64-3.7/src/buffer.o -Wno-write-strings -DHAVE_UNISTD_H -DHAVE_PWD_H -DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG_INT=8 -I/usr/include
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-19y7mmge/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-k1aqu9nj-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-19y7mmge/pyodbc/

I've read in some old support issues here about needing python3-dev and unixodbc-dev for Ubuntu by Amazon Linux 2 is definitely rhel like.  Here are some dependency checks.  That error message above looks pretty terse to me as well.  If there's a better way to look into the error, please let me know.
[ec2-user@tableau_master ~]$ yum list installed | grep python3
Repository packages-microsoft-com-prod is listed more than once in the configuration
python3.x86_64                     3.7.6-1.amzn2.0.1                 @amzn2-core
python3-libs.x86_64                3.7.6-1.amzn2.0.1                 @amzn2-core
python3-pip.noarch                 9.0.3-1.amzn2.0.2                 @amzn2-core
python3-setuptools.noarch          38.4.0-3.amzn2.0.6                @amzn2-core

[ec2-user@tableau_master ~]$ yum list installed | grep unix
Repository packages-microsoft-com-prod is listed more than once in the configuration
unixODBC.x86_64                    2.3.1-14.amzn2                    @amzn2-core
unixODBC-devel.x86_64              2.3.1-14.amzn2                    @amzn2-core

That error unable to execute 'gcc': No such file or directory means it looks like you don't have the "gcc" c++ compiler available.  I'm guessing that needs to be installed, which can probably been done as per the install instructions for CentOS.  (Amazon Linux is based on Red Hat which is similar to CentOS).

You are correct, no c++ compiler on the AL2 distro.    Here is a complete set of instructions for installing on AL2 after I troubleshot.  Happy to add to docs with your blessing.
sudo yum install gcc-c++.x86_64
sudo yum install python3-devel.x86_64
sudo yum install unixODBC.x86_64
sudo yum install unixODBC-devel.x86_64

#replace <release_num> with the current release
sudo ln -s /usr/libexec/gcc/x86_64-amazon-linux/<release_num>/cc1plus  /usr/bin/

pip3 install pyodbc --user


1.18.x

please add support for 1.18 :)

Updated
https://github.com/crpmax/mc-bots/issues/7#issuecomment-1000543175


also support VARBINARY columns

qitab/qmynd@b48cd4e41f76e7f930b1efeedbe04c4efc86c52d added support for BINARY columns; this just extends it to also handle VARBINARY.

Thanks for the patch!


psh: sysexec whitelist support for * wildcard

Sysexec whitelist can contain * wildcard.
If command matches whitelist entry up to * it will be accepted (whatever follows).
* must be specified as standalone argument

JIRA: BES-159

Please update the documentation also.

Documentation added in https://github.com/phoenix-rtos/phoenix-rtos-doc/pull/9


ServerContext.IsCancelled needs to Pluck() for synchronous server calls

The current behavior will never return true because the corresponding event is never seen.

In fact, it does already.


Add basic support for yarn and pnpm workspaces

This reduces the number of package install steps in repros and saves time by avoiding duplicate installs.
Obviously, the pnpm change won't take effect until pnpm is re-enabled (though I've tested it locally).

Missing a null check.


BUGFIX: Close request return channel when finished with it.

Coverage remained the same at 34.875% when pulling e7adf0b439e930728e68cba6aabf4706d43b30c9 on TS_fix_messagesender_close_request into 0f6431375be3567060555484f95e424711997a91 on master.


Update build.gradle

Description
Fixed issue #

Added yourNewMethodName() that allows ...
Compatibility

OS
Implemented

Android
✅❌

Checklist

[ ] I mentionned this change in CHANGELOG.md.
[ ] I have tested this on a device/simulator for each compatible OS
[ ] I added the documentation in README.md.

Please explain your motivation, fill in the template, .. :)


Storages with managed memory produce non-managed output in ufuncs

What I posted in slack:
Getting an unexpected error converting a particular gt4py storage to a cupy array. I need to make a cupy view which I can modify, which directly modifies the GPUStorage. This normally works, but not in this example with nans:
>>> s = gt4py.storage.empty("gtcuda", [0, 0], [3, 3], float, managed_memory=True)
>>> s
GPUStorage([[0., 0., 0.],
            [0., 0., 0.],
            [0., 0., 0.]])
>>> s = s * cupy.nan
>>> s
GPUStorage([[nan, nan, nan],
            [nan, nan, nan],
            [nan, nan, nan]])
>>> s[1, 1] = 1
>>> s
GPUStorage([[nan, nan, nan],
            [nan,  1., nan],
            [nan, nan, nan]])
>>> cupy.asarray(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mcgibbon/miniconda3/envs/fv3util/lib/python3.7/site-packages/cupy/creation/from_data.py", line 68, in asarray
    return core.array(a, dtype, False, order)
  File "cupy/core/core.pyx", line 1785, in cupy.core.core.array
  File "cupy/core/core.pyx", line 1813, in cupy.core.core.array
  File "cupy/core/core.pyx", line 2814, in cupy.core.core._convert_object_with_cuda_array_interface
  File "cupy/cuda/memory.pyx", line 130, in cupy.cuda.memory.UnownedMemory.__init__
  File "cupy/cuda/runtime.pyx", line 503, in cupy.cuda.runtime.pointerGetAttributes
  File "cupy/cuda/runtime.pyx", line 201, in cupy.cuda.runtime.check_status
cupy.cuda.runtime.CUDARuntimeError: cudaErrorInvalidValue: invalid argument

The same error occurs if I use data.gpu_view() instead of cupy.asarray.

As a work around,
instead of s = s * cupy.nan
cupy.multiply(s, cupy.nan, out=s) should work (or also out pointing to some other gpu/managed

My understanding is that we are taking a different approach for the ufuncs now. Is this issue still relevant @gronerl @egparedes ?

obsolete


double _d suffix when building debug modules with python debug

Not sure who is the real culprit, but with python-3.5.2 and setuptools 40 on windows, if i build a python module (numpy, pandas) in debug mode (python setup.py build --debug), i have to use a release python. If i use a debug pyhton, i end up with pyd files that end with _d_d.pyd

Nevermind, our python build had an old patch in distutils that added _d suffix if setup.py was used with build --debug, but did not test if it was a debug python build, that we did not have at the time, which already adds _d