Streamice impose velocities

What changes does this PR introduce?
New feature: allows to impose streamice velocity on either B-grid or C-grid as needed. This is for those using shelfice_remeshing functionality, which currently allows to impose a dynamic mass tendency, i.e. adot-\nabla\cdot{uH} where adot is surface mass balance and uH is horizontal ice-shelf mass flux. But with a fixed velocity (u) this term should change as thickness changes. With this functionality pkg/streamice can be used to achieve this effect, and velocities can be imposed to avoid simulating the ice-shelf momentum balance.
It also addresses minor issues/bugs:
-- allows termination condition of fixed-point adjoint velocity solve based on absolute error OR relative (prev just relative), and terminates iteration when error increases (which will be due to small errors in linear system solve)
-- does not include metric terms in stress tensor when using cartesian grid
-- change formulation of regularised coulomb basal stress to avoid instability in adjoint
-- allow specification of ice-dynamical parameters through their log (rather than sqrt)
What is the current behaviour?
velocity is always solved for in streamice, not imposed.
What is the new behaviour
Please see as above
Does this PR introduce a breaking change?
YES -- behaviour to terminate adjoint fixed point loop whenever error increases, where "error" is the difference between the latest 2 iterates. verification halfpipe_streamice converges strongly yielding many orders of error reduction but then stagnates, likely due to rounding errors, and causes "error" to increase. New behaviour truncates the iteration -- accuracy is on a similar order to previous but results only agree to 8 decimals.
Other information:
Suggested addition to tag-index
o pkg/streamice

files changed: STREAMICE.h, streamice_readparms.F, streamice_init_fixed.F,
streamice_init_varia.F, streamice_timestep.F, streamice_advect_thickness.F,
streamice_fields_load.F, tools/OAD_support/ad_template.streamice_vel_phistage.F,
streamice_visc_beta.F, streamice_visc_beta_hybrid.F
fix regularised coulomb basal stress impl to avoid issues with adjoint
cancel metric terms in stress tensor for cartesian grid
allow (time-indep) velocities to be imposed from external file rather than
solved for
allow for specification of log of controls as well as sqrt
allow termination of adjoint fixed point loop based on relative and
absolute error
terminate adjoint fixed point loop when change between most recent iterates begins to increase (indicating unstable growth due likely to small errors in linear solve -- affects halfpipe_streamice oad verification)

@jm-c @mjlosch i have now actioned the comments made, regarding the breaking of the openad test, i have determined that this is in fact correct behaviour. i have now updated the P/R overview accordingly. I am unsure whether the tag-index text needs updating?

@dngoldberg thanks for making these changes (and updating this PR description).
I will generate a new "output_oadm.txt" file for halfpipe_streamice (now that we know
that these little changes in AD-grad are expected).
And regarding tag-index, the best is to update/complement the section "Suggested addition to tag-index"
in this PR description, and this will be added (but you don't have to do it) to tag-index before merging
this PR.
My suggestion here (but I might miss something so you don't need to folow this if you
feel otherwise) would be to re-order the few bullets point to have current point 2 & 4
next to each other and add there that these 2 set of changes affects AD-grad of AD verification
experiment "halfpipe_streamice".
On Fri, Dec 24, 2021 at 06:56:25AM -0800, dngoldberg wrote:

@jm-c @mjlosch i have now actioned the comments made, regarding the breaking of the openad test, i have determined that this is in fact correct behaviour. i have now updated the P/R overview accordingly. I am unsure whether the tag-index text needs updating?
--
Reply to this email directly or view it on GitHub:
https://github.com/MITgcm/MITgcm/pull/529#issuecomment-1000866607
You are receiving this because you were mentioned.
Message ID: @.***>

thank you @jm-c. i have modified the tag-index suggested text. in fact bullets 2 and 4 do not affect the verification and it is only the final bullet point as indicated.

The taf test would have worked, if the parameter declaration
      INTEGER streamice_max_nl
      PARAMETER ( streamice_max_nl = 100 )

hadn't been removed from "STREAMICE.h". In fact, the experiment is part of the TAF AD test suite and usually works. I think a minimum test one can do (even without access to TAF) is to generate the makefile and run the forward compilation (because that already fails now):
cd streamice_halfpipe/build
../../../genmake2 -mods ../code_ad -nocat4ad ...
make CLEAN && make depend && make

After fixing this (I enclosed the declaration by "if ALLOW_AUTODIFF_TAMC"), the new code leads to numerous recomputations (10, all related to err_max, which did not seem to have bothered us before) and on my MacBook Pro:
default    10     ----T-----  ----S-----  ----U-----  ----V-----
G D M    C  A  F        m  s        m  s        m  s        m  s
e p a R  o  d  D  m  m  e  .  m  m  e  .  m  m  e  .  m  m  e  .
n n k u  s  G  G  i  a  a  d  i  a  a  d  i  a  a  d  i  a  a  d
2 d e n  t  r  r  n  x  n  .  n  x  n  .  n  x  n  .  n  x  n  .

Y Y Y Y 13>10< 5 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- pass  halfpipe_streamice  (e=0, w=10, lfd=1, dop=1, sm=0)
Start time:  Mon Dec 27 11:22:48 CET 2021
End time:    Mon Dec 27 11:26:56 CET 2021

Right now I don't now if this expected for my mac, but have no time to check this on a linux box.
@dngoldberg Do you want me to fix this, or would you rather do this yourself?

@dngoldberg While going through the recomputations I noticed that for streamice_err_norm .lt. 1.0 the computation of err_max seems inconsistent (in both streamice_get_vel_fp_err.F and streamice_get_vel_resid_err.F), as you compute err_tempu and err_tempv, but you only use err_tempv in IF (err_tempv .ge. err_max) err_max = err_tempv. I am assuming that this part of the code is never used, but it is likely to confuse TAF and probably should be fixed in a way that you intended.

Hi @mjlosch i see what you say about streamice_max_nl -- i had not grepped for its use and would have seen its relevance to TAF-specific computations. I will address this right away.
I was alerted to a message about err_max in streamice_get_vel_fp_err.F and streamice_get_vel_resid_err.F but do not see it here. in these S/R's, err_tempu is used, please see e.g. line 99 of  streamice_get_vel_resid_err.F:
 err_tempv = MAX( err_tempu,

I am not sure in fact what is leading to recomputation issues? Back when i was able to use TAF, there were always a few recomputation warnings, but I determined they did not in the end actually lead to significant additional runtime -- but i apologise for not being able to be more helpful than this.
In contrast to the OpenAD experiment, the TAF experiment does not converge the forward velocity computation (to save time) -- and i have observed before this makes the results very sensitive to machine, so this might be the reason for 13 and 10 decimals.

@dngoldberg thanks for fixing this so quickly.
About the recomputations triggered by err_max: In theory, we should be able to get rid of all recomputation warnings by adding appropriate store directives (or other directives). In practice, the extra storing may not be worth it, because the recomputations are not excessive (i.e. you save a multiplication of a field with another one by storing the product, or recomputing err_max to use it in the AD computations). In this case, however, err_max is recomputed a couple of times within bi/bj-loops. Getting rid of this recomputation (with two nchklev_1*streamice_max_nl tapes) reduces the runtime of the simple halfpipe_streamice tests from 4min to 3min (I didn't measure this very carefully, though). The problem is that I am not quite sure if this also works in a multithreaded environment. I think, the tape declaration in the_main_loop.F needs the maximum number of threads, and the computation of ikey_nl in "streamice_vel_solve.F" needs to take the threads into account as well, but I might be wrong.
Would you like me to contribute this fix? I can push it, and if you don't like it, we can always revert. If you like it, we can address the threading issue (if it turns out to be an issue).

Thank you @mjlosch I think this might have been a recomputation warning that I was not able to determine how to get rid of, but was satisfied that it was benign enough to let alone. That is good to hear that there is a noticeable drop in run time though.
This sounds like a great change to make and I am very happy for you to implement it. Not sure about threading. I don't think there are any multithreading verification tests? In which case, it is possible that the TAF adjoint of halfpipe_streamice was never run with multithreading, it is not something which i have been able to determine how to do myself. If there is a threading issue, i am very happy to revisit it.

@dngoldberg See if you like my changes. I am not sure how important they are runtime-wise.
I feel like there  should be an
#ifef ALLOW_AUTODIFF
include AUTODIFF_OPTIONS.h

Hi @mjlosch  thank you for clearing up the unused variables, this is clearly something i had not taken care of. from the code alone it looks fine but have checked with a few forward experiments.
ALLOW_AUTODIFF_TAMC is defined in ECCO_CPP_OPTIONS.h which is included/used for both the TAF and OAD experiments. I think there was a move to purge use of this file in some experiments but, if it needs to be done here, perhaps this can be done with a separate P/R?

@dngoldberg I noticed that my commit: cd65017cc48cab2f7dfb5e5ada1cf59c380f1d64 to restore the tag-index file (adding back 1 blank line) has been somehow lost/overwritten by your commit: ce77d233aae4577bd967244f7639650addcd5eda
Should I do it again ?
Also, It might be time to remove the "Draft" status of this PR, do you agree ?

@mjlosch i have moved this from draft to ready. I have also made one small change to streamice_timestep to print a message to stdout when the velocity solve is overridden. Hopefully it is in line with code alignment standards, but otherwise it does not affect any mode tests.

@dngoldberg now that I have approved of the PR I find that it would be nice to get rid of the "Unused variable" warnings while we are at it. It's pretty straightforward in most cases (streamice_init_fixed.F is a bit intricate and I would like you to check if I did it right). Would you like me to push these changes?

hi @mjlosch i do see what you are saying about unused vars. I think most of the warnings arise from the fact that these variables need to be declared in ifdef blocks.
I have done this already for streamice_init_fixed.F as the file is already part of this PR -- however the warnings are extensive touching many many files, and as you suggest i believe this is too involved for the current PR which i would really really love to see merged. I am committed to helping with these unused variables and am happy to lead a PR addressing these now that (for the first time) i have piped a list of all the warnings -- if it is OK, can this be a separate PR to be raised by me, and 529 can be closed???

@dngoldberg no problem. That's a good plan. Just because I have them already, I will send you my modified files offline for this new PR, where would also address the oad-experiment as in #586.
Maybe they are of use.

hi @mjlosch thanks for this, i will hold on to them.

@dngoldberg I will do a quick check before merging this PR. Thanks.

LGTM. I did a quick check (including running shelfice_remeshing exp. from verification_other) and seems OK.
I could merge this PR soon (maybe tomorrow ?) if no one disagree.


Add support for Data Type

Related roadmap page

Support for four family types:

[ ] object including Map types
[ ] scalar

[ ] built-in
[ ] enums
[ ] custom scalars

[ ] array
[ ] external

Supported functionality:

[ ] Inheritance for objects; mindful of restrictions
[ ] Type Expression
[ ] Union types
[ ] Shortcuts & syntatic sugar
[ ] Runtime polymorphism
[ ] Examples
[ ] Inline declaration

Data Types support for:

[ ] URI Parameters
[ ] Query parameters
[ ] Request & Response Headers
[ ] Request & Response Body

JSON
XML
Form

[ ] Documentation
[ ] Test coverage

I take this bug, can you assign it to me?

@tardyp done! but I think I shouldn't have needed to add you to the spotify org - I just added you as a collaborator, but let me double check sine github org permissions are not that easy to immediately understand.

Hi  @econchick & @tardyp, not sure what the progress is but the links to the different sections inside the spec are out of date. I also like to offer you as much help as possible from our experts as needed. Let me know.

Hi Christian,
Both myself and @econchick had a very busy second quarter, so this is why this is not progressing as we would like. For my side, I am still very interrested in raml potencial for python webapps, and willing not make it move forward as time allows.

@sichvoge hey Christian!
You're right that the links are outdated - but (FYI @tardyp 😄 ) I have been working on it! and current progress can be seen in the v.0.2.0-dev branch (although I have yet to push latest commits from the past couple of days which really work on this particular ticket).
I've updated this issue with appropriate links (and I will do that for the other issues at some point) and current status of progress.

I just forked, are the issues you have listed for this repo the only issues to tackle in order to accomplish the upgrade to 1.0?

I think @econchick and maybe you should review the todos in this issue, and maybe others with the current spec version.


notif: Support migration of Android notification channels

(Split from #717)
Needed for #340, when updating notification channels to use a custom notification sound.

Thanks for the review @chrisbobbe! Pushed a new revision PTAL.

Thanks! LGTM, over to @gnprice.

Thanks for the reviews @chrisbobbe and @gnprice!
@gnprice pushed a new revision with update test bindings. PTAL.

Thanks for the revision! All looks good — merging.


Update documentation

So people knows about latest improvements

Trust me, this work is underway. Until then, #47 and #38 READMEs will just have to suffice.


Relative links to images don't work with preview "New Repos landing pages" enabled

When the preview feature "New Repos landing pages" is enabled relative links to images in the same directory don't work. I've tried the other methods of linking an image ("Absolute path in Git" and "Absolute path in TFVC") but these didn't seem to work either.
The browser I've used was Firefox 70.0 (64-bit) on windows. According to the inspector the links weren't correct simply causing a 404.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fa622358-f5bb-1f5c-177a-081a37b6acef
Version Independent ID: 72f7fd16-a822-66c8-3a34-8905645ce5f2
Content: Markdown syntax for files, widgets, and wikis in Azure DevOps - Azure DevOps
Content Source: docs/project/wiki/markdown-guidance.md
Product: devops
Technology: devops-collab
GitHub Login: @chcomley
Microsoft Alias: chcomley

@AndreSteenveld Thank you for the feedback.
Unfortunately this is an issue more apt for feedback from the product team.
Please see if this post addresses the issue, else start a new issue and link back here.

@msebolt Thanks for the quick reply, the issue you linked didn't seem to be related although someone else reported this exact same issue there as well (New repo landing page images are not rendered) so I'm guessing this is a known issue now :).

I agree with @AndreSteenveld, the relative path doesn't seem to work to display images in Azure DevOps wiki.


Add the zsh snap

[x]   General Comment
<img border=0 src='https://avatars.githubusercontent.com/u/3689821?v=3' height=16 width=16'> @elopio nice work!
I haven't tested it, but I looked it over and it seems to be :+1: except for the comments I left.

This pull request is now managed with CodeReviewHub. Refresh.

@elopio nice work!
I haven't tested it, but I looked it over and it seems to be :+1: except for the comments I left.

Thanks for the review. I pushed the fixes.

Nice work! Merging!


so outdated it conflicts with everything else

Hello,
This package relies on such an old version of solana it breaks every single package it interacts with including some of your own pacakges. raydium-contract-instructions, amongst others. Can you consider updating it to latest? It is shockingly behind in versions

Because upgrading the solana version requires upgrading the anchor version to 0.30.1, but after trying to upgrade, a Stack offset error would be thrown. You can refer to https://github.com/coral-xyz/anchor/pull/2939.
And this error will be solved in the next anchor version, so after the anchor is upgraded, we will perform a corresponding version update.
Thanks!


Added lock protection

Added lock protection to metasmoke.py and chatcommands.py regarding to the read operation of apiquota from GlobalVars.apiquota. The lock may be needed as the write operation in bodyfetcher.py may not be atomic, and hence without lock, half-written and corrupted data may be read - the same reason why metasmoke.py in method send_statistics acquired posts_scan_stats_lock before reading post scan statistics.

If this is done, then bodyfetcher.py needs to be modified to release the lock during the time it sleeps for backoff, or this thread could stall waiting for the lock for over a minute, potentially an unbounded period of time (usually backoffs are 10 seconds, but they can be any value).
A better overall solution would be to implement getters and setters in GlobalVars that automatically acquire and release the locks when the values are read/written (this could be generically applied to all values stored in GlobalVars which may be changed somewhere other than in GlobalVars, or maybe, during startup elsewhere). That would make it such that the individual threads wouldn't need to worry about getting and releasing the locks, unless they wanted to implement a atomic read/modify/write.

(@Makyen) I am now trying to implement a separate apiquota lock. (Also that maybe we should have documentation about which lock is about which fields)

Yeah, documentation would be good, or at least comments in the code that explain what's what. The issue of knowing which lock is for which value is one of the things that would be solved by using getters and setters (which are written to both automatically acquire/release locks) for any of the values which are permitted to be changed outside of globalvars.py (and maybe the startup portion of ws.py).

The major problem about getters/setters is that it requires a major refactoring all over the code base, as by the original design, now there are countless many direct read/write to GlobalVars fields, and hunting them down would be a huge trouble. Will try to read the code again and document those locks.

Using getters and setters specifically means you don't need to hunt those down. They mean that code can just use the variables as they normally would and have the locks handled "behind the scene".

!!/approve


Updated the example demos

Fixed issue where recent changes caused subscription demo to fail. Also added a simple chat message demo.

Thanks for the submission. I’ll take a look soon.

@rodkeys I found that, given the same peerID, the subscription key in this example generated via generateSubscriptionKey does not match the one generated by the go code.
Here's a tweaked generateSubscriptionKey function which will make the keys match.
(FWIW, the keys matching is critical to messages being routed properly)
import {
  fromB58String,
  toB58String,
  decode,
  encode
} from 'multihashes';
import { fromBuffer } from 'node-bitarray';
import BufferShift from 'buffershift';

export async function generateSubscriptionKey(peerID) {
  const mh = fromB58String(peerID);
  const decodedMh = decode(mh);
  const digest = decodedMh.digest;
  const prefix = new Buffer(new Uint8Array(digest.slice(0, 8)));

  const shifted = new Buffer(prefix.length);
  prefix.copy(shifted);
  BufferShift.shr(shifted, 48);

  const checksum = await crypto.subtle.digest('SHA-256', shifted);
  const subscriptionKey = encode(Buffer.from(checksum), 'sha2-256');
  return toB58String(subscriptionKey);
}


bug: copy_and_exit doesn't copy text

Hi, I'm currently using the option copy_and_exit: c y, in Normal Mode, I select the text and press c or y, it exits out of Normal Mode but doesn't copy any text. I'd have to press C-C to copy it.
I installed warpd using the AUR package.

Are you using X? Does this happen everywhere, or only in certain programs? Do you have xclip installed?

Yes I do use xclip. And I only tested it in Firefox. Is xclip the problem?

The copy feature works by emitting control+insert, which should copy the selection to the primary clipboard and then doing xclip -o|xclip -selection CLIPBOARD to make the primary and clipboard selections consistent. I just tested it and it seems to work fine on firefox. Can you see if the problem can be reproduced using other programs?

Thank you for the answer. I tried it out in gedit and it works. I also turned off Vimium C in my Firefox and it also works. So it turns out it's not a warpd problem at all :smile:
However I'm struggling find a way to make this work with Vimium C. I can physically press C-insert and it will copy the text even with Vimium C. But with Vimium C turned on, warpd can't copy text.
Is there a way that I can set up so that when I press a key, warpd will exit and press that key? Thank you very much!

Come to think of it, I wonder if a "fall through" feature already exists in warpd. If a key is not mapped, it acts like warpd isn't there. Or maybe something like after selecting with warpd, I press y or C-c and it exits warpd, then press the correct keys.

I can physically press C-insert and it will copy the text even with Vimium C.

I will investigate when I have some time.

Come to think of it, I wonder if a "fall through" feature already exists in warpd. If a key is not mapped, it acts like warpd isn't there.

This has been proposed before, but is non trivial to implement. There are currently no plans for such a feature.


Don't require user/token and log request on options

This takes care of a regression that was introduced in ff3f2ab298. The
handle_options_request was moved and renamed to be consistent with
the other options actions, but these before_action hooks were not
updated.
@miq-bot add-label api, bug
@miq-bot assign @abellotti

:+1: @abellotti good for you?

well, maybe only for /api,  but I still want to keep authentication and logging for the OPTIONS /api/:collection calls enabled, some can take a bit of time, I like to see in logs auth, frequency, duration of these calls to help us enhance later.

@abellotti I updated the description to better describe the problem - the regression with logging was that we were not calling log_request_initiated on options requests, this fixes that.
WRT to adding authentication for OPTIONS requests, I'm concerned that this could cause issues in browsers which will issue an OPTIONS request prior to making say a PUT/PATCH to check what is available.

true that could cause issues, but it hasn't so far with all the UI/SUI uses.  We do provide a lot of information on some of the OPTIONS /api/:collection, so I'd prefer to keep those secure/authenticated until it really is an issue.

Just a note here.. the UI and SUI can't hit any problems with CORS, because we're always using the API from the same domain & port combination - no CORS :).

I'm concerned that even though this may not be an issue for UI/SUI, this is violating the CORS protocol and could impact anyone trying to develop against our API. To quote: "CORS-preflight request never includes credentials." - https://fetch.spec.whatwg.org/#cors-protocol-and-credentials

@imtayadeway looks like that need is coming in earlier that we thought.
A couple of things here.
1 - when I was looking at the logging done with OPTIONS, the "API Request:" line is logged later than  normal requests.
In order to get API Request/Request/Parameters/Response  (normal sequence without the Authentication/Authorization lines) could you move the
before_action :log_request_initiated, :only => [:options]

to right after:
before_action :require_api_user_or_token, :except => [:options]

2- I ran OPTIONS on all collections, and I'm ok with the data being returned without authentication. The only one that I wasn't sure about is for container_deployments.  Can you look into that and see if the information is discriminating in any way, host/ipaddresses/etc.   If it is, can that information can be provided to those users a different way and if so/until then see can we require auth just for OPTIONS /api/container_deployments.
Thanks

I applied the patch, but it solved the problem for me only for top level entities like:
https://<IP_ADDRESS>/api/templates
https://<IP_ADDRESS>/api/clusters
https://<IP_ADDRESS>/api/vms
but for
https://<IP_ADDRESS>/api/templates/89 fails with error 404 and in browser log I can see a message about failed preflight request

XMLHttpRequest cannot load https://<IP_ADDRESS>/api/templates/95. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. The response had HTTP status code 404.

Also when I run chrome with disabled security google-chrome --disable-web-security --user-data-dir all request are passing

@mkrajnak thanks for the feedback! Would you mind testing against the latest revision of this branch?


Spawnmass to ip

Mass boost :D
Example : Me : i joined the game and my spawnmass the have 4k
Other : I joined the game and my spawnmass the have 1k

duplicate!!! [closed]

@Fan7asy
make issue to duplicate

for other why duplicate #262

I not know this project, but i can imagine:
var ips = ["<IP_ADDRESS>","<IP_ADDRESS>"];
ips.forEach( e => { e.mass = 4000;});


Webhook pod leadership lock mismatch

Running dynatrace-oneagent-operator v0.8.2, I have a situation where the webhook pods cannot complete startup due to a mismatch in leadership locks between the bootstrapper and the webhook.  There are two pods running - on pod A, the bootstrapper has the leadership lock and has completed the bootstrap process.  However, the on Pod B, the webhook pod has the leadership lock.  The webhook on Pod A is stuck waiting to become leader, while in Pod B the webhook is crashlooping - the certificates it requires are never created due to Pod B bootstrap pod being stuck waiting for the lock.

Hi. Both the bootstrapper and the webhook containers (should) use different leader locks. Can you provide logs from both containers, so that I can take a look?
Also, was this caused by a new installation, or an upgrade?

After posting the issue I realised I could fix the issue by scaling the deployment down to one replica and then back up to two, which worked to fix the problem.  So unfortunately I only have one half of the logs to hand now:
logs dynatrace-oneagent-webhook-6dbb9ccc85-8ktmw webhook
{"level":"info","ts":"2020-10-23T08:49:25.206Z","logger":"cmd","msg":"Go Version: go1.14.9"}
{"level":"info","ts":"2020-10-23T08:49:25.207Z","logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":"2020-10-23T08:49:25.207Z","logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
{"level":"info","ts":"2020-10-23T08:49:25.207Z","logger":"cmd","msg":"Version of dynatrace-oneagent-operator: v0.8.2"}
{"level":"info","ts":"2020-10-23T08:49:25.207Z","logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":"2020-10-23T08:49:26.702Z","logger":"leader","msg":"Found existing lock","LockOwner":"dynatrace-oneagent-webhook-6dbb9ccc85-6472b"}
{"level":"info","ts":"2020-10-23T08:49:26.731Z","logger":"leader","msg":"Not the leader. Waiting."}
{"level":"info","ts":"2020-10-23T08:49:27.870Z","logger":"leader","msg":"Not the leader. Waiting."}
{"level":"info","ts":"2020-10-23T08:49:30.266Z","logger":"leader","msg":"Not the leader. Waiting."}

logs dynatrace-oneagent-webhook-6dbb9ccc85-8ktmw bootstrapper
{"level":"info","ts":"2020-10-23T08:40:42.414Z","logger":"cmd","msg":"Go Version: go1.14.9"}
{"level":"info","ts":"2020-10-23T08:40:42.414Z","logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":"2020-10-23T08:40:42.414Z","logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
{"level":"info","ts":"2020-10-23T08:40:42.414Z","logger":"cmd","msg":"Version of dynatrace-oneagent-operator: v0.8.2"}
{"level":"info","ts":"2020-10-23T08:40:42.417Z","logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":"2020-10-23T08:40:43.715Z","logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
{"level":"info","ts":"2020-10-23T08:40:43.715Z","logger":"leader","msg":"Continuing as the leader."}
{"level":"info","ts":"2020-10-23T08:40:44.817Z","logger":"cmd","msg":"Registering Components."}
{"level":"info","ts":"2020-10-23T08:40:44.817Z","logger":"cmd","msg":"Starting the Cmd."}
{"level":"info","ts":"2020-10-23T08:40:44.818Z","logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"webhook-bootstrapper-controller","source":"channel source: 0xc00055b450"}
{"level":"info","ts":"2020-10-23T08:40:44.818Z","logger":"controller-runtime.controller","msg":"Starting Controller","controller":"webhook-bootstrapper-controller"}
{"level":"info","ts":"2020-10-23T08:40:44.818Z","logger":"controller-runtime.controller","msg":"Starting workers","controller":"webhook-bootstrapper-controller","worker count":1}
{"level":"info","ts":"2020-10-23T08:40:54.818Z","logger":"webhook.controller","msg":"reconciling webhook","namespace":"dynatrace","name":"dynatrace-oneagent-webhook"}
{"level":"info","ts":"2020-10-23T08:40:54.818Z","logger":"webhook.controller","msg":"Reconciling certificates..."}
{"level":"info","ts":"2020-10-23T08:40:54.922Z","logger":"webhook.controller","msg":"Reconciling Service..."}
{"level":"info","ts":"2020-10-23T08:40:55.024Z","logger":"webhook.controller","msg":"Reconciling MutatingWebhookConfiguration..."}

The logs for dynatrace-oneagent-webhook-6dbb9ccc85-6472b showed the bootstrapper detect that 8ktmw was the leader, while the webhook container found a lock with its name, then looped through "Waiting for certificates to be available." until it crashed.
I'm not sure the history of the deployment that led to this situation, but if both pods start at the same time, and both bootstrapper and webhook run through the same code up to the leadership check, then a race condition seems likely?  Since the locks are different, it seems like there's no guaranteed that both will be claimed by containers in the same pod.

scaling the deployment down to one replica and then back up to two
That is the reason in fact, the current implementation expects to run with one replica. Well, it's a bit more of an issue with the bootstrapper (who generates and renews the SSL certificates for the webhook HTTPS server), who may then need to coordinate with other instances.

Any reason to have multiple replicas? I presume high availability, but to confirm.

Hi - it is indeed for availability.  We can reduce the replicas to 1 for now, but we would definitely prefer two are available.

Thank you for the feedback. I've created an internal ticket to look at it.


Does not work when Floating IP is used on Digital Ocean

When the script is run for the first time, it will automatically detect the Public IP.
But when the Floating IP is mentioned instead of the default Public IP, the set up is not working. That is, the OpenVPN client is not able to establish a connection with the server.

Use DNS name instead of IP. When floating IP is being used, just update DNS record.

hi @angristan and thank you for the great script. I am also trying to get it working on a DigitalOcean droplet with floating ip. I'm on CentOS 7.
I've looked at this and this but I coudn't get it working... I'm afraid I don't understand enough of iptables otherwise I'd be glad to help investigate the further.

Alright, I managed to get it working!
So here's how:

Make sure you have floating ip enabled on the droplet with http://<IP_ADDRESS>/metadata/v1/floating_ip/ipv4/active (if so, will return true)
Get anchor-ip with curl -s http://<IP_ADDRESS>/metadata/v1/interfaces/public/0/anchor_ipv4/address
Get floating-ip with curl -s http://<IP_ADDRESS>/metadata/v1/floating_ip/ipv4/ip_address
Run ./openvpn-install.sh (uninstall first if you have it installed already) and follow the process providing the anchor-ip for all instances when the ip of the vpn is asked for
Comment out/replace line 1 (iptables -t nat -I POSTROUTING (...)) of both /etc/iptables/rm-openvpn-rules.sh and /etc/iptables/add-openvpn-rules.sh and use instead iptables -t nat -A POSTROUTING -s <IP_ADDRESS>/24 -j SNAT --to-source anchor-ip
Add local anchor-ip in /etc/openvpn/server.conf
Restarted OpenVPN with systemctl restart<EMAIL_ADDRESS>Update in your local machine the user.ovpn config to connect to the vpn using the floating-ip (replacing the line remote (...) with remote floating-ip 1194)

Thanks a lot.
In step 4, when it asks you for the public IP for the first time, you should provide the anchor-ip. From this, the script will sense/detect that the IP is behind NAT, so it will ask you for the real public IP in the next question; and, this time you supply it with the floating-ip. Doing it this way will save you from modifying the .ovpn file in step 9, as it will already contain the correct configuration for remote. i,e. remote floating-ip 1194. I have tested it and it works.


Search Editor: Add settings for search option default values

Since the new Search Editor treats each search as a new search, it doesn't remember the options for "Match Case", "Match Whole Word", and "Use Regular Expression" (as the sidebar search does). This makes logical sense, but it's inconvenient if you need to toggle those options manually for every search.
It'd be very helpful to specify default search options, so that for example, "Use Regular Expression" was always enabled for each new search.
Ref https://github.com/microsoft/vscode/issues/88154#issuecomment-578559837

@JacksonKearl This is still an upcoming feature, right? Can we expect it in the March or April releases?

@karlhorky We'll probably add either default or persistent options, but I don't have a specific timeline for it.

You can try out "search.searchEditor.experimental.reusePriorSearchConfiguration": true in tomorrow's insiders (not the next Insiders to be released but the one after it) and let me know what you think.

Going to let this cook as an experimental setting for a bit. If others come across this issue and this works, please upvote so I know it's desired, or comment if this doesn't work as you'd expect.

@JacksonKearl thanks!
Which issue should we give a thumbs-up to? Your comment with the configuration setting (https://github.com/microsoft/vscode/issues/89353#issuecomment-611537268)?
I would suggest against using adamlogic's original description at the top: I think this would be misleading, since a lot of users upvote issues that they want implemented (not even knowing that it is already done), instead of meaning to upvote your implementation thereof.

You can try out "search.searchEditor.experimental.reusePriorSearchConfiguration": true in tomorrow's insiders (not the next Insiders to be released but the one after it) and let me know what you think.

Is this available yet (in Insiders)? I can't find that setting

@carlocardella its hidden - you’ll have to paste the json manually.

Took the setting out of experimental, now you'll be able to find it in the UI settings editor. Also related #95582, which allows you to pass args in the keybinding configuration:
    {
        "key": "ctrl+shift+g",
        "command": "search.action.openNewEditor",
        "args": {
            "query": "${selectedText}",
            "filesToInclude": "${relativeFileDirname}"
        },
        "when": "editorTextFocus"
    }

Explicitly passed args override prior config.

Nice! This will appear for users in the stable May 2020 update?

Yes. And it will be out in the next Insiders release.

@JacksonKearl When using this, and running it with no text selected, I get '${selectedText}' can not be resolved. Make sure to have some text selected in the active editor.
Is there a way to define a single keybinding that can default to empty string if there is no text selected? Perhaps a ${selectedTextOrEmpty} variable?
    {
        "key": "ctrl+shift+g",
        "command": "search.action.openNewEditor",
        "args": {
            "query": "${selectedTextOrEmpty}",
            "filesToInclude": "${relativeFileDirname}"
        },
        "when": "editorTextFocus"
    }

By default it will use your selectedText or the empty string, you don’t need to include anything for query

Just received the VS Code May 2020 (1.46) update and the configuration below works well, thanks @JacksonKearl !
"search.searchEditor.reusePriorSearchConfiguration": true


Replace list attribute with sets

Lists are ordered. If a user changes the order of a list, Terraform will attempt to update the resource. If there are other resources that depend on the list attributes, they'll be destroyed and recreated. That's a big problem for things like CMEK.
A better alternative is to use a set. It's backwards compatible since it's represented the same way in state, but Terraform ignores the order. Terraform automatically sorts set elements alphabetically, so this required updating some test expecteds.
This also means we no longer need to sort regions or other fields according to the plan.
Using the set plan modifiers required pulling in new framework code, so I also took the opportunity to update the framework from 1.3.3 to 1.3.5 for a few minor bug fixes.
Commit checklist

[x] Changelog
[x] Doc gen (make generate)
[x] Integration test(s)
[x] Acceptance test(s)
[x] Example(s)

Have you tested/is there a good way to test this change with a TFstate that uses the list attributes?

With the test framework? No. But I manually tested a few different moderately complex clusters.

TFTR!


Fee calculation for minting an NFT and sending to a script address

I have a situation where I am minting an NFT and sending to a script address, but getting fee calculation issues
fromList [("err",Object (fromList [("fee",Number 234490.0),("kind",String "FeeTooSmallUTxO"),("minimum",Number 234666.0)])),("tx",Object (fromList [("txid",String "d7e02c59")])),("kind",String "TraceMempoolRejectedTx"),("mempoolSize",Object (fromList [("bytes",Number 0.0),("numTxs",Number 0.0)]))]
ogmios_1              | {"severity":"Info","timestamp":"2022-06-03T10:21:10.181262067Z","thread":"1276","message":{"WebSocket":{"contents":{"agency":"ServerAgency TokBusy","tag":"RejectTx","event":"receive","reasons":[{"feeTooSmall":{"requiredFee":234666,"actualFee":234490}}]},"tag":"WebSocketClient"}},"version":"v5.2.0"}

It turns out this was an issue for my specific wallet,  addr_test1qpkl55y6av6lvu9gfjkqr0cmtkxn7qcezgg0q3vr0m2huda6qh8x3elu6qa9t48ymn0dluh805ws2we38uxwxcnzyygq5jl946
I created a new wallet addr_test1qp7l9wfw40l8ac2at0g9mtc32uhnljhrkupxs5dttkwj3eswrtzmr8ql4rc58d89s5lckr5rkx6m9dv75fu464wfguaqmaceek and the transaction went through. My first wallet has a pretty niche selection of value/utxos, I wonder if this is a min. Ada issue.

Since we don't know what's causing this we can remove it from stage 4 and include a possible fix in a future patch release


inventory-kamera: Add version 1.1

Closes #XXXX

Relates to #XXXX

[x] I have read the Contributing Guide.

/verify

/verify


Azure AD B2B Direct Federation does not work with DirectFedAuthURL and sub-domain of a DNS-verified domain

Hello,
I am setting up Direct Federation in AAD External identities, however it looks like the behavior is not the one expected as per your documentation. I am using fictious domain names to explain the odd behavior.
We have the domain contoso.com  validated in our AAD and it is configured to use our AD FS for authentication. This is working fine.
We want to set up direct federation for a sub-domain my.contoso.com that is currently not verified in AAD (only the parent domain contoso.com is verified in AAD). This direct federation is aimed to authenticate users from an LDAP directory with an AD FS dedicated for this purpose.
When configuring the Direct Fed with these parameters, the configuration succeeds and authentication with users like<EMAIL_ADDRESS>against our tenant is redirected to the AD FS https://my.contoso.com:

Domain name: my.contoso.com
Entity ID: http://my.contoso.com/adfs/services/trust
Passive requestor endpoint: https://my.contoso.com/adfs/ls/
Certificate is added

However, in reality, our AD FS is not configured to listen on https://my.contoso.com but on https://login-my.contoso.com. In this case, by following the doc, we created this DNS record:
my.contoso.com. IN  TXT  DirectFedAuthUrl=https://login-my.contoso.com/adfs/ls/
Then we configured the direct federation with these parameters:

Domain name: my.contoso.com
Entity ID: http://login-my.contoso.com/adfs/services/trust
Passive requestor endpoint: https://login-my.contoso.com/adfs/ls/
Certificate is added
The configuration succeeds in Azure Portal, however when a user authenticates to our tenant and enters the<EMAIL_ADDRESS>email, it is not redirected to https://login-my.contoso.com but to the AD FS for contoso.com parent domain.

Do you know why the configuration of direct federation passes but does not do become active when configuring as above with the DirectFedAuthURL DNS TXT records? It is easy to repro.
If you have a way to pass this info to the product team, that would be nice, at least to know if there is a bug, or if this is not a supported configuration and to include this in the documentation.
Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7b1d9428-2dbb-c80b-c2cd-b4841d3d8bd3
Version Independent ID: 20af5f8d-479c-66ec-aacd-d3f9cb79c814
Content: Federation with a SAML/WS-Fed identity provider (IdP) for B2B - Azure AD
Content Source: articles/active-directory/external-identities/direct-federation.md
Service: active-directory
Sub-service: b2b
GitHub Login: @msmimart
Microsoft Alias: mimart

@StefanPlizga
Thanks for your feedback! We will investigate and update as appropriate.

Hallo
@MarileeTurscak-MSFT @StefanPlizga
Is there any news on this issue? As we are facing the same problem for a client of ours.
Regards
The Identity Hub


Add opencensus-exporter-trace-zipkin to list of artifacts to release.

Coverage remained the same at 79.176% when pulling 19dec9722eec390d4d6b16e2e93d107bbd05783f on bogdandrutu:zipkin_export into 5298e6c602b527df97bf914704b9f1339d28f345 on census-instrumentation:master.

Codecov Report

Merging #733 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #733      +/-   ##
============================================
- Coverage     77.35%   77.32%   -0.03%
  Complexity      772      772
============================================
  Files           142      142
  Lines          3520     3520
  Branches        331      331
============================================
- Hits           2723     2722       -1
  Misses          733      733
- Partials         64       65       +1

Impacted Files
Coverage Δ
Complexity Δ

...census/implcore/trace/export/SpanExporterImpl.java
90% <0%> (-1.67%)
6% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5298e6c...19dec97. Read the comment docs.

👍


All things open event page

Description
This is the event page for the all things open event.

@veextee  this is built out I'll just need final copy, images etc to complete it.

@veextee  Can I get  review on this?

@veextee  will need to review + approve and this can be merged in.


Error when building

I've added the action to my GitHub actions and this is what I'm getting when it runs:
Run<EMAIL_ADDRESS>/usr/bin/docker run --name fbc64f7908e354d81c43489949f8e5b0f1540e_b30bc5 --label fbc64f --workdir /github/workspace --rm -e GITHUB_TOKEN -e HOME -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e GITHUB_ACTIONS=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/ros/ros":"/github/workspace" fbc64f:7908e354d81c43489949f8e5b0f1540e
Building native extensions. This could take a while...
Successfully installed jaro_winkler-1.5.3
Successfully installed parallel-1.18.0
Successfully installed ast-2.4.0
Successfully installed parser-<IP_ADDRESS>
Successfully installed rainbow-3.0.0
Successfully installed ruby-progressbar-1.10.1
Successfully installed unicode-display_width-1.6.0
Successfully installed rubocop-0.75.1
Successfully installed rubocop-performance-1.5.0
Successfully installed rack-2.0.7
Successfully installed rubocop-rails-2.3.2
11 gems installed
cannot load such file -- pry
/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/github/workspace/lib/core/spec/support/rubocop/redundant_factory_bot.rb:3:in `<top (required)>'
/github/workspace/lib/core/spec/support/rubocop/linters.rb:3:in `require_relative'
/github/workspace/lib/core/spec/support/rubocop/linters.rb:3:in `<top (required)>'
/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_loader_resolver.rb:13:in `block in resolve_requires'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_loader_resolver.rb:11:in `each'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_loader_resolver.rb:11:in `resolve_requires'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_loader.rb:45:in `load_file'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_loader.rb:86:in `configuration_from_file'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/config_store.rb:44:in `for'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/target_finder.rb:112:in `excluded_dirs'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/target_finder.rb:90:in `find_files'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/target_finder.rb:60:in `target_files_in_dir'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/target_finder.rb:31:in `find'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/runner.rb:64:in `find_target_files'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/runner.rb:34:in `run'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/cli.rb:210:in `execute_runner'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/cli.rb:80:in `execute_runners'
/usr/local/bundle/gems/rubocop-0.75.1/lib/rubocop/cli.rb:51:in `run'
/usr/local/bundle/gems/rubocop-0.75.1/exe/rubocop:13:in `block in <top (required)>'
/usr/local/lib/ruby/2.6.0/benchmark.rb:308:in `realtime'
/usr/local/bundle/gems/rubocop-0.75.1/exe/rubocop:12:in `<top (required)>'
/usr/local/bundle/bin/rubocop:23:in `load'
/usr/local/bundle/bin/rubocop:23:in `<main>'
/usr/local/lib/ruby/2.6.0/json/common.rb:156:in `parse': 767: unexpected token at '' (JSON::ParserError)
	from /usr/local/lib/ruby/2.6.0/json/common.rb:156:in `parse'
	from /action/lib/index.rb:79:in `block in run_rubocop'
	from /action/lib/index.rb:78:in `chdir'
	from /action/lib/index.rb:78:in `run_rubocop'
	from /action/lib/index.rb:121:in `run'
	from /action/lib/index.rb:132:in `<main>'
##[error]Docker run failed with exit code 1

No clue where to start debugging.
This is my current action file
name: Rubocop

on: [push]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1
    - name: Set up Ruby 2.6
      uses: actions/setup-ruby@v1
      with:
        ruby-version: 2.6.x
    - name: Rubocop Linter
      uses<EMAIL_ADDRESS>      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

hm I am seeing this error cannot load such file -- pry. I will investigate further. Is the project public by chance @rpbaltazar?

Also, what is your ruby version? Seeing several issues on the actions/setup-ruby repo and wondering if that could be causing this.

https://github.com/story-branch/story_branch/actions -> works
https://github.com/rails-on-services/ros/actions -> does not work

@rpbaltazar interesting. It looks like the only difference is one of those projects has a config file for rubocop and the other doesn’t. I’ll dig in to this tonight.

The story_branch is a ruby only project while the ROS is a multi project rails structure. not sure if it causes some sort of problems. Particularly because most of the rails projects under ROS are engines so they have a gemfile and a gemspec. I do use pry on the ros so, might be something along those lines...

@rpbaltazar I believe I have fixed the issue. Can you try upgrading too v0.2.0? The docs have been updated. Let me know!

Just tried it and still facing the same issue:
https://github.com/rails-on-services/ros/commit/8a832128e9ccfd4bd4036d64417b884a1092b811/checks?check_suite_id=280826517

@rpbaltazar Weird. I’ll look into this more. It’s worth noting you can run a rubocop action (minus the annotations) pretty easily. I can paste in the config you need if it would help and you don’t really care about annotations.

@rpbaltazar
Your project it's required a custom rubocop linter.
In the redundant_factory_bot.rb file
This Rubocop linter action runs in a docker container with only this gems
rubocop rubocop-performance rubocop-rails rubocop-minitest rubocop-rspec

Because of this when run rubocop thows this kind of error.
Please check if you need to require pry in this file.

@MiguelSavignano Thank you for pointing that out! It must have slipped through when we were debugging the linter... 🤦♂
Let me remove it and try again the rubocop linter

Thanks you @MiguelSavignano


[Question]: How to store two index in redis?

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
I have two index that keep different data as below code
index_policy = VectorStoreIndex(nodes = base_nodes+objects)
index_leave = VectorStoreIndex.from_documents(list_of_documents_leave)
query_engine_leave = index_leave.as_query_engine(similarity_top_k=2,filters = filters,streaming=True)
query_engine_policy = index_policy.as_query_engine(similarity_top_k=6,streaming=True)
But I want to save those index (index_policy,index_leave) in redis database to persist data and I want to call index from redis database instead of local paths. Moreover, with index_leave, I also used ExactMatchFilter for filter information in some columns.
Can you recommend how to develop the code from my requirement?

I got an error

Deprecated kwarg, index_name, found upon initialization. RedisVectorStore now requires an IndexSchema object

@logan-markewich  Can you help me for this case? if you want more information don't hesitate ask

@Ninlawat-Puhu You can use the redis vector store
https://docs.llamaindex.ai/en/stable/examples/vector_stores/RedisIndexDemo/?h=redisve


Code availability

I note that the paper says the code is available here and I'm wondering if there's any estimated timeline when it might be provided. Thanks!

I'm currently cleaning the source code and will release it within the next few days.

@hoseinzadeehsan Great! Thanks 🙂


django_libsass.SassCompiler: not found

I am not using vagrant and postgres
And i manage to install every dependencies which need to run wagtail
And successful migration after running localhost i am getting this error

/bin/sh: 1: django_libsass.SassCompiler: not found

I pasted the copy of pip freeze output here. See i having
Django==1.6.5
Pillow==2.5.2
South==1.0
Unidecode==0.04.16
argparse==1.2.1
beautifulsoup4==4.3.2
compass==0.1.4
django-appconf==0.6
django-compressor==1.4
django-libsass==0.2
django-modelcluster==0.3
django-taggit==0.12.1
django-treebeard==2.0
html5lib==0.999
psycopg2==2.5.3
requests==2.3.0
six==1.7.3
unicodecsv==0.9.4
wagtail==0.5
wsgiref==0.1.2
If you need Full traceback please click here

It's still relevant. Doesn't work within virtual environment.

@tezro If you're having problems, please open a new issue with a full description of your setup and the installation steps you've taken.

Well, to be honest, the setup is absolutely the same: pip install django-libsass libsass runs without problems, but Django is raising "/bin/sh: django_libsass.SassCompiler: command not found" for any version of each of them...

If pip freeze shows that libsass is successfully installed, then the problem is different to the one discussed on this ticket. Without a full description of that problem, I can't really help any further.

I've now put together a wiki page detailing steps to try when encountering this error: Troubleshooting /bin/sh: django_libsass.SassCompiler: command not found


test: perform out of bounds check when writing to slice

Tests this PR: https://github.com/noir-lang/noir/pull/5694

Benchmark results
Metrics with a significant change:

protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 3,330 (+59%)
protocol_circuit_output_size_in_bytes (public-kernel-tail): 11,326 (-31%)
avm_simulation_time_ms (AuthRegistry:set_authorized): 6.72 (-37%)
avm_simulation_time_ms (Token:_increase_public_balance): 76.9 (+697%)
avm_simulation_time_ms (FPC:pay_refund_with_shielded_rebate): 72.2 (-48%)
protocol_circuit_proving_time_in_ms (base-parity): 1,082 (-36%)
protocol_circuit_proof_size_in_bytes (public-kernel-tail): 27,204 (-21%)
protocol_circuit_num_public_inputs (public-kernel-tail): 457 (-33%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,759
1,551 (-1%)
704 (-1%)
756
768 (-1%)

proof_construction_time_sha256_30_ms
11,824
3,174
1,405 (-1%)
1,448
1,478

proof_construction_time_sha256_100_ms
44,081
12,457
5,647 (-1%)
5,546 (-1%)
5,514

proof_construction_time_poseidon_hash_ms
79.0 (+1%)
34.0
34.0
58.0
88.0 (+1%)

proof_construction_time_poseidon_hash_30_ms
1,529
418
200
229 (+4%)
265

proof_construction_time_poseidon_hash_100_ms
5,652
1,529 (-1%)
676
726 (-2%)
743 (-1%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
740
740
740

l1_rollup_calldata_gas
6,692
6,720
6,704

l1_rollup_execution_gas
611,959
612,138
612,273

l2_block_processing_time_in_ms
254 (-2%)
451
795 (-3%)

l2_block_building_time_in_ms
9,505
18,584 (-2%)
36,868 (-1%)

l2_block_rollup_simulation_time_in_ms
9,505
18,584 (-2%)
36,867 (-1%)

l2_block_public_tx_process_time_in_ms
7,901 (-1%)
16,781 (-1%)
34,975 (-1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
2,776
3,673 (-1%)

node_database_size_in_bytes
12,730,448
16,928,848 (+1%)

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Function
time_ms
bytecode_size_in_bytes

GasToken:_increase_public_balance
91.4
8,139

GasToken:set_portal
11.6
2,362

Token:constructor
118
31,107

FPC:constructor
94.8 (-4%)
24,011 (+7%)

GasToken:mint_public
77.7
6,150

Token:mint_public
90.1 (+8%)
11,720

Token:assert_minter_and_mint
76.9 (-14%)
8,028

AuthRegistry:set_authorized
:warning: 6.72 (-37%)
4,537

FPC:prepare_fee
273 (-3%)
8,812

Token:transfer_public
33.9 (+7%)
47,432

FPC:pay_refund
62.8 (-5%)
12,114

Benchmarking:increment_balance
1,008 (-1%)
7,450

Token:_increase_public_balance
:warning: 76.9 (+697%)
8,960

FPC:pay_refund_with_shielded_rebate
:warning: 72.2 (-48%)
12,663

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.158 (+3%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
2.17 (+1%)
3.79 (-4%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.113 (+1%)
0.107 (-4%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
11.0 (-6%)
18.0 (+1%)
30.8 (+1%)
59.9 (+1%)
112 (-1%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.105 (-6%)
0.105 (+1%)
0.0999
0.103 (+1%)
0.101

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
13.9 (-3%)
26.4 (+4%)
43.2
82.8
159 (-2%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.105 (-3%)
0.103 (+1%)
0.104
0.102
0.102 (+2%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
16.5
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
133 (+1%)
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.105
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
63,593 (-2%)
665,010 (-1%)

Transaction size based on fee payment method
| Metric |  |
| - |  |

I'm not sure why there's no gate counts diff, even though this is based on another PR that did produce changes. I'll wait for that other PR to get merged (or, well, for the next noir sync once that other PR gets merged on noir's side).


Click not handled after you return to a page

Thanks for this library! It's working well for me, except...
handleClick is assigned as an event listener to the DOM node in componentDidMount. However, if you navigate to another 'page' in your Single Page App, and then go back to your previous 'page', componentDidMount doesn't fire again for that first page, and the original event listener is lost? (That's what happens for me anyway - testing with Chrome v39). This means that when you now click another link on that first page, it won't be caught - it will result in a full page request.
How about also adding the event listener in componentDidUpdate? I tried it and it fixed the problem for me. (I was worried it might keep adding more and more event listeners, but it looks like addEventListener shouldn't allow duplicates*).
If this sounds good, it's just a case of adding:
componentDidUpdate: function() {
    this.getDOMNode().addEventListener('click', this.handleClick, false);
},

(I could issue a pull request if you like).
*Or an extra cautious way could be to do removeEventListener just before the addEventListener?

Can you get me some more details so I can reproduce?

Are you navigating to a url not handled by the router?
Is the componentWillUnmount lifecycle method not being called on the component you're adding the RouterMixin to?

If the DOM tree isn't changing, then the event listener shouldn't be lost.

Ah, that's good to know - perhaps I'm inadvertently doing something a bit strange then!
Let me see if I can reduce it down to a test case that I can share, then. I'll post an update here soon.

Any updates on what the issue was?

Sorry for the delay, Larry. Just trying to build a little test case now. Hope to post it here tonight or tomorrow.

OK, here is a quick test case: https://github.com/poshaughnessy/react-mini-router-test
NB. I used a standalone version of your library, that I made using browserify on the command line, because:

I didn't want to introduce a CommonJS compile step, for this simple test case
I tried installing it with bower, but it errored with: ENOTFOUND Package juliangruber/isarray=juliangruber/isarray not found?

Instructions / steps to reproduce the problem (also included in the README):

npm install
node server
Fire up localhost:3000 in the browser
(Bring up the Network tab in the debugging tools so you can see when a full page request is made)
Click Page one - this will load without a full page request
Click Page two - this now causes a full page request

Tested in Chrome v40 and Firefox v35.
Hope this helps!

@poshaughnessy Thanks! That will be extremely helpful in debugging. I've attached this issue to the 1.1 milestone, so expect to see it resolved this month.

There is actually a very simple workaround, at least for us it worked well - wrap the component produced by the route function (that is different for each route) into a <div> - which will get the listener attached to itself, does not change, and thus doesn't lose the listeners.
I.e. instead of
render: function() {
        return this.renderCurrentRoute();
    },

do this
render: function() {
        return (<div>{this.renderCurrentRoute()}</div>);
    },

in your root component with the RouterMixin.

I ran into this issue today. @jakubholynet 's solution worked.

Still running into this, @jakubholynet's solution helped me out! Thank you.  @larrymyers it'd be awesome if this was noted on the README if the code fix is a little ways off!


calculate and visualise statistics for the dataset

developed the script somef -> statistics.py . it does:

Preprocessing Functions: Functions to preprocess text and check for external links.
Initialize Statistics: Initializes dictionaries to count installation methods, results, code blocks, images, hyperlinks, and tokens for each repository.
Process Data: Iterates through each key in the JSON data to populate the dictionaries.
Calculate Installation Methods: Calculates the number of installation methods per repository.
Keyword Analysis: Analyzes the occurrence of specific installation keywords.
Trends and Observations: Prints trends based on the keyword analysis.
Installation Types: Categorizes installation types based on keywords.
Compile Statistics: Compiles overall statistics into a DataFrame and prints it.

visualisation is at readmes_statistics.py - it does:

Keyword Extraction: For each result in an item, the script preprocesses the text, uses CountVectorizer to find the top 10 keywords, and updates the keyword counters.
Heatmap Preparation: The script aggregates keywords for each repository and creates a DataFrame of the top 10 keywords.
Plot Heatmaps: Heatmaps are created for each repository with the top 10 keywords and their counts.

basic_statistics.py has more detailed info:

Line Counts: Calculate the total number of lines, blank lines, unique lines, and the ratio of blank and repeated lines.
Token Counts: Calculate the number of tokens, characters, repeated token ratio, unique tokens per line, and average tokens per line.
Code Block Counts: Count the occurrences of code block markers (```, `) and their counts per line.
Token Length: Calculate the average length of tokens.
N-gram Ratios: Calculate the ratio of unique bigrams and trigrams.
Stop Words: Count the number and ratio of stop words, and stop words per line.
Uncommon Words: Count the number of uncommon words (words not in WordNet).

updates:

run the cli somef but stop at N = 256. TODO: rerun and get all if needed now

updates:

the run cli somef still crashes due to storage limitation. only N=300 readmes has been parsed
now the script basic_statistics.py performs descriptives on many subjects such ids with and without installation instructions as well as visuals about programming languages been used per year (issue on data_created versus data_updated)
more detailed report on code blocks is stored in code_blocks_stats.py - the issue is that I cant create multiple heatmap per year - it only plots bars

visualisation is at readmes_statistics.py - it does:
* Keyword Extraction: For each result in an item, the script preprocesses the text, uses CountVectorizer to find the top 10 keywords, and updates the keyword counters.

* Heatmap Preparation: The script aggregates keywords for each repository and creates a DataFrame of the top 10 keywords.

* Plot Heatmaps: Heatmaps are created for each repository with the top 10 keywords and their counts.

readme_statistics.py removed as it does not provide any value.

developed the script somef -> statistics.py . it does:
* Preprocessing Functions: Functions to preprocess text and check for external links.

* Initialize Statistics: Initializes dictionaries to count installation methods, results, code blocks, images, hyperlinks, and tokens for each repository.

* Process Data: Iterates through each key in the JSON data to populate the dictionaries.

* Calculate Installation Methods: Calculates the number of installation methods per repository.

* Keyword Analysis: Analyzes the occurrence of specific installation keywords.

* Trends and Observations: Prints trends based on the keyword analysis.

* Installation Types: Categorizes installation types based on keywords.

* Compile Statistics: Compiles overall statistics into a DataFrame and prints it.

statistics.py to be removed

main updated:

re run extractor_all.py and got 716 repos out of 1800
last file: installation_readmes-01.json
04-basic_statistics.py runs over all again and 05-hypter_links.stats.py and 06-code-block-stats.py
double check how to process unknown data in the script


Multilanguage Support - Missing languageProfileId for Sonarr API Requests

Issue:
When having more than one Language Profiles enabled in Sonarr, every TV Show added within Searcharr defaults to English (or languageProfileId = 1)
As I like to watch some TV shows in English and other in German I manually select the language Profile in Sonarr when adding a new Series.
The parameter is missing in Sonarr API Documentation but I've confirmed that it's working with a CURL Request. You can also see the parameter in Chrome Dev Tools.
curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"qualityProfileId":4,"tvdbId":78107,"languageProfileId":2,"title":"The Office","titleSlug":"the-office","images":[{"coverType": "banner","url": "/MediaCover/166/banner.jpg?lastWrite=637961218386683368","remoteUrl":"https://artworks.thetvdb.com/banners/v4/series/401998/banners/62a90eafd0318.jpg"},{"coverType": "poster","url": "/MediaCover/166/poster.jpg?lastWrite=637961218388043342","remoteUrl":"https://artworks.thetvdb.com/banners/v4/series/401998/posters/62cc2623984e8.jpg"},{"coverType": "fanart","url": "/MediaCover/166/fanart.jpg?lastWrite=637961218390003303","remoteUrl":"https://artworks.thetvdb.com/banners/v4/series/401998/backgrounds/62a0d9f3da7a2.jpg"}],"seasons":[{"seasonNumber": 1,"monitored": true}],"path":"/tv/The Office"}' \
  http://sonarr:8989/api/v3/series?apikey=<API KEY>

Solution:
The easiest solution would probably be a static parameter in settings.py. The best solution would probably be a choice between different languageProfiles when adding a Series in Searcharr.
Thank you very much, Searcharr works great so far! 👍

It appears Sonarr is deprecating language profiles and adding language support to custom formats instead. After upgrading to Sonarr v4 in my test instance, the response to the languageprofile API endpoint has changed to this (I had multiple languages enabled in the language profile before, and it was not named "Deprecated"):
[
  {
    "name": "Deprecated",
    "upgradeAllowed": true,
    "cutoff": {
      "id": 1,
      "name": "English"
    },
    "languages": [
      {
        "language": {
          "id": 1,
          "name": "English"
        },
        "allowed": true
      }
    ],
    "id": 1
  }
]

As a result, I am going to remove language profile support from Searcharr v3.


STOR-1698: Fix volume provisioning when duplicate region or zone tags exist

https://issues.redhat.com/browse/STOR-1698
Clean backport of https://github.com/kubernetes-sigs/vsphere-csi-driver/commit/6e820bb5bd6f1fcdd5f1eaa74206d2fd31606904
Fix volume provisioning when duplicate region or zone tags exist (#2814)

* Remove duplicate hosts in GetHostsForSegment
* Ignore duplicate tags in GetTopologyLabels

/cc @openshift/storage @duanwei33

/retest
/label docs-approved
/label px-approved

An error occurred (AccessDenied) when calling the ListHostedZonesByName operation: User: arn:aws:iam::726924432237:user/ci-vmware-route-53 is not authorized to perform: route53:ListHostedZonesByName with an explicit deny in a service control policy ???
/retest

An error occurred (AccessDenied) when calling the ListHostedZonesByName operation: User: arn:aws:iam::726924432237:user/ci-vmware-route-53 is not authorized to perform: route53:ListHostedZonesByName with an explicit deny in a service control policy ???

Ah, there was an announcement, they're working on it.

Pre-merge verified with this open PR, provisioning succeed with duplicated tags like below:
OCP-DC region-A
    workloads-03 region-A zone-C
       host3 zone-C

/label qe-approved

/retest-required

/retest

/retest

Common flake, and everything else looks good.
/override ci/prow/e2e-vsphere-ovn-upgrade


update pom dependencies to avoid systempath

should resolve gradle-ide-3.3.jar as a normal Maven dependency

this is now obsolete since Gradle is used instead of Maven


Configure pss for memcache caching

This sets up the configuration necessary for the change introduced in https://github.com/dpla/primary-source-sets/pull/166

👍


Nlbwmon issue with Stack Protection Strong

When build with the following stack protection
Following is encountered
Mon Aug  7 10:40:33 2017 kern.info kernel: [   45.628673]
Mon Aug  7 10:40:33 2017 kern.info kernel: [   45.628673] do_page_fault(): sending SIGSEGV to nlbwmon for invalid write access to 00000000
Mon Aug  7 10:40:33 2017 kern.info kernel: [   45.637289] epc = 771a0c64 in libc.so[77180000+a0000]
Mon Aug  7 10:40:33 2017 kern.info kernel: [   45.642456] ra  = 564a91a8 in nlbwmon[564a4000+a000]
Mon Aug  7 10:40:33 2017 kern.info kernel: [   45.647550]

  @@ -178,7 +178,7 @@ menu "Global build settings"
     	choice
     		prompt "User space Stack-Smashing Protection"
     		depends on USE_MUSL
    -		default PKG_CC_STACKPROTECTOR_REGULAR
    +		default PKG_CC_STACKPROTECTOR_STRONG
     		help
     		  Enable GCC Stack Smashing Protection (SSP) for userspace applications
     		config PKG_CC_STACKPROTECTOR_NONE
    @@ -196,7 +196,7 @@ menu "Global build settings"

     	choice
     		prompt "Kernel space Stack-Smashing Protection"
    -		default KERNEL_CC_STACKPROTECTOR_REGULAR
    +		default KERNEL_CC_STACKPROTECTOR_STRONG
     		depends on USE_MUSL || !(x86_64 || i386)
     		help
     		  Enable GCC Stack-Smashing Protection (SSP) for the kernel

Using the version in lede-17.01 current

I have tested it is fixed with the commit
https://github.com/jow-/nlbwmon/commit/73bfd7f26728408b296a18d30994eb852ec2fc4a
Please push the patch to lede-current and lede-stable.
Thanks


what does report_task_failure do?

@task_failure.connect
def report_task_failure(task_id, exception, args, kwargs, traceback, einfo):
    client = get_raven_client()
    client.captureException(einfo.exc_info)

When I run celery, celery shows error message because of this code.
I changed code like this, and the error doesn't show again.
@task_failure.connect
def report_task_failure(task_id, exception, args, kwargs,
                        traceback, einfo, sender, signal):
    client = get_raven_client()
    client.captureException(einfo.exc_info)

But I cannot still understand why we use report_task_failure.
It shows exc_info, but I cannot use additional argument sender and signal.

I think it can use **kwargs

@item4 but report_task_failure is used like report_task_failure(sender=sender), report_task_failure(signal=INFO)

>>> def func(**kwargs):
...    print(kwargs.keys())
...
>>> func(a=1)
dict_keys(['a'])

@miaekim No problem.

I reverted the code to initial code. It works well.... but I cannot understand why these errors occurred. If I notice same error, I will report again.


Any plan on making the library work with Adonis V6 ?

Any plan on making the library work with Adonis V6 ?

Hello @danygiguere,
I have made a jwt guard for Adonisjs v6 here is the repository


Implement: Async.ForEachAsync()

Implement a static Async class with a ForEachAsync() method that works something like Parallel.ForEach() does for threading.
We're going to use this for inspiration but we're going to make our API look more like Parallel.ForEach().

I implemented two versions of the API, one that supports a CancellationToken and the other does not.  Stephen Taub's example linked above, includes support for an async selector that can be used to asynchronously decide which of the work items to process.  We're not implementing that right now; our methods simply iterate over all of the work items in the collection passed, which is probably the common case.
Here are the new APIs:
        /// <summary>
        /// Iterates over a set of work items and executes an async action for each item.
        /// The method returns when all of the actions have completed.
        /// </summary>
        /// <typeparam name="TWorkItem">The work item type.</typeparam>
        /// <param name="workItems">The work item collection.</param>
        /// <param name="action">The async action to be executed on each work item.</param>
        /// <param name="maxParallel">Optionally specifies the maximum number of tasks to execute in parallel (defaults to <c>1</c>).</param>
        /// <returns>The tracking <see cref="Task"/>.</returns>
        /// <exception cref="AggregateException">Thrown when any of the actions failed.</exception>
        /// <remarks>
        /// <para>
        /// The actions will be executed on threads from the thread pool which means
        /// that the number of tasks that can be executed in parallel will be limited
        /// by the number of available pooled threads.
        /// </para>
        /// <note>
        /// The order in which work items are executed is not defined.
        /// </note>
        /// </remarks>
        public static async Task ForEachAsync<TWorkItem>(
            IEnumerable<TWorkItem>  workItems,
            Func<TWorkItem, Task>   action,
            int                     maxParallel = 1)

        /// <summary>
        /// Iterates over a set of work items and executes an async action for each item.
        /// The method returns when all of the actions have completed.
        /// </summary>
        /// <typeparam name="TWorkItem">The work item type.</typeparam>
        /// <param name="workItems">The work item collection.</param>
        /// <param name="action">The async action to be executed on each work item.</param>
        /// <param name="cancellationToken">The cancellation token.</param>
        /// <param name="maxParallel">Optionally specifies the maximum number of tasks to execute in parallel (defaults to <c>1</c>).</param>
        /// <returns>The tracking <see cref="Task"/>.</returns>
        /// <exception cref="AggregateException">Thrown when any of the actions failed.</exception>
        /// <remarks>
        /// <para>
        /// The actions will be executed on threads from the thread pool which means
        /// that the number of tasks that can be executed in parallel will be limited
        /// by the number of available pooled threads.
        /// </para>
        /// <note>
        /// The order in which work items are executed is not defined.
        /// </note>
        /// </remarks>
        public static async Task ForEachAsync<TWorkItem>(
            IEnumerable<TWorkItem>                      workItems,
            Func<TWorkItem, CancellationToken, Task>    action,
            CancellationToken                           cancellationToken,
            int                                         maxParallel = 1)

DONE


replace "master" with "main" in docs pictures

Kudos to the team for the great job of flawless replacements of all "master" mentions with "main" in the docs! Multiple pictures in docs still use the word "master" for the main branch. I could, as nice as I can, replace the mentions in the pictures and submit a PR. Should I try it or is there a special person in the team who takes care of the graphics?

@karentamrazyan goof catch! I fixed the wording for all images in docs.

fixed


Add Contiv link for K8s networking

Add Contiv networking (http://contiv.io) or (https://github.com/contiv/netplugin) as a networking option for kubernetes in http://kubernetes.io/docs/admin/networking/

Done.


MIC-726 endpoint for reconciliation

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.66%. Comparing base (8104a30) to head (70addcd).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #183   +/-   ##
=========================================
  Coverage     91.65%   91.66%
- Complexity      721      723    +2
=========================================
  Files           130      130
  Lines          2205     2207    +2
  Branches        161      161
=========================================
+ Hits           2021     2023    +2
  Misses          128      128
  Partials         56       56

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Support for Swift Package Manager?

I'm a long time fan of DateTools and very excited to see you're working toward a Swift release! I'm wondering if there are any plans to support Swift Package Manager (SwiftPM) in the future? DateTools in Server-side Swift via SwiftPM would be amazing. Since DateTools would, hypothetically, be dependent only on Foundation and not UIKit or AppKit or anything like that, and Foundation has been ported to Linux with Swift 3, it seems like a lot of the building blocks are in place for the migration already.

Thanks for using DateTools! We definitely will support the swift PM. If you feel like you can help to get that up and running, feel free to issue a pull request!  There is still much to do.


closed #311: Parse.File sends incorrect Content-Type to API

See issue #311 for details and discussion.
Maybe test coverage should be improved for this functionality.

[Current coverage][cc-pull] is 86.97%

Merging [#313][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.01%

@@             master       #313   diff @@
==========================================
  Files            42         42
  Lines          3853       3855     +2
  Methods         660        660
  Messages          0          0
  Branches        812        814     +2
==========================================
+ Hits           3352       3353     +1
- Misses          501        502     +1
  Partials          0          0

Powered by Codecov. Last updated by [6c00cc8...54d4cd5][cc-compare]
[cc-base-branch]: https://codecov.io/gh/ParsePlatform/Parse-SDK-JS/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/ParsePlatform/Parse-SDK-JS/compare/6c00cc825e0f34a88b010ea90da0c89e1a6f7b4b...54d4cd5a2631439aee810b0019dda080c3e58e50?src=pr
[cc-pull]: https://codecov.io/gh/ParsePlatform/Parse-SDK-JS/pull/313?src=pr

I'll find some time to cover this with tests


Improve the performance of QM.update()

Provides a 60% speedup to QM.update() and CQM.set_objective().

Codecov Report

Merging #1163 (47ca5eb) into main (53eedbe) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1163      +/-   ##
==========================================
- Coverage   94.19%   94.17%   -0.03%
==========================================
  Files          88       88
  Lines        9355     9369      +14
==========================================
+ Hits         8812     8823      +11
- Misses        543      546       +3

Impacted Files
Coverage Δ

dimod/include/dimod/quadratic_model.h
95.95% <ø> (ø)

dimod/constrained.py
94.25% <100.00%> (-0.04%)
:arrow_down:

dimod/quadratic/quadratic_model.py
92.47% <100.00%> (-0.34%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53eedbe...47ca5eb. Read the comment docs.


fix jsonpb unmarshal when single enum value is provided for a repeate…

…d enum field

ping @bcmills

I'd like to follow up on this PR.  It would be nice to have this merged in if you can follow through the review comment.  Thanks.

Apparently this is a duplicate of #627, which was recently merged.


Crashes without error message / [feature request] treat subscripts as "mark" if no sequence present

Sometimes a numeric subscript (e.g., ₁ ) is used like a decoration rather than indicating an ordering in a sequence:
given
X₁ ∈ ℝ^(m×n)
X₂ ∈ ℝ^(n×k)

X₁X₂

currently this triggers some bug in the parser which causes it to crash without error message:
Exception: sequence item 0: expected str instance, int found

A workaround for now is to put backticks around everything:
given
`X₁` ∈ ℝ^(m×n)
`X₂` ∈ ℝ^(n×k)

`X₁``X₂`

so the feature request portion of this issue is to make this work without backticks (assuming of course that X_i isn't also defined/used anywhere). Not sure if this will be possible. At the very least, the parser shouldn't crash.

Now X₁ is the same as X_1.
I fixed the crash but still displayed the error msg

I am leery of treating subscripts differently depending on whether a sequence is present or not. That seems like it would lead to a lot of user confusion. I am comfortable with the rule that if you want "parsable" I❤LA to be left untouched as a variable name, you must use backticks. Beyond that, I'd like to see code examples with and without this rule.

Here's where it shows up in the gallery:
https://github.com/i-heart-la/i-heart-la.github.io/issues/16
https://github.com/i-heart-la/i-heart-la.github.io/issues/8


Kill non-json r2 usage

For example r2.cmd("iS") + regex 😰 to be replaced with r2.cmdj("iSj") -> https://github.com/pinkflawd/r2graphity/blob/master/graphity.py#L70

https://github.com/pinkflawd/r2graphity/blob/master/graphity.py#L426
-> pdj


swayr: 0.19.0 -> 0.20.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/swayr/versions.
meta.description for swayr is: "A window switcher (and more) for sway"
meta.homepage for swayr is: "https://git.sr.ht/~tsdh/swayr"
meta.changelog for swayr is: ""
Updates performed

Rust version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.20.0 with grep in /nix/store/kk8qwj40gkakrpkk5xb865d1n1jg9syi-swayr-0.20.0
found 0.20.0 in filename of file in /nix/store/kk8qwj40gkakrpkk5xb865d1n1jg9syi-swayr-0.20.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
swayr

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/kk8qwj40gkakrpkk5xb865d1n1jg9syi-swayr-0.20.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A swayr https://github.com/r-ryantm/nixpkgs/archive/181cd290c7a841f0bf1d93b849173bddb5af0997.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/kk8qwj40gkakrpkk5xb865d1n1jg9syi-swayr-0.20.0
ls -la /nix/store/kk8qwj40gkakrpkk5xb865d1n1jg9syi-swayr-0.20.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

swayr

Maintainer pings
cc @Artturin for testing.

Result of nixpkgs-review pr 186223 run on x86_64-darwin 1

1 package built:

swayr

Package builts and runs fine on NixOS 22.05 Linux x64


iOS Handle response with "Please upgrade app"

Encumbering the share use case with "new version available" popups would not be great UX.
When should would check? when should we prompt users? Acceptable timings are :

on Success screen
after failure
on Splash screen


Inchling can't break some blocks

Hey there!
First I wanted to thank you for making this origin! I think it's super cool that someone put into effort to make this! That said, I've been encountering a very strange issue. Basically while the inchling can break most blocks albeit much slower, for whatever reason, they can't break things like flowers and sugarcane. Not even standing in front of it and punching it for a really long time does anything. I have no idea why this might be the case, but if I can't even clear some grass out of the way, that's gonna prove to be a bit of a problem. I'd really appreciate if you could look into this issue because I'm not convinced that it's intended. Anyway,
I hope you're having a good day so far!

Resolved an issue with the "Slow Miner" power that didn't allow users to break certain blocks such as grass, flowers, crops, and sugar cane. This fix involves the addition of all breakable blocks to a JSON tag file, thereby fixing the unbreakable block issue as of version 2.0.2
https://modrinth.com/datapack/improved-inchling-origin/version/2.0.2


Why do centers get updated?

Hi,
I don't quite understand why the centers could get updated in David's implementation. Could anyone please explain the mechanism behind it?
My question is:
In facenet.center_loss(...) implementation, the centers are updated by a  scatter_sub OP and the resulting tensor is returned:
centers = tf.scatter_sub(centers, label, diff)
loss = tf.nn.l2_loss(features - centers_batch)
return loss, centers

However, the returned tensor(centers) is not used later in train_softmax.py:
prelogits_center_loss, _ = facenet.center_loss(prelogits, ...)

So why the centers could get updated?
Thanks.

@ugtony
I have been suspecting the same thing. Finally I got some time to do some testing. To narrow down, I use weight_decay_factor=0.0, and center_loss_factor=1.0, in order to analyze how reg_loss is changing overtime. I have two setup (1) current implementation (2) my own: put center_op to the training loop to make sure centers gets recalculated every batch.  RegLoss seemed to decrease much faster. Can't wait to see the results after more epochs. I hope the change could make the performance even better.
I had printed the centers before, the centers did change over time but kept very small. So I wasn't sure whether it is updated or not. Maybe the centers were shifted just because of the numerical error?

Hi @ugtony, @JianbangZ,
Thanks for spotting this! I guess the running of the center_op got dropped at some point. I added a control dependency in the center loss function on a local branch:
    with tf.control_dependencies([centers]):
        loss = tf.reduce_mean(tf.square(features - centers_batch))

I 'm currently training on casia with wd=2e-4 and cl=2e-2 and maybe the performance is slightly better after 37k iterations but it's too early to tell.
Please also note that the scaling of the center loss was changed some time ago, so the center loss factor should be multiplied by the batch size.

@davidsandberg I don't think the center loss factor should be multiplied by the batch size. Each iteration the softmax loss is also mini-batch based, same as center loss. So if you multiply by batch size, the other loss should be multiplied by batch size as well. Correct me if I am wrong.

@JianbangZ I think you're right, softmax and center loss are both normalized by batch size, so center loss factor could be kept unchanged while batch size changes.
But I guess what davidsandberg try to remind us is that the center loss factor used in his earlier version (implemented by l2_loss, which is sum of square difference) should be multiplied by batch size before used in the current version(mean of square difference).

@ugtony @davidsandberg Share my results here. I train with casia. I use center_loss_factor=5e-3, and when computing the loss, I multiplied it by batch_size=100. Eventually I got accuracy around 98.4% (pretty much the same as before), but the validation rate @ 0.001 is 0.93~0.94 (compared to 0.91~2 before).
Thoughts?

What value do you set for center_loss_alpha?
You set center_loss_actor to 5e-3 and mutually the loss by 100, so the effective factor is 5e-1? I think it might be too large according to Fig.5 in the paper.

my center_loss_alpha=0.6 (I try to set it around the middle of [0, 1] according to the paper)
I tried center_loss_factor = 5e-5, after multiplying by batch_size it equals to 5e-3 (which is OK according to the paper). However, the results was worse, worse than the center op fix. (accuracy 98.2%, validation rate ~ 90%)

I just finished my training with center_loss_factor=2e-2 and center_loss_alpha=0.95, the performance is pretty similar to before.
Another training with center_loss_alpha=0.5 is still running, but the performance isn't better so far.
Still cannot reproduce the results of the paper. :(


how to generate mesh?

which part of the code is doing Sec4.5 "Neural Non-rigid Tracking for 3D Reconstruction" of the paper?

same question.. Have u found that?


Copy editing + fixing typos :)

While the contribution is appreciated, I'm not sure all these changes are valuable or correct. The files in doc for example are notes for developers and comment on language design, so it feels strange to perform copy-editing on them.
Some of the changes lose the nuance of discussions about programming languages in particular, and others don't provide much beyond changing the flow of a sentence or two.


ErrorPlugin cannot be configured

v1.25.0
https://docs.php-http.org/en/latest/plugins/error.html
 Unrecognized option "error" under "httplug.clients.cms.plugins.1".

Any plans to support it?

Argh, i could use the service already:
plugins:
  - httplug.plugin.error


ELECT-1580: Clarity parse

Description
This PR is more of a draft to get edits/discussion from @st421, but aims to write a quick converter for Clarity election results. There are a few issues I want to chat about:

Right now it will only work for precincts. Getting county level results working should be a pretty minimal change, but I want to talk about implementation.
I haven't yet added a method to aggregate multiple county XML files into a consolidated feed. Right now, it will just return a list of parsed feeds if multiple XML files are passed to the converter. Want to talk about final implementation to see if we need it before I go there.
Will add unit tests once we make sure this conceptually works.

Jira Ticket
https://arcpublishing.atlassian.net/browse/ELECT-1580
Test Steps

elexclarity 105369 GA --level=precinct --filename="tests/fixtures/atkinson_precincts_11-3.xml"
elexclarity 105369 GA --level=precinct"

Hmm @st421 one thing I noticed today just going through the data... Gwinnett County in GA keys the races in English and Spanish (on the website and in the XML) https://results.enr.clarityelections.com/GA/Gwinnett/105437/web.264614/#/summary?v=270282%2F
This means 1) Keys/names for races aren't necessarily consistent from county to county and 2) we're going to have to do a little bit of cleanup somewhere in the pipeline.

Hmm @st421 one thing I noticed today just going through the data... Gwinnett County in GA keys the races in English and Spanish (on the website and in the XML) https://results.enr.clarityelections.com/GA/Gwinnett/105437/web.264614/#/summary?v=270282%2F
This means 1) Keys/names for races aren't necessarily consistent from county to county and 2) we're going to have to do a little bit of cleanup somewhere in the pipeline.

Just as a followup to this, the senate keys include:
'US Senate (Loeffler) - Special'
'US Senate (Loeffler) - Special Election'
'US Senate (Loeffler) - Special/Senado de los EE.UU. (Loeffler) - Especial'
'US Senate (Perdue)'
'US Senate (Perdue)/Senado de los EE.UU. (Perdue)'


Add ckpt_path option to LightningModule.test()

Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure to update the docs?
[x] Did you write any new necessary tests?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Fixes #2046.
If ckpt_path is "best" (default), it loads the best weights saved by ModelCheckpoint for the test loop.
If ckpt_path is a path to a checkpoint file, it loads the weights from the file for the test loop.
If ckpt_path is None, it uses the weights from the end of training for the test loop.
If model parameter is set, ckpt_path is ignored.
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

@williamFalcon @Borda I finally found some time to work on this, and I saw that you guys have already finalized the changelog entry for 0.8, so I wanted to check with you which version this should go. Obviously, I think it'd be great if we could squeeze this in with 0.8, but I don't want to create unnecessary delays in the release, so let me know!

This is an awesome addition, maybe we should add some documentation about this behavior to:
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/source/test_set.rst


COW from  Array into MutableArray

https://doc.rust-lang.org/std/borrow/enum.Cow.html
https://doc.rust-lang.org/std/sync/struct.Arc.html#method.make_mut

This is a very neat idea. I will mark it as investigation as I am not sure about its implications, but it is definitely something we should explore!

Refer COW in ClickHouse:
https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/COW.h

Wow.. this is interesting.

@sundy-li , would you consider this to be done? #794 seems to be exactly what this is about, right?


fix autopush deployment

Update autopush website-app to deploy in default-pool. This fixed deploying autopush when trying manually
Background:

autopush deployments have been failing since website cron tests have been added
there are 2 different node pools used in the autopush instance, one with 4 CPU and one with 8 CPU
I noticed that a lot of website-app pods were deployed in the 8 CPU node pool and they seemed to have a lot of restarts. In a past experiment, we've learned that website-app pods have trouble starting in node pools that have more CPU than 4.
So I tested moving all the cron jobs to the 8 CPU node pool and all the website-app pods to the 4 CPU node pool and it seemed to work. (Will see how well it does once this PR is in though)

thanks for the review!


Piotr/set sensor data

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: PiotrKarczmarz sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


NGEN seg faults

While working on #141 I encountered these issues:
Current behavior
With the refactored_example_realization_config.json:
[ngen]# ./ngen data/catchment_data.geojson "" data/nexus_data.geojson "" data/refactored_example_realization_config.json
Hello there 0.1.0
Building Nexus collection
Building Catchment collection
Running Models
Time step 0
Pass-through kernel being used for GIUH calculations; raw input value `0` being passed through and immediately returned as output.
        Catchment cat-87 contributing 0.000301563 m/s to
Segmentation fault

and with the data/example_realization_config.json
[ngen]# ./ngen data/catchment_data.geojson "" data/nexus_data.geojson "" data/example_realization_config.json
Hello there 0.1.0
Building Nexus collection
Building Catchment collection
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

Expected behavior
NGEN should run over the test data.
Steps to replicate behavior (include URLs)

Build the 'ngen' target and run with the command Nels posted in #141

Screenshots

@christophertubbs Didn't you make the original example_realization_config.json? I think I might need your assistance.

This issue arises from the CLI parsing of the subset strings.  An empty string "" gets added to the subset id vector, which is no longer empty, so the feature building doesn't add any features to the feature collection, but the realization manager isn't directly connected to the feature collection, so it still tries to run the model, but when feature information is accessed, it breaks.  A fix should be up shortly.


Regarding bulk conversation

Bro bro, its working fine. But can you plz update your code so that it can convert bulk pdisk link. Like in one post 10-15 links at once. I tried but failed sadly. Plz plzz do this for me. Thanks

Bro bro, its working fine. But can you plz update your code so that it can convert bulk pdisk link. Like in one post 10-15 links at once. I tried but failed sadly. Plz plzz do this for me. Thanks

can you just send me post on telegram so that i can check it
others all bulk conversion working fine

Bro bro, its working fine. But can you plz update your code so that it can convert bulk pdisk link. Like in one post 10-15 links at once. I tried but failed sadly. Plz plzz do this for me. Thanks

can you just send me post on telegram so that i can check it others all bulk conversion working fine

t.me/paritoshpky