Live DASH stream does not work in Shaka but works in dash-if-reference-player

What version of Shaka Player are you using?
2.0.6

Can you reproduce the issue with our latest release version?
Yes
Can you reproduce the issue with the latest code from master?
Yes

Are you using the demo app or your own custom app?
demo app (compiled from latest git because the test stream is not HTTPS so I could not test from online demo)

What browser and OS are you using?
Latest stable Chrome on Windows 10

What did you do?
Input DASH live stream (nimble streamer based) http://live1.karwan.tv/karwan.tv/vin-tv/manifest.mpd into demo app - see that streams does not start

What did you expect to happen?
Stream should start - the same stream seems to work fine at http://mediapm.edgesuite.net/dash/public/nightly/samples/dash-if-reference-player/index.html

What actually happened?
Shaka did not start

I have attached the debug logs
Using native EME as-is.
mediasource.js:49 Using native MSE as-is.
promise.js:118 Using native Promises.
vttcue.js:36 Using native VTTCue.
drm_engine.js:1278 It is recommended that a robustness level be specified. Not specifying the robustness level could result in unexpected behavior in the future, potentially including failure to play.
(anonymous) @ drm_engine.js:1278
shaka.media.DrmEngine.probeSupport @ drm_engine.js:1277
shaka.Player.probeSupport @ player.js:399
shakaDemo.init @ main.js:133
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40596.03366661072
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40596.03766655922
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40597.03766655922
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40597.03966665268
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40598.0406665802
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40598.04166650772
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40599.04266667366
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40599.04766654968
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40600.04566669464
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40600.04966664314
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
dash_parser.js:1091 Updating manifest...
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40601.04666662216
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40601.05066657066
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40602.04766654968
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40602.051666498184
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40603.04966664314
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40603.05266666412
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40604.051666498184
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40604.054666519165
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40605.053666591644
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40605.056666612625
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
dash_parser.js:1091 Updating manifest...
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40606.4076666832
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40606.40966653824
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40608.401666641235
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40608.40366649628
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40610.40566658974
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40610.40566658974
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
dash_parser.js:1091 Updating manifest...
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40612.41066646576
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40612.4116666317
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40614.401666641235
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40614.402666568756
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40615.40666651726
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40615.40666651726
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (video:2) cannot find segment: currentPeriod.startTime=0 lookupTime=40617.40366649628
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
streaming_engine.js:1117 (audio:1) cannot find segment: currentPeriod.startTime=0 lookupTime=40617.404666662216
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1117
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1069
shaka.media.StreamingEngine.update_ @ streaming_engine.js:958
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:860
dash_parser.js:1091 Updating manifest...

We encounter the same issue. We've been using dashjs and hear good words about shaka. But this issue is blocking us from moving to shaka

Attached is the very first mpd file fetched for our live stream. We would appreciate if you could point out what need to change for this manifest, in order for it to work.

Dashjs would issue the following requests and start to play. The segment index numbers are simply derived from t= and d= in the SegmentTimeline
Request URL:http://<IP_ADDRESS>:5555/sdash/LIVE$channel1/10.mpd/S!d2EKREFTSF9VTkVOQxIKVP7...8Bnw__/QualityLevels(1048576)/Fragments(video=Init)
Request URL:http://<IP_ADDRESS>:5555/sdash/LIVE$channel1/10.mpd/S!d2EKREFTSF9VTkVOQxIKVP7...8Bnw__/QualityLevels(64000)/Fragments(audio_482_und=Init)
Request URL:http://<IP_ADDRESS>:5555/sdash/LIVE$channel1/10.mpd/S!d2EKREFTSF9VTkVOQxIKVP7...8Bnw__/QualityLevels(1048576)/Fragments(video=14903730905310280)
shaka errors out with "cannot find segment" (see below). The manifst does not have UTCTiming tag but we added the following in our code, where manfiestUri is the uri of this particular manifest.
player.configure({
manifest: {
dash: {
'clockSyncUri': manifestUri
}
}
});
We would appreciate if you could point out what need to change for this manifest, in order for it to work.
mediakeys.js:52 Using native EME as-is.
mediasource.js:49 Using native MSE as-is.
promise.js:118 Using native Promises.
vttcue.js:36 Using native VTTCue.
player.js:557 No preferred audio language set.  We will choose an arbitrary language initially
(anonymous) @ player.js:557
playerEval.js:143 The video has now been loaded!
streaming_engine.js:1128 (video:1) cannot find segment: currentPeriod.startTime=0 lookupTime=1490373094.6326666
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1128
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1080
shaka.media.StreamingEngine.update_ @ streaming_engine.js:969
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:872
streaming_engine.js:1128 (audio:4) cannot find segment: currentPeriod.startTime=0 lookupTime=1490373094.6346667
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1128
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1080
shaka.media.StreamingEngine.update_ @ streaming_engine.js:969
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:872
streaming_engine.js:1128 (video:1) cannot find segment: currentPeriod.startTime=0 lookupTime=1490373095.6346667
shaka.media.StreamingEngine.lookupSegmentPosition_ @ streaming_engine.js:1128
shaka.media.StreamingEngine.getSegmentReferenceNeeded_ @ streaming_engine.js:1080
shaka.media.StreamingEngine.update_ @ streaming_engine.js:969
shaka.media.StreamingEngine.onUpdate_ @ streaming_engine.js:872
streaming_engine.js:1128 (audio:4) cannot find segment: currentPeriod.startTime=0 lookupTime=1490373095.6366665

mpd-shaka.txt
mpd file attached.

This is first time I report an issue. Not sure why the lines are crossed out.  Apologize for reporting one issue with multiple comments.

Hi @radiantmediaplayer, @ebeffim, we will take a look.  Thank you for the report.

The manifest at http://live1.karwan.tv/karwan.tv/vin-tv/manifest.mpd is failing due to clock sync issues with your server.
I fetched a copy of the manifest at unix timestamp<PHONE_NUMBER> (attached).  The manifest's availabilityStartTime attribute says 2017-03-25T14:59:56Z, which is timestamp<PHONE_NUMBER>.  This means the stream has been available for<PHONE_NUMBER> -<PHONE_NUMBER> = 106586 seconds.  So the largest timestamp in the media should be close to 106586 (give or take one segment).  The timeShiftBufferDepth attribute is 45 seconds, to the smallest timestamp in the media should be close to 106586 - 45 = 106541.
The first segment in the video stream's SegmentTimeline is at media time<PHONE_NUMBER>2.  Divide by timescale 90000, and you get time 133079.  The last timestamp in the timeline is<PHONE_NUMBER>2, + 6 segments with length 900000, which gives<PHONE_NUMBER>2.  Divide by timescale 90000, and you get time 133139.
So SegmentTimeline shows the range 133079 to 133139, while the MPD element shows the range 106541 to 106586.  The ends of those ranges are offset by 133139 - 106586 = 26553 seconds = roughly 7.3 hours.
Your server's clock is therefore off by 7.3 hours, which causes it to present an inaccurate timeline, which causes Shaka Player to request segments that won't be available until 7.3 hours in the future.
I don't know much about Nimble Streamer, so I can't say for certain how you should fix this, but I have a few guesses.  One of these should solve your problem:

If you're in control of your server and its clock, make sure your server is using NTP and always has an accurate clock.  This is critical for live streaming.  The DASH-IF Interoperability Points v4.0 has this to say on the subject:

3.5. Synchronization Considerations
In order to properly access MPDs and Segments that are available on DASH servers, DASH servers and clients should synchronize their clocks to a globally accurate time standard. Specifically it is expected that the Segment Availability Times as computed from the MPD according to ISO/IEC 23009-1 [4], section <IP_ADDRESS> and additional details in ISO/IEC 23009-3 [6], section 6.4 are accurately announced in the MPD.

If you're not in control of the server and its clock, you must provide a UTCTiming element in the manifest so that the client has a way to synchronize with the server's idea of the current time.  You may need to update Nimble Streamer, configure it, or file a support request with the author.  Again, quoting DASH-IF Interoperability Points v4.0:

4.7.2. Service Provider Requirements and Guidelines
If the Media Presentation is dynamic or if the MPD@availabilityStartTime is present then the service shall provide a Media Presentation as follows:

The segment availability times announced in the MPD should be generated from a device that is synchronized to a globally accurate timing source**, preferably using NTP.
The MPD should contain at least one UTCTiming element ...
If the MPD does not contain any element UTCTiming then the segments shall be available latest at the announced segment availability time using a globally accurate timing source.

Please feel free to reference this analysis and recommendations in any bug report you may file with Nimble.  Thanks!

Thanks for detailing your analysis. It seems that the manifest I included in my report has the same "time difference" issue. And here is my analysis of that manifest.
The publishTime="2017-03-24T16:32:18Z" (which is the same as the Date HTTP header) is<PHONE_NUMBER> sec and with availabilityStartTime="1970-01-01T00:00:00Z", it means that the stream has been available for<PHONE_NUMBER> seconds. with timeShiftBufferDepth of 50 second, the range of timestamp expected is between<PHONE_NUMBER> to<PHONE_NUMBER> seconds.
The first segment in the video stream's SegmentTimeline is at media time<PHONE_NUMBER>5310280. Divide by timescale 10000000, and we get time<PHONE_NUMBER>. Compare with expected starting time<PHONE_NUMBER>, the difference is<PHONE_NUMBER> -<PHONE_NUMBER> = 14408 seconds which is about 4 hours difference...

Hi,
I'm Nimble streamer team representative. I agree with your analysis.
Let me explain why your have availability time problem here.
When stream created we init availability stream time with current utc time. As provided manifest has timescale == 90000 this means source stream is mpegts. For mpegts streams time usually starts from arbitrary positive value. Moreover time may overrun and jump into the future from time to time and anyone who works with mpegts knows about that.
I know IOP requirements and in case if your push rtmp/rtsp stream what start from 0 your will not see such issues. In ideal world availability time requirements meat and everything will work fine. But consider the case when time on the server was wrong during manifest build or source stream moved timestamp back u

Hi,
I'm Nimble streamer team representative. I agree with your analysis.
Let me explain why your have availability time problem here.
When stream created we init availability stream time with current utc time. As provided manifest has timescale == 90000 this means source stream is mpegts. For mpegts streams time usually starts from arbitrary positive value. Moreover time may overrun and jump into the future from time to time and anyone who works with mpegts knows about that.
I know IOP requirements and in case if your push rtmp/rtsp stream what start from 0 your will not see such issues. In ideal world availability time requirements meat and everything will work fine. But consider the case when time on the server was wrong during manifest build or source stream moved timestamp back.
I don't want to say it's player issue but even DASH-IF already agreed there is something wrong with all these time processing in DASH. It looks like standard was made in assumption that there will not be any problems with time. But in anyway bitmovin and dash.js plays http://live1.karwan.tv/karwan.tv/vin-tv/manifest.mpd file.
You can easily check this here http://player.wmspanel.com/
Again I don't want to say shaka work not according to the spec. But if player see segments in manifest even with wrong start time I think it should player them. IOP doesn't provide us an answer how we should handle stream sources with timestamps that periodically reset and jump in the future. I see such streams every day

@joeyparrish I did contact Nimble streamer team and you can find their feedback above. I understand this is a grey area but as @alexpokotilo mentioned our concern is that this stream plays fine with other, notable, DASH-capable players. Shaka is a great DASH player and we find it much easier to deal with and more stable than other DASH capable players we have tested and used. I will get back to our user and explain a workaround in the meantime.

Hi,
we will arrange additional effort to follow IOP and DASH standard. Please consider to add additional robustness to the player as well

Hi @radiantmediaplayer, @alexpokotilo,
In the past (Shaka Player v1), we attempted to work around problems in DASH manifests such as this one.  Those work-arounds caused bugs and maintenance challenges that my team could not keep up with.  In some cases, these work-arounds event caused us to fail on IOP-compliant content.
Therefore, we decided that Shaka Player v2 would be more strict in how we interpret and implement the spec.  This means we won't try to cope with or fix up non-compliant content.  I believe that Shaka Player is a more stable, interoperable product because of that decision.
I apologize for being inflexible on this point, but the beauty of open source is that you can always make a fork and change Shaka's behavior until you are able to solve the content issue.  And when your content does follow IOP, you will be able to confidently play that content not just in Shaka Player, but in any interoperable player.
Thanks for reaching out, and please do let us know if you need anything else.

@joeyparrish that is allright. Your approach makes sense and we are happy with the features and stability that shaka 2 is currently bringing. We have a workaround for this specific content so I am closing now.


steps-to-upload-an-app-on-Playstore.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

@RandellDawson  any idea why the CI build failed for my commit?

Yes, I believe it was because you had uppercase letters in your folder name.

@RandellDawson  are you a part of the freecodecamp org?

I am a volunteer helping out with reviews for FCC.

Oh, that sounds good. I was wondering if there is a way I could volunteer too or is it like invite only.@RandellDawson

Thank you!


Multicolor/size issues

Setting colors per datapoint doesn't work:
import ipyvolume
x = np.array([1,2,3,4,5])
y = np.array([2,4,3,2,1])
z = np.array([9,8,3,4,1])
#c = np.array(['red','green','blue','yellow','cyan'])
#c = np.array([(1,1,0), (1,1,0), (1,1,0), (1,1,0), (1,1,0)])
#c = [[1,1,0], [1,1,0],[1,1,0],[1,1,0],[1,1,0]]
c = [1,1,0]
ipyvolume.quickscatter(x, y, z, color=c, size=0.1, marker="sphere")

The result is white spheres in every case.
Also, setting the size doesn't work:
import ipyvolume
x = np.array([1,2,3,4,5])
y = np.array([2,4,3,2,1])
z = np.array([9,8,3,4,1])
s = np.array([1, 2, 3, 4, 5])/10
ipyvolume.quickscatter(x, y, z, color='blue', size=s, marker="sphere")

The result is no spheres at all.
Probably I'm misunderstanding the API, but an example would help.
Finally, I think the case where the color is given as a 1D array of floats should be reserved for a colormap, NOT interpreted as an rgb value. By default, the matplotlib default colormap should be used, or a custom colormap given by the parameter cmap=.
Tagging @jeammimi

I think both cases were fixed in the last commit
Also try setting the size to 1, because apparently 0.1 is quite small.

In the first example, replace size by 10. Initially, size (or s argument) was a fraction of the box size, but since humans aren't that good with numbers between 0 and 1, I switched to percentages, so 100 is the size of the box/volume, 10 is 10% of it. It will be in the changelog for the next release, sorry for this, that's the fate of early adopters I'm afraid ;). For the 2nd example it's the same, try removing the /10. Also, i'm not adopting matplotlib's short argument names (s versus size), since we have tab-completion in the notebook, and that leads to more readable code.
I'm working on examples, stay tuned for the next release.
I agree with you on the colormap.
What is planned for color is these options:

shape is 0 dim, and it's a string, interpret as color
shape is 1 dim, items are strings, seperate color for each item
shape is 2 dim, items are strings, sequence of the above
no color scale / colormap is set
shape is 1 dim, items are floats, it should be of length 3 -> rgb values (possibly later 4 for opacity)
shape is 2 dim, items are float, it should be of shape (len(x), 3) -> rgb values
shape is 3 dim, items are float, it should be (sequence_length, len(x), 3) -> rgb values
color scale / colormap is set

shape is 0 dim, item if float, use the colorscale to map to color
shape is 1 dim, it's a seperate color for each item
shape is 2 dim, again a sequence

without having a default colormap/colorscale, it will try to interpret it as rgb values, when a colorscale is set, it will use that. For now you have to use matplotlib's colormaps directly (opacity is ignored):
import ipyvolume
import matplotlib.cm
x = np.array([1,2,3,4,5])
y = np.array([2,4,3,2,1])
z = np.array([9,8,3,4,1])
c = matplotlib.cm.afmhot(np.linspace(0, 1, len(x)))
ipyvolume.quickscatter(x, y, z, color=c, size=10, marker="sphere")

I'd like to delay colormap/colorscale in ipyvolume until https://github.com/ipython/ipywidgets/issues/1153 takes more shape.
Let me know if this answer your questions.

Thanks - it works!

👍  for letting me know


Alertmanager 0.21 docs

cc @juliusv @simonpasquier
closes #1672
Signed-off-by: Julien Pivotto<EMAIL_ADDRESS>

how much time does it take to have include indexed @juliusv ?

@roidelapluie Hm, my contact at Algolia told me that they reindex things every 24h unless they trigger things manually. It's weird though, I don't see my recent change to our Algolia config (https://github.com/algolia/docsearch-configs/commit/2c0c306f3b4139bd4bc89d45adbe9e3216b932dc#diff-faa65298ccae9f682c816a36a3e79a02) to index the include:true|false meta tag working yet. When I change the facetFilters to ['include:true'], nothing is found. I'll check with him about that...

looks OK now @juliusv

:+1: Awesome, thanks!


Parsed cancellation token down (from the control to MediatR and EFCore)

By parsing down the cancellation token we be able to cancel a request while calling the database.
That saves a lot of resources on the server.

Thank you very much for contributing.


Update all examples in run_config

[ ] Update to recent lfric version (after repo split), which will also fix rose-picker issue (using python, not python3)
[ ] Properly handle linking, most example will fail linking, likely because the wrong linker is used. Might need more logic for zero config, or the next issue:
[ ]   Potentially port our LFRic fab base class for all examples, which will properly allow compiler customisation
[ ]  ATM the site from which configuration scripts are picked up are hard-coded. Again, the previous issue would fix that, but if we don't add the base class, maybe add some other option (or just accept and document it)

Migrated to https://github.com/MetOffice/fab/issues/360


Fixup name generation for -fmicrosoft; uppercase type names

Following our discussion, here's a proposed fix for the type names. Please let me know if this works as intended. Thanks!

I tried this out today and it doesn't seem to work for my scenario which involves external types. Consider:
module Test
module B = LowStar.Buffer
assume
val _MY_EXTERNAL_TYPE : Type0

let p_MY_EXTERNAL_TYPE = B.pointer _MY_EXTERNAL_TYPE

let test (x:p_MY_EXTERNAL_TYPE) : bool = true

Extracting this using --codegen krml and then running krml -fmicrosoft -skip-compilation out.krml` produces
/*
  This file was generated by KaRaMeL <https://github.com/FStarLang/karamel>
  KaRaMeL invocation: /home/nswamy/everest/karamel/krml -fmicrosoft -skip-compilation out.krml
  F* version: 46cbaf03
  KaRaMeL version: d6db69ab
 */

#include "Test.h"

BOOLEAN TestTest(TestMyExternalType *X)
{
  return TRUE;
}

I pushed a fix. With your test file, I now get:
/*
  This file was generated by KaRaMeL <https://github.com/FStarLang/karamel>
  KaRaMeL invocation: krml out.krml -skip-compilation -fmicrosoft -d c-names
  F* version: ce8b1957
  KaRaMeL version: fb04bf2a
 */

#ifndef __Test_H
#define __Test_H

#include "krmllib.h"

typedef TEST__MY_EXTERNAL_TYPE *TEST_P_MY_EXTERNAL_TYPE;

BOOLEAN TestTest(TEST__MY_EXTERNAL_TYPE *X);

#define __Test_H_DEFINED
#endif

is this the desired behavior?

ping @nikswamy @tahina-pro ... should this be merged?

Thanks for this fix. I am merging it after offline discussion with @protz ... much appreciated!


Addition of more audio metrics

🚀 Feature

Motivation

Feedback based on the newly release audio package (https://twitter.com/ai_rishikesh/status/1412707569710166019), here are a couple of other audio metrics it would be great if we added:

PESQ (https://en.wikipedia.org/wiki/Perceptual_Evaluation_of_Speech_Quality)
STOI (https://ieeexplore.ieee.org/document/5495701)
MCD (https://github.com/MattShannon/mcd)

cc: @quancs
Pitch

Alternatives

Additional context

PESQ and STOI are in plan. The PESQ is limited by its copy right owner. And seems we cannot translate its c++ code to pytorch see here.
PESQ refs: https://github.com/vBaiCai/python-pesq, https://github.com/ludlows/python-pesq, https://github.com/aliutkus/speechmetrics
STOI refs: https://github.com/mpariente/pystoi, https://github.com/mpariente/pytorch_stoi, https://github.com/aliutkus/speechmetrics

Is it good to use pytorch to wrap the c++ implementation of PESQ, like other implementations?
@SkafteNicki

@Borda can you help with answering what to do here? They use MIT license but they are against replications

according to their statement, we cannot use even modified their code... :/
I guess the only way would be to integrate it as dependency or ask @vBaiCai @SKb10 if they are willing to contribute to TM

according to their statement, we cannot use even modified their code... :/
I guess the only way would be to integrate it as dependency or ask @vBaiCai @SKb10 if they are willing to contribute to TM

Can we use the c++ code of PESQ as @vBaiCai and @SKb10 do? If we add the c++ code into TM, is it a duplication?
The statement are copied below:
The user cannot:

1.	alter, duplicate, modify, adapt, or translate in whole or in
      part any aspect of the PESQ Algorithm and or PESQ Software
2.	sell, hire, loan, distribute, dispose or put to any commercial
      use other than those permitted below in whole or in part any
      aspect of the PESQ Algorithm and or PESQ Software

I checked the speechmetric implementation...
It uses the two implementation as dependency......

according to their statement, we cannot use even modified their code... :/
I guess the only way would be to integrate it as dependency or ask @vBaiCai @SKb10 if they are willing to contribute to TM

This might be the only way...

Sorry, I forgot our disscussion here... @Borda
I closed my pull request of pesq... and I have invited ludlows to do this job.

Can you give an invitation to mpariente for stoi? His stoi is widely used in audio community.

As for MCD, I am not familiar with it. You can invite the owner of  MCD package (https://github.com/MattShannon/mcd).


Dark mode setting being read incorrectly

Enabling dark mode in the settings is not being reflected in the services.
To Reproduce
Steps to reproduce the behavior:

Join dark mode in the settings (also enabling adaptive & universal dark mode)
Open/Reload a service
Open the dev tools for the service and read the console
You will see that the light theme is being set

Ferdi Version: 5.4.0
Electron: 6.0.10
Node.js Version: 12.4.0
Platform: Linux

There has been some enhancement shipped via https://github.com/getferdi/ferdi/commit/eff719b87c60097342d393922048662c32255d88 on this matter (available in 5.4.1-beta.2). But I recommend waiting for the next release before updating as the current betas have known issues.

Closing in favor of https://github.com/getferdi/ferdi/issues/158.


[#5916] improve(auth): Remove AuthorizationPlugin single instance implement

What changes were proposed in this pull request?

Remove AuthorizationPlugin single instance implement in the BaseAuthorizaiton.java
Updatge ITs codes.

Why are the changes needed?
Fix: #5916
Does this PR introduce any user-facing change?
N/A
How was this patch tested?
CI Passed.

@xunliu
Could you please fix the CI. I have checked it and it's all good for me.


use Promises for tasks

We're writing majority of code using async/await these days. So our first thought was to use it for kue-tasks too. Something like this:
queue.process('hello', async (job) => {
  let tmpResult = await someLongBoringOperation(job.data.name);
  await otherLongOperation(tmpResult, job.data.planet);
});

That doesn't work as kue doesn't distinguish promises from regular functions. Workarounds are mentioned in Readme, but they hurt elegance of app code.
Is there a chance that kue will have "native" support for promises?
The only complication I see is optional workerCtx param. Otherwise, the code should be trivial

I'm thinking of re-writing Kue 1.0 with bluebird and so any contributions are welcome :+1:

Why not standard es6 promises? I personally find it slightly annoying when third party modules use their own promise implementation.

:+1:

+1

Why not standard es6 promises?

I'm not sure if ES6 restriction is a good choice for Kue users, of which many may be on lower node.js versions currently!

@behrad there's https://github.com/stefanpenner/es6-promise or https://github.com/zloirock/core-js... Or going all in with a babel compilation step before publishing on the npm registry.

@olalonde http://programmers.stackexchange.com/questions/278778/why-are-native-es6-promises-slower-and-more-memory-intensive-than-bluebird
@behrad +1 for bluebird

I would suggest using https://github.com/kevinbeaty/any-promise
This lets users choose which Promise implementation they prefer.

I think you should something like this line
https://github.com/glazedio/frisbee/blob/master/src/frisbee.js#L15
so the user can use the polyfill he wants,
https://github.com/matthew-andrews/isomorphic-fetch is a good option because works on browser and on node

I would suggest using https://github.com/kevinbeaty/any-promise
This lets users choose which Promise implementation they prefer.

👍

Promises are an ECMAScript standard now though... you don't see that many npm modules using any-array or equivalent to let uses chose which Array implementation they prefer.

Would love to see standard promises used!  Authoring my nodejs code in TypeScript these days, so I'm definitely at the "await buffet"!

Just in-case someone if looking for promisifying the current version of Kue with Bluebird
var kue = require('kue');
var bPromise = require('bluebird');
bPromise.promisifyAll(kue.Job.prototype);
bPromise.promisifyAll(kue.Job);

var queue = kue.createQueue(); // kue.createQueue() returns a singleton instance of queue object
//  so its safe to call careateQueue() multiple times in your code
bPromise.promisifyAll(queue.constructor.prototype);

Are there any plans to support promises? I'd love to help implementing that if it is something you guys want to add

Bluebird does not seem to correctly promisify queue.process method, but surely it can be done manually with something like:
const queue = kue.createQueue(options)

queue.processAsync = (name, concurrency, handler) => {
  return this.queue.process(name, concurrency, (job, done) => {
    return handler(job)
      .then(() => done(null))
      .catch(done)
  })
}

queue.processAsync('myJob', 1, async job => {
  await Promise.resolve('one')
  await Promise.resolve('two')
  return Promise.reject(new Error('Not implemented'))
})

Bluebird does not seem to correctly promisify queue.process method:
Bluebird.promisifyAll(kue.prototype)

But surely it can be done manually with something like:
const kue = require('kue')
const queue = kue.createQueue(options)

kue.prototype.processAsync = (name, concurrency, handler) => {
  return this.queue.process(name, concurrency, (job, done) => {
    return handler(job).then(() => done(null)).catch(done)
  })
}

And used as follows:
const queue = kue.createQueue(options)

queue.processAsync('myJob', 1, async job => {
  await Promise.resolve('one')
  await Promise.resolve('two')
  return Promise.reject(new Error('Not implemented'))
})

@ruchevits i realize your post is a few months old, but this is no longer working. this.queue returns undefined.

Can someone explain what the above method does?

@notflip if you look at the proposal it must not to be working, since it's just a proposal. What are you doing and what are you trying to achieve?


knot-dns: 3.0.6 -> 3.0.7

Fixes various bugs and introduces a few requested features.
https://gitlab.nic.cz/knot/knot-dns/-/tags/v3.0.7

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

21.05: the update is focused on bugfixes, and I believe upstream is being sufficiently careful.

Result of nixpkgs-review pr 127044 at 28dd3b61 run on aarch64-linux 1

2 packages built successfully:

knot-dns
knot-resolver

@GrahamcOfBorg test knot


fix: ValueError: sni_callback cannot be set on TLS_CLIENT context

fixes #55

Awesome, thanks. My guess is that the issue came with this change: https://github.com/python/cpython/pull/25455
I can't test right now, but as soon as someone confirms I'll merge


Add MSECost

Context: Add a MSECost class as discussed in https://discuss.pennylane.ai/t/package-for-classification/395
Description of the Change: Added first draft of both the class and its related tests, mainly to converge on the requirements.
Benefits: Lower barrier to entry for ML applications.
Possible Drawbacks:
Related GitHub Issues:

Thanks @trbromley ! I fixed the formatting errors, so now I have a clearer idea of the code conventions.
Before I move on to add more tests and docs, do you agree with the cost function being just (self.qnodes(*args, **kwargs) - target) ** 2, which assumes a single value to compare with the target (e.g. no batch of values to calculate the mean of)?

Thanks @trbromley ! I fixed the formatting errors, so now I have a clearer idea of the code conventions.

Awesome!

Before I move on to add more tests and docs, do you agree with the cost function being just (self.qnodes(*args, **kwargs) - target) ** 2, which assumes a single value to compare with the target (e.g. no batch of values to calculate the mean of)?

It looks good to me, also talking with @josh146 one thing in the back of our minds was whether we'd need to write a similar function to qml.sum, which lazily evaluates the QNodeCollection in a way that works with all interfaces. However, I think the way you have it now ((self.qnodes(*args, **kwargs) - target) ** 2) should work independent of the interface.
You raise a good point regarding batching! Currently the qnode is called in a very general way: self.qnodes(*args, **kwargs). If we were to loop over a batch we would have to think carefully about how the user inputs the data - for example in KerasLayer we require qnodes to have an inputs argument. At the same time, you're right that it's not really mean squared error!
I just had a look at how this is handled in other ML libraries and they typically allow a reduction argument to specify to calculate, e.g., the sum or the mean.
So I think we can either:

Create a SquaredError class along with other fundamental functions like AbsoluteError
Converge on how we would allow for batching
Create a Mean class that is fed these functions, as well as some helper classes like MeanSquaredError for specific instances

Or:

We directly make MeanSquaredError now which also has a reduction argument and we decide on how the batching works.

What are your thoughts?

Before I move on to add more tests and docs, do you agree with the cost function being just (self.qnodes(*args, **kwargs) - target) ** 2, which assumes a single value to compare with the target (e.g. no batch of values to calculate the mean of)?

It looks good to me, also talking with @josh146 one thing in the back of our minds was whether we'd need to write a similar function to qml.sum, which lazily evaluates the QNodeCollection in a way that works with all interfaces. However, I think the way you have it now ((self.qnodes(*args, **kwargs) - target) ** 2) should work independent of the interface.

I have actually tested it with all the interfaces and seems to work. Do you have a case in mind that could make it fail?

I just had a look at how this is handled in other ML libraries and they typically allow a reduction argument to specify to calculate, e.g., the sum or the mean.

Yes, PyTorch for instance does that.

So I think we can either:
* Create a `SquaredError` class along with other fundamental functions like `AbsoluteError`

* Converge on how we would allow for batching

* Create a `Mean` class that is fed these functions, as well as some helper classes like `MeanSquaredError` for specific instances

Or:
* We directly make `MeanSquaredError` now which also has a reduction argument and we decide on how the batching works.

I think that the latter is both easier to do and more compatible with the other libraries. Maybe a check on the shape of the input (e.g. nrows > 1) could be enough?

I have actually tested it with all the interfaces and seems to work. Do you have a case in mind that could make it fail?

This should work fine as is, since ((self.qnodes(*args, **kwargs) - target) ** 2) only contains subtraction and exponentiation, which will be understood by all ML libraries. Anything more complicated, and we would have to take more care :)

@nvitucci, let's go with SquaredError (and could potentially add AbsoluteError in this PR or another*), i.e., without batching.
We still need to decide how batching should best be implemented in PL so best to avoid it here :+1: After we've finished with this PR, we can have a think about an interesting next thing to focus on!
So I think all that remains is to add details to the docstrings.
*note that AbsoluteError may require something more complicated, like a lazy abs similar to qml.sum.

@nvitucci, let's go with SquaredError (and could potentially add AbsoluteError in this PR or another*), i.e., without batching.

Sure, we can do just this in the current PR. Just a question: how would the class be called then? Just SquaredError instead of MSECost? Or maybe SquaredErrorCost? Or some abbreviation such as SqErrorCost or SECost?

Good question! What about SquaredLoss? Looking here, maybe we should reserve cost for things applied over the batch.

@trbromley , I was just thinking the same thing! This way it would be even more consistent with PyTorch terminology as well (e.g. https://pytorch.org/docs/stable/nn.html#mseloss). Agreed!

Ah sorry, I actually meant SquaredErrorLoss. Same for you?

Ah sorry, I actually meant SquaredErrorLoss. Same for you?

:thinking: I'm neutral, with a slight preference to have shorter if possible, but up to you :smile:

Thanks @trbromley ! I decided to go for SquaredErrorLoss because it echoes the MSELoss from PyTorch, but I am still open to different opinions. I also added more tests and took a stab at improving the docs, hopefully following the existing conventions.

Thanks @trbromley for your comments! Should I mark them "resolved" by myself, or do you prefer to double-check and do it?
I'll be very happy to add myself as a contributor :) Should I add this as a "New feature" in release 0.10.0?

I'll be very happy to add myself as a contributor :) Should I add this as a "New feature" in release 0.10.0?

Yes feel free to add a sentence or two in the New features since last release part :smile:

Hey @trbromley, @josh146, I think I've done all the changes. I have one final question: is it ok to still have the module be called cost even if it only contains a Loss class so far? I think it will make sense to have both cost and loss classes there, but could it be unclear?

Thanks @nvitucci, great to have this nearly in (just waiting on @josh146 to approve after the requested changes).

Hey @trbromley, @josh146, I think I've done all the changes. I have one final question: is it ok to still have the module be called cost even if it only contains a Loss class so far? I think it will make sense to have both cost and loss classes there, but could it be unclear?

I think it's ok for now, let's see how the cost module evolves. The qnn subpackage is relatively new and we're still feeling it out, so it's likely things will move about and change as we figure out the best way to do things :rocket:
We should also talk about the next feature to work on if you'd like to keep contributing :smile:

I think it's ok for now, let's see how the cost module evolves. The qnn subpackage is relatively new and we're still feeling it out, so it's likely things will move about and change as we figure out the best way to do things :rocket:

Agreed!

We should also talk about the next feature to work on if you'd like to keep contributing :smile:

I absolutely do! Is the discussion forum the best place to start drafting some other ideas?


Semantic release should generate setup.py as well.

With PR #259 in mind, this change might be already obsolete:
diff --git a/pyproject.toml b/pyproject.toml
index bcd8ee3..29d4217 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -189,4 +189,4 @@ version_variable = "src/package/__init__.py:__version__"
 upload_to_pypi = false
 commit_subject = "bump: release v{version}"
 changelog_file = "CHANGELOG.md"
-build_command = "pip install 'flit >=3.2.0,<4' && flit build"
+build_command = "pip install 'flit >=3.2.0,<4' && flit build --setup-py"

The trusted builder calls
https://github.com/jenstroeger/python-package-template/blob/1cd7aa0cb52b2395aeef33c8f46b914ac45b92e4/.github/workflows/build.yaml#L72-L76
which in turn does already generate the setup.py for the source distribution:
https://github.com/jenstroeger/python-package-template/blob/1cd7aa0cb52b2395aeef33c8f46b914ac45b92e4/Makefile#L127-L137

Closing, see also https://github.com/jenstroeger/python-package-template/pull/259#discussion_r924344463.


refactor: deduplicate util function from admin/server modules

…single utils class
Signed-off-by: Matt Furlong<EMAIL_ADDRESS>
Commit Message: Deduplicate utility function that was copy-pasted from admin module into the server module.
Additional Description:
Risk Level: Low.
Testing: Unit test was refactored into server directory.
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

@furlongmt the presubmit is legit, the fix is in the output: https://dev.azure.com/cncf/envoy/_build/results?buildId=123048&view=logs&j=55bc2ab9-36c1-5ab8-58a6-8c258728eeff&t=33b57bf9-ef13-58c8-bc2b-f804aeb16d39&l=606

Adding @lizan for cross-company review

@furlongmt The presubmit failures are legitimate (one of them was that the namespace wasn't specified, i.e. Utility::serverState instead of just serverState)


home-assistant-custom-components.midea_ac_lan: add missed dependency

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 353216

x86_64-linux

:white_check_mark: 1 package built:

home-assistant-custom-components.midea_ac_lan


The enumerated list is being turned into a series of 1s

tested with https://github.com/stateful/baldaquin/blob/main/README.md (internal repo). The enumerated list is being turned into a series of 1s... I know this might not be trivial but I wonder if it's possible to parse them as is (text) vs an enumerated list?
Originally posted by @sourishkrout in https://github.com/stateful/runme/pull/71#pullrequestreview-1209125271

Tested this again according to https://github.com/stateful/runme/pull/71#issuecomment-1342546996. Still getting a series of 1s. Anything I'm missing?

@sourishkrout which version have you tried? Please print it with runme --version.
➜ ./runme --version
runme version stateful 0.4.0-rc.0-4-geef463c-eef463c (eef463cd8aeb5f101badceb5524ae9249e7ec54c) on 2022-12-12T11:25:54Z

➜ ./runme fmt --flatten "https://raw.githubusercontent.com/stateful/baldaquin/main/README.md?token=REDACTED"
[...]

## Development

1. Ensure you have [dev](https://github.com/stateful/dev) setup and running

2. Install MacOS dependencies

```sh
brew bundle --no-lock
```

3. Setup pre-commit

```sh
pre-commit install
```

[...]

In the extension it also looks correct:

Weird. This is what I'm using: runme version stateful 0.4.0-rc.0-1-gdf03678-df03678 (df0367895dbb401dbed554bc90e2f673c025f3d7) on 2022-12-12T13:47:46Z
On Mon, Dec 12, 2022 at 6:30 AM Adam Babik @.***> wrote:

@sourishkrout https://github.com/sourishkrout which version have you
tried? Please print it with runme --version.
➜ ./runme --version
runme version stateful 0.4.0-rc.0-4-geef463c-eef463c (eef463cd8aeb5f101badceb5524ae9249e7ec54c) on 2022-12-12T11:25:54Z
➜ ./runme fmt --flatten "https://raw.githubusercontent.com/stateful/baldaquin/main/README.md?token=REDACTED"
[...]
Development

Ensure you have dev setup and running

Install MacOS dependencies

brew bundle --no-lock

Setup pre-commit

pre-commit install

[...]
In the extension it also looks correct:
[image: Screenshot 2022-12-12 at 12 29 41]
https://user-images.githubusercontent.com/277870/207034465-9e1f644c-f8b2-41dc-b677-b34e67367033.png
—
Reply to this email directly, view it on GitHub
https://github.com/stateful/runme/issues/78#issuecomment-1346318565, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5FH4VOCN4HZDZCVBVJULWM4EFPANCNFSM6AAAAAAS3CS7C4
.
You are receiving this because you were mentioned.Message ID:
@.***>

OK, you were right 🤦 . Sorry, turns out the series of 1's was in the underlying file.


error in loading pharo 8

There is an error when trying to load Willow-bootstrap and willow-JQueryUI using
the Metacello script
(prior to this Willow 12.1.0 was loaded first in Pharo 8 / Windows 10)
Possibly this has something to do with the following line in setUpDependencies:
baseline: 'Willow' with: [ spec repository: 'github://ba-st/Willow:v12/source' ]
This is worked around by pulling in BaselineOfWillow-Bootstrap / JqueryUI thru iceberg, then editing Willow:v12 to Willow:12.1.0. it then successfully loads thru iceberg
right-click metacello(default)
as responded Probably v12 must be equivalent to the latest v12.x release.

@sanjayminni are you doing something like this??
Metacello new
	baseline: 'Willow';
	repository: 'github://ba-st/Willow:v12.1.0/source';
	load.

Metacello new
	baseline: 'WillowBootstrap';
	repository: 'github://ba-st/Willow-Bootstrap:v11.3.2/source';
	load.

If that is the case you must be getting a debugger with some kind of Metacello conflict message.
Your options here are:

Load Willow using v12 instead of an specific version (or just let Willow-Bootstrap and Willow-JQueryUI load it for you, you don't need to load the dependencies by hand). All the latest major versions are prepared to work without conflicts, so if you load Willow:v12 and Willow-Bootstrap:v11 everything will just work.

If you still want to load an specific version and use it in the dependent projects you need to lock the Metacello config. For example:

Metacello new
	baseline: 'Willow';
	repository: 'github://ba-st/Willow:v12.1.0/source';
	lock;
	load.

Metacello new
	baseline: 'WillowBootstrap';
	repository: 'github://ba-st/Willow-Bootstrap:v11.3.2/source';
	onLock: [ :lock | lock honor ];
	load.

BTW, this is a general problem with Metacello not Willow specific.

Yes I am getting the debugger with Metacello conflict message

I had loaded Willow separately first followed by trying to load Willow-Bootstrap
Metacello new
baseline: 'Willow';
repository: 'github://ba-st/Willow:release-candidate/source';
load.
Metacello new
baseline: 'WillowBootstrap';
repository: 'github://ba-st/Willow-Bootstrap:release-candidate/source';
load.
I will work thru alternatives suggested by you, but in the meanwhile have loaded it by editing BaselineOfWillowBootstrap
I wish I had known of this Metacello quirk earlier as I have faced this on several occasions with different packages


added generateRandomObject method

For the nested objects, to reach StackOverFlow in java, the object must exceed 320k of nested objects. I don't think that's it could an object with 320k nested objects. The only case we need to watch out for is an object that composes itself (like singleton pattern).

Yes, I am talking about that case @HamzaBenyazid


Add an intro message for vagrant up/reload

Just posting ideas...
==> drupalvm_project: Mounting NFS shared folders...
==> drupalvm_project: Mounting shared folders...
    drupalvm_project: /var/www/drupalvm => /Users/cindy/Projects/Personal/drupal-vm
==> drupalvm_project: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> drupalvm_project: flag to force provisioning. Provisioners marked to run always will still run.

==> drupalvm_project: Machine 'drupalvm_project' has a post `vagrant up` message. This is a message
==> drupalvm_project: from the creator of the Vagrantfile, and not from Vagrant itself:
==> drupalvm_project:
==> drupalvm_project: Your Drupal VM Vagrant box is ready to use!
==> drupalvm_project: * Visit the dashboard for an overview of your site: http://dashboard.drupalvm-project.dev (or http://<IP_ADDRESS>)
==> drupalvm_project: * You can SSH into your machine with `vagrant ssh`.
==> drupalvm_project: * Find out more in the Drupal VM documentation at http://docs.drupalvm.com

My reasoning is that rather than account for all the different scenarios, just make the message useful for beginners and let eg teams who have some fancier setup (disabled dashboard, blabla) either a) disable the message by setting vagrant_post_up_message: false or b) set their own welcome message to introduce co-workers to the VM.

I like this idea! Does vagrant_post_up_message: false actually work, though? It seems like it would cause the else to be triggered...

Oh right! I'll fix that.

Works for me!


After inserted 1 billions data in to tidb, the one of TiKV never able to startup, neither restart.

Bug Report

What version of TiKV are you using?

[root@ip-172-16-5-33 bin]# ./tikv-server --version
TiKV
Release Version:   5.2.2
Edition:           Community
Git Commit Hash:   7acaec5d9c809439b9b0017711f114b44ffd9a49
Git Commit Branch: heads/refs/tags/v5.2.2
UTC Build Time:    2021-10-25 12:00:54
Rust Version:      rustc 1.56.0-nightly (2faabf579 2021-07-27)
Enable Features:   jemalloc mem-profiling portable sse protobuf-codec test-engines-rocksdb cloud-aws cloud-gcp
Profile:           dist_release

What operating system and CPU are you using?

40 cpus, 2 numa nodes.
processor	: 39
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
stepping	: 1
microcode	: 0xb00001f
cpu MHz		: 1994.958
cache size	: 25600 KB
physical id	: 1
siblings	: 20
core id		: 12
cpu cores	: 10
apicid		: 57
initial apicid	: 57
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 invpcid_single intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
bogomips	: 4399.36
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

Steps to reproduce

Using BR to restore 60K empty tables (60K table generate by betting)
Using betting to generate 1 billions data
nohup ./bin/go-ycsb load mysql -P workloads/betting -p recordcount=1000000000 -p mysql.host=<IP_ADDRESS> -p mysql.user=root -p mysql.port=44000 --threads 200 -p dbnameprefix=test_betting_ -p databaseproportions=1.0 -p unitnameprefix=unit1_ -p unitscount=200 -p tablecount=60000 -p loadbatchsize=1000&

Manual: https://pingcap.feishu.cn/docs/doccnijeesmUrKiHLHay6rMwnaf
What did you expect?
TiKV can work without problem.
What did happened?
TiKV down and never able to startup

Error log from tikv_stderr:

[2021/12/01 15:23:21.831 +08:00] [FATAL] [mod.rs:376] ["logger encountered error"] [err="Cannot allocate memory (os error 12)"]

other error log:
28492 [2021/12/01 16:05:09.604 +08:00] [FATAL] [server.rs:831] ["failed to start node: Engine(Other("[components/raftstore/src/store/fsm/store.rs:1056]: \"[components/raftstore/src/store/peer_storage.rs:768]: [region 36305] 36307 validate state fail: Other(\\\"[c      omponents/raftstore/src/store/peer_storage.rs:678]: term of raft state < commit term of apply state, region 36305, raft state hard_state { term: 13 vote: 36307 commit: 815 } last_index: 815, apply state applied_index: 815 commit_index: 815 commit_term: 14 truncate      d_state { index: 772 term: 11 }\\\")\""))"]
Kernel log:

dmesg -T:

Linux has enough memory:

check NUMA nodes:

more log in <IP_ADDRESS>:/data4/log/

It's reproduced! We have met this panic 2 times in users' environments and haven't located the root cause.
term of raft state < commit term of apply state, region 36305, raft state hard_state { term: 13 vote: 36307 commit: 815 } last_index: 815, apply state applied_index: 815 commit_index: 815 commit_term: 14 truncate d_state { index: 772 term: 11 }

The region has just applied catching up logs for merge before TiKV is crashing for OOM. When catching up logs in merge, source peer only updates logs without changing any properties in raft state machine:
https://github.com/tikv/tikv/blob/237fe392eccc003cb4554dd58b03fa5cf5464f4d/components/raftstore/src/store/peer.rs#L824-L828
So when handling raft ready, it can persist the wrong term with the correct index and logs, hence corrupt the internal states. Marking the corrupted peer as tombstone may lead to deadlock or OOM depends on whether merge has finished. The safest way is to scale out the panic store.
To fix the bug, perhaps we should construct the whole AppendLog message to let raft state machine update its all states correctly. Otherwise we may still hit another pitfall in the future.

The problem is introduced by https://github.com/tikv/tikv/pull/9982 since 5.0.1. /cc @NingLin-P
Because min_matched is set to min_committed, so even an old leader is crashed and not online, the new leader can still propose a the prepare merge, which will include log entries from different term.

The problem is introduced by #9982 since 5.0.1. /cc @NingLin-P

@BusyJay  Sorry I don't get it.

so even an old leader is crashed and not online, the new leader can still propose a the prepare merge, which will include log entries from different term

So if the old leader crashed, region merge should be forbidden? forever? And how does whether min_matched is set to min_committed could affect whether there are log entries from different term? And how does if there are log entries from different term could trigger this issue as it is about comparing the term of the follower and the term of log entries, where there must have log entries from the current leader's term?
IMO, the fundamental fault is not changing the follower's term while changing its log entries, and that causes the follower's term even less than the term of log entries that it already has. And it doesn't seem a good idea to me to rely on any external checks to cover up for such fundamental fault.

So if the old leader crashed, region merge should be forbidden? forever?

No, PD can remove the failed peer and merge can be resumed.

And how does whether min_matched is set to min_committed could affect whether there are log entries from different term?

In the past, when a new leader is elected without hearing responses from all followers, it won't propose prepare merge. So when prepare merge is proposed, all followers' term should be as large as leader. So all logs in the gap should have "log's term <= any(followers' term)". So it's OK not to check term in the past.
After setting min_matched to min_committed, a new leader is allowed to propose prepare merge without syncing all followers, so there can be a follower whose term is less than leader, hence the pitfall.

Although it is not my question I got your point. BTW if we reset Progress::committed_index to 0 like Progress::matched as https://github.com/tikv/raft-rs/pull/375#discussion_r439275718 suggest, this issue won't be triggered too.


Setup github actions for pkg releases (Fix #12)

Codecov Report

Merging #23 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master       #23   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines          156       186   +30
=========================================
+ Hits           156       186   +30


[style]: Improve the UI Experience of Grabtern Community Section

Description:
The current UI of Grabtern Community section on Home page is very basic, we need to improve it.
I want to add some hover zooming effect and some box shadow to the profile Card to make the UI Experience more enhanced.
Screenshot

Record

[x] I agree to follow this project's Code of Conduct
[x] I'm a GSSoC'23 contributor
[x] I want to work on this issue

Please assign me this issue under GSSoC'23
Thanks

please check the development version, not the production one

@harmeetsingh11 actually it is not needed... could you seperate grabtern community to different page??

Yes, I can do that.

cool


exit with an error if errors (auth errors, etc) are encountered during the promotion step

E.g., this job should have exited with an error: https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/post-k8sio-cip/1187747140168847360 because the promotions failed due to an authorization error.

We just need to keep around a state variable that is globally written-to (use mutexes/locks to protect it across the various goroutines), which stores the state of the promoter, and exit with it.

/remove-lifecycle stale

/remove-lifecycle stale

/close
See #219


Allow buildkite-agent to run a job when JWK is unavailable but failure behaviour is set to warn

Context
Recently, while working on an agent migration plan, I noticed that the agent without JWK configuration is not compatible with an agent with JWK, even when the failure behaviour is set to warn.
Because the agent that started with the JWK configuration adds a job signature to the pipeline's job when it uploads, a requested job can fail if it lands on the agent that doesn't have a JWK key configuration.
Changes
This PR updates a job runner not to reject a signed job when JWK is not configured and verification-failure-behaviour is set to warn. It also adds a new error, SignalReasonUnableToVerifySignature, to the job runner so the misconfiguration can be monitored.
Testing

[x] Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.

[x] Code is formatted (with go fmt ./...)

@CheeseStick thanks for posting this PR, we have just merged some changes which overlap with this code.
The fix for missing JWKS and warnings is a good pickup, we need to impove how we handle this, however the change to SignalReasonUnableToVerifySignature is something I need to dig into further as there are some security concerns.
That said having some specific log messages which are logged in the agent is what we are leaning towards.
Let me pull this down and do some testing.

@CheeseStick thanks for posting this PR, we have just merged some changes which overlap with this code.
The fix for missing JWKS and warnings is a good pickup, we need to impove how we handle this, however the change to SignalReasonUnableToVerifySignature is something I need to dig into further as there are some security concerns.
That said having some specific log messages which are logged in the agent is what we are leaning towards.
Let me pull this down and do some testing.

Thanks!!! 🙇

We have reviewed this soluton, and based on recent exprience, this change also makes it clearer when builds fail because agents aren't configured with a key, overall this is a plus.


Add SSL encryption

It should be able to  encrypt the TCP connection between the terminal emulator and the host using SSL.

Can be achieved by using an SSL VPN internally and disabling accepting all outgoing connections outside the VPN.


Drag from one conversation to another?

Hey Cristian! Love your module!
I have a request; would it be possible to drag from one conversation to another? With all the prepwork on factions and whatnot, I am considering making one large 'cast' conversation, then drag who I need for a particular scenario into a new convo, and run it. For that, I'd need to be able to pull a participant from one list to another, including any registered faction.
Would that at all be possible? It would be an amazing Quality of Life improvement!

I was thinking, being able to tie the faction registration to a token would suffice as well already, as I'd be able to make quick lists on the fly with that. But dragging would still have my preference.

Hey and sorry for the late reply. I haven't had time to look over CHUD the last couple of weeks.
That said, I have started work on the new update for the module and I will be adding support for participant drag and drop. This was an oversight from my part. I have no idea how I missed this when implementing drag and drop considering that you can drag and drop a conversation sheet into another one (and many other item types), but somehow I forgot to add the option to drag a participant from a sheet to another one.
Rest assured, this is a very easy addition and I will be adding this feature to the next version of the module coming sometime in the next couple of weeks.
In the meantime, as a workaround, you can create individual sheets for each actor and drag and drop them that way.

With release 5.4.0 you can now drag and drop participants from one conversation to another. This works on all drag and drop compatible elements (you can drag and drop a participant from the active conversation onto a sheet, etc.).
If there are any issues or questions, feel free to contact me and I will try to solve them as soon as possible.


Indicate Selected Lint Error

[ ] add selection state to error panel
[ ] select lint error on click
[ ] select lint error on linting annotation click

This behavior was implemented as part of the prototype: https://github.com/philippfromme/linting-playground
It cannot easily be implemented in the Camunda Modeler since there is no easy access to a global event bus.

Follow-up to https://github.com/camunda/camunda-modeler/pull/3118

I prototyped this feature in the Camunda Modeler (cf. https://github.com/camunda/camunda-modeler/pull/3142). In order for this feature to work I had to make changes that instead of reusing existing patterns introduce another one (cf. https://github.com/camunda/camunda-modeler/pull/3142/files#diff-b650ad882ccc34a87aa822cb69b9bffc2e3c1513db7dbdff0fc8aaec11fd99a2R1949). After discussing this with @nikku I think we should not implement this feature without improving the way we communicate in the Camunda Modeler first. I will bring this topic up in our next hour of code.

This feature and the current limitations were discussed in today's hour of code. In order to implement the feature without adding more technical debt the communication between components of the Camunda Modeler should be improved (cf. https://miro.com/app/board/uXjVPUDD7Sw=/).

@philippfromme is the envisioned architecture compatible with Web Modeler (i.e. exposing an API to use)?

The changes in the Desktop Modeler are independent of the Web Modeler. Depending on how the Web Modeler was built it might be trivial to implement this feature. There is no shared functionality between the Desktop and Web Modeler as the connection between the bpmn-js instance and the error panel has to be established by the application.


Invoke-WebRequest: Cannot convert 'System.String[]' to the type 'System.Uri'

When querying the Office 365 Management API with PowerShell Core (7.1.3), it fails to correctly parse/process certain hyperlinks. PowerShell 5.1 works.
I believe this is due to the presence of $ in the URL.
A quick in-line fix for my script to work under PS v7 is simply $URL = ($URL -replace "\$","``$")
Steps to reproduce
$headerParams = @{'Authorization'="Bearer TOKENVALUE"}
$URL = "https://manage.office.com/api/v1.0/TENANCYGUID/activity/feed/subscriptions/content?contenttype=Audit.General&endtime=2021-03-16T13:21:48.1859513Z&starttime=2021-03-15T13:21:48.1859513Z&nextpage=20210315183117827000466$20210315132148185000000$emea0010"
Invoke-WebRequest -Method Get -Headers $headerParams -Uri $URL

Expected behavior
It works.

Actual behavior
Cannot convert 'System.String[]' to the type 'System.Uri' required by parameter 'Uri'. Specified method is not supported.

Environment data
PSVersion                      7.1.3
PSEdition                      Core
GitCommitId                    7.1.3
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0ΓÇª}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This is expected. $emea0010 is a valid variable name, and in a double-quoted string PowerShell will attempt to interpolate its value. If the variable doesn't exist, it will be replaced by an empty string (effectively removing the variable name from the string).
The most straightforward solution is to use single quotes when defining the URL, this disables the interpolation and tells PowerShell to treat the string as literal:
$headerParams = @{ Authorization = "Bearer TOKENVALUE" }
$URL = 'https://manage.office.com/api/v1.0/TENANCYGUID/activity/feed/subscriptions/content?contenttype=Audit.General&endtime=2021-03-16T13:21:48.1859513Z&starttime=2021-03-15T13:21:48.1859513Z&nextpage=20210315183117827000466$20210315132148185000000$emea0010'
Invoke-WebRequest -Method Get -Headers $headerParams -Uri $URL
``

As you mentioned, you can also escape the `$` with a backtick to indicate that symbol is literal and does not constitute the start of a variable name.

@vexx32 Hi Joel, thanks for the reply. In providing example code to replicate I expanded the URL (at which point it becomes obvious why that'd happen - doh!). Actually, $URL isn't within single/double quotes but direct to variable from an object property. The complete code would be:
$AggregateListArray = [System.Collections.ArrayList]@()
$URL = "https://manage.office.com/api/v1.0/$tenantGuid/activity/feed/subscriptions/content?contentType=Audit.General&PublisherIdentifier=$Publisher"
$Count = 1
do {
	Write-Verbose -Message "Request $Count"
	$Data = Invoke-WebRequest -Method Get -Headers $headerParams -Uri $URL
	$Aggregates = $Data.content | convertfrom-json
	foreach($Aggregate in $Aggregates)
	{
		$AggregateListArray.Add($Aggregate) | out-null
	}
	$Count++
	$URL = $Data.Headers.NextPageUri
}
until($Null -eq $URL)

The conversion [uri]$uri works well for URI from your repro steps. Are you sure you create the URI correctly? I mean perhaps the issue is not in Invoke-WebRequest but in script code which creates the URI.

I think I've discovered the cause and it wasn't what I originally thought. The issue is header data. Using $Data when it first becomes populated: ($Data.Headers.NextPageUri).gettype()
PowerShell 7:

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String[]                                 System.Array

PowerShell 5:
IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String                                   System.Object

Apologies for the initial mislead. I suspect this is expected/known behaviour with all the changes to the web cmdlets in PS7.

@robinmalik it's unclear to me if you've fixed your problem. Do you still see a bug here?

@joeyaiello Hi Joey. Sorry for that. The problem is worked around but I wouldn't expect a singular string in the headers (in this case, NextPageUri) to become an array object. I don't know what has changed as part of the web cmdlets in PS7, only that I recall seeing many mentions of such in the release notes. Is there anything I can do to help?


Hardcoded object manager persister

DoctrineOrmLoader.php Ln. 135
Returns us hardcoded object of class Fidry\AliceDataFixtures\Bridge\Doctrine\Persister\ObjectManagerPersister
    protected function createPersister(EntityManagerInterface $manager): PersisterInterface
    {
        return new ObjectManagerPersister($manager);
    }

I was trying to decorate service fidry_alice_data_fixtures.persistence.persister.doctrine.object_manager_persister
but gave up :)

It's a protected function so you can extend it.
I'm not too happy with how AliceBundle is right now... The root of the issue is that you can pass a couple of options which configure which services are used or how they are used. Hence we actually need to bypass quite a few services registered by AliceDataFixtures which makes it hard and fragile.
I however could not take a deep look on how this could be solved

@theofidry

It's a protected function so you can extend it.

We cannot extend a final class:

PHP Fatal error:  Class MyDoctrineOrmLoader may not inherit from final class (Hautelook\AliceBundle\Loader\DoctrineOrmLoader)

We are using a modified ObjectManagerPersister and need to use it in the HauteLook/AliceBundle for consistency but are unable to due to the above.

Hm that's definitely a mistake, feel free to do a PR to remove it

DoctrineOrmLoader  is no longer private so this should not be a problem anymore


[docs]: updating mistral and hugging face chat model pages

for both could we add a link in intro where you can see/search through available models?


Correctly handle Node deletion in NodeController

Fixes #11631
It needs a test - I'll add it tomorrow.

ok to test

Added test and rebased PR.

LGTM

@k8s-bot ok to test

LGTM
Sorry for not getting to this sooner


Russian translation added

Hi John, I am very interesting with your style-guide. And I have submitted to you the Russian version of your document. Thanks for your courses, these are very helpful in my work. I am also translating the articles here (codehint.ru) for Russian  audience to help learn AngularJS, and ASP.NET MVC.

looks good, thanks for the efforts.
i noticed this is a bit outdated as i have added more material. would you consider updating this to have the latest text?

Ok. I have started translating 16 days ago. I will compare and add the missing text

Ок, John. It`s done. I found three new sections.

thanks


Store the visitor ID regex pattern as class variable

Such that it only has to be compiled once, little performance gain, but still.
Also changed Matcher.find() to Matcher.matches(), we don't need extra functionality of what "find()" does.

Thanks for your contribution. Merged.


Unable to send funds to different wallet

Unable to send funds from pirate chain lite wallet app on Macbook to wallet on different exchange.
Error message reads: Couldn't understand Response: {"error":"Error: SendResponse { error_code: -26, error_message: "18: bad-txns-joinsplit-requirements-not-met" }"}

Rescan your wallet


Dependencies between engines

We have the following architecture:

A core ember-cli app
Multiple ember-engines, let's name them A and B

Basically engine A has an dependency to engine B due the fact that A requires a model which is declared in B.
Unfortunately this doesn't seem to work (model not found).
Is there any way how engine A can access a model of engine B? Maybe through declaring this dependency?
This is urgent, so we're thankful for any advice.
Thanks for your time and work!

You should extract the model (and any other shared pieces) into an addon. Addons are the current, recommended way of sharing code between multiple contexts (whether those are Engines or Applications).

If you want to query the same models, then you need to share your Ember Data Store service, then you can query the same models from your engine.

I'm at the same company as @phortx . I try to explain the problem in more detail.
The core application mounts two engines as following:
Router.map(function() {
    this.mount('engine-a', { as: 'engine-a', path: 'engine-a' });
    this.mount('engine-b', { as: 'engine-b', path: 'engine-b' });
})

Now engine-a defines the model modelA. modelA.js is placed in addon/models/. engine-b defines the model modelB. modelB.js is placed in addon/models/ as well (of course in the other engine's directory). modelB has the property modelA: DS.belongsTo('model-a').
Now we want to create a record within the engine-b as following:
// controllerB.js in addon/controllers/

let store = this.get('store');
let modelB = store.createRecord('model-b', {...});

What we get is: No model was found for 'model-a'
A simple store.createRecord('model-a', {...}); within engine-b doesn't work either.
After setting up the store dependecy (not the session) as explained by @knownasilya I'm getting this error: No model was found for 'model-b'. So now even the engine-b's own model is unknown.

The models need to be defined in the parent app.

Amazing! Thank you very this very fast response :)

The models need to be defined in the parent app.

Let's assume, that this is not a solution due to the fact that the engines should be replaceable/toggable. In consideration for separation of concerns, it would be very nice (means: necessary) to have the models in their respective engines.
Do you think of any way how we could archive that.

I mean you could put shared models into an in-repo addon, then each engine using that addon would have access, but in that scenario, you'd probably have n + 1 stores (one for parent and one for each engine).

Hmm, was just pondering this myself. I've got a similar goofy edge case where Engine A has a need to take a look at some models that logically belong to Engine B (in this case it's not even a formal belongsTo relation; it's a fuzzy search based on some oddball non-model data), and I'm trying to figure out if it's possible to avoid "hoisting" Engine B's models to the host app just to allow for this one query.
Right now I'm just keeping all the models in the host app, but even in the in-repo case, it feels off to store a fundamental piece of the engine outside of the engine.
[no-easy-solution.jpg]

Right now I'm just keeping all the models in the host app, but even in the in-repo case, it feels off to store a fundamental piece of the engine outside of the engine.

full ack. We implemented the same solution this night and it makes me feel bogus. We're looking forward for a clean resolution of this issue in the future.
Isn't that all just a matter of the resolver?

With models defined in the host application and all engines depending on the host's store service, doesn't it become completely impossible to write acceptance at engine level? Reflecting on loose coupling, aren't we back at square one when the acceptance test for - for example - a 'Save' button in engine B can only be run in the host application?

@jonnedeprez you mock the store in your tests. Then you test the store/api in your parent app. The engine should assume that the store works, and you shouldn't be testing the store in the engine, just functionality that depends on it.

@knownasilya hmm. I get the point, but it would be a pity to say goodbye to Ember Mirage... In non-engines apps I got used to populating the mirage db with a few records and write an acceptance test for a route.

I have t o agree with @knownasilya

You can use ember-cli-mirage in engines. I created an addon for my app, e.g. my-app-test-helpers, including the mirage definitions. You can then hook them up in each of your engines like this: https://github.com/samselikoff/ember-cli-mirage/issues/920
This way, I have acceptance tests in each engine for the application parts of this engine, but the mock server is the same for all the engines (as is the real API server)

Cool, thanks for the links @mydea.

@phortx put your models in addon/models/modelA and export it in app/models/modelA (standard behaviour: ember g model modelA -ir addonname)

I am currently struggling with something similar, and I am not sure that I understand everything correctly.
In my host application, I have the model house. In my in-repo engines I have the model 'room', located in /lib/myEngines/app/models/.
And here is the problem, I want that the room model has a belongsTo relationship.
If I understand correctly, I have to put the the 'room' model in an other engines, so that the application and my engine can both use it?

The room model must be in the application because you are associating it with its model. Then all you need to do is pass the store to any engines that need to use house and room.


Communication between isolates and Hive is not working

I try to save data on the main thread and read the data in an isolate. even when calling .flush() it is not working.
is there any way to force reload data from disk without using cached values?

I try to save data on the main thread and read the data in an isolate. even when calling .flush() it is not working. is there any way to force reload data from disk without using cached values?

As far as I know, the hive database doesn't work on multiple threads. So it doesn't work on an isolated function.


403 Forbidden with Custom Authentication

I am using Identity Server 4 as the custom IDP, it's able to redirect to my login UI and fill in the username / password, however, it redirects back to 403 Forbidden page after successful login.
I tried the different settings for login.nameClaimType in staticwebapp.config.json:

http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name
given_name
email
even get rid of the whole nameClaimType

Here is my staticwebapp.config.json
{
  "navigationFallback": {
    "rewrite": "index.html",
    "exclude": ["/images/*.{png,jpg,gif}", "/css/*"]
  },
  "auth": {
    "identityProviders": {
      "customOpenIdConnectProviders": {
        "idsvr": {
          "registration": {
            "clientIdSettingName": "IS4_ClientId",
            "clientCredential": {
              "clientSecretSettingName": "IS4_ClientSecret"
            },
            "openIdConnectConfiguration": {
              "wellKnownOpenIdConfiguration": "https://2b36-1-145-110-180.au.ngrok.io/.well-known/openid-configuration"
            }
          },
          "login": {
            "nameClaimType": "email",
            "scopes": ["openid", "profile"]
          }
        }
      }
    }
  }
}

I am wondering what /.auth/complete tries to validate? IdToken? Or AccessToken?

It turned out that my id-token did not contain any claims, once the claims are returned back, I managed to login to the SWA.

Hi
what "once the claims are returned back". How did you solve it?
Thanks a lot
Monica

It turned out that my id-token did not contain any claims, once the claims are returned back, I managed to login to the SWA.

did you do additional config to return claims


TypeError: forward() takes 2 positional arguments but 3 were given

Describe the bug:
I compress my model with L1NormPruner and speedup it, then an error occurs. It seems the parameters of torch.rand(3,3,32,15) goes wrong. How can I solve this problem?
This is the code for the pruning part of my project:
config_list = [{
        'sparsity_per_layer': 0.5,
        'op_types': [ 'Conv2d']
    }, {
        'exclude': False,
        'op_names': ['bn1']
    }]

    pruner = L1NormPruner(Encrypted_model, config_list)
    print(Encrypted_model)
    _, masks = pruner.compress()
    pruner._unwrap_model()
    ModelSpeedup(Encrypted_model, torch.rand(3, 3, 32, 15).to(device), masks).speedup_model()

This is the structure and forward of my model：
class polynom_act(nn.Module):

    def __init__(self, alpha=None, beta=None, c=None):
        super(polynom_act, self).__init__()
        self.alpha = nn.Parameter(torch.randn(1), requires_grad=True)
        self.beta = nn.Parameter(torch.randn(1), requires_grad=True)
        self.c = nn.Parameter(torch.randn(1), requires_grad=True)

    def forward(self, x):
        return (self.alpha * (x ** 2) + self.beta * x + self.c)

# my model
class HENet2D(nn.Module):
    def __init__(self, num_classes, filters=128):
        super(HENet2D, self).__init__()
        first_f = filters
        second_f = filters * 2
        third_f = filters * 4
        self.conv1 = nn.Conv2d(3, first_f, kernel_size=(3, 3), stride=1, padding=1)
        nn.init.xavier_uniform(self.conv1.weight)
        self.bn1 = nn.BatchNorm2d(first_f)
        self.bn1.weight.data.fill_(1)
        self.bn1.bias.data.zero_()
        self.act1 = polynom_act()
        self.avg_pool1 = nn.AvgPool2d(kernel_size=2, stride=2)
        self.conv2 = nn.Conv2d(first_f, second_f, kernel_size=3, stride=1, padding=1)
        nn.init.xavier_uniform(self.conv2.weight)
        self.bn2 = nn.BatchNorm2d(second_f)
        self.bn2.weight.data.fill_(1)
        self.bn2.bias.data.zero_()
        self.act2 = polynom_act()
        self.avg_pool2 = nn.AvgPool2d(kernel_size=2, stride=2)
        self.conv3 = nn.Conv2d(second_f, third_f, kernel_size=3, stride=1, padding=1)
        nn.init.xavier_uniform(self.conv3.weight)
        self.bn3 = nn.BatchNorm2d(third_f)
        self.bn3.weight.data.fill_(1)
        self.bn3.bias.data.zero_()
        self.act3 = polynom_act()
        self.avg_pool = nn.AdaptiveAvgPool2d((1, 1))
        self.linear = nn.Linear(third_f, num_classes)

    def forward(self, x):
        out = self.act1(self.bn1(self.conv1(x)))
        out = self.avg_pool1(out)
        out = self.act2(self.bn2(self.conv2(out)))
        out = self.avg_pool2(out)
        out = self.act3(self.bn3(self.conv3(out)))
        out = self.avg_pool(out)
        out = out.view(out.size(0), -1)
        out = self.linear(out)
        return out

This is error:
(pytorch) keyesgheza@KeyesdeMacBook-Pro training-nni % python train.py --filters 64 --type 2d
Saving the model
Epoch: 0, Loss: 2.9199186978132827
Saving the model
Epoch: 1, Loss: 1.1922207526538684
Saving the model
Epoch: 2, Loss: 0.7090284396772799
Test set accuracy: 0.57
Saving the final model to saved_model/2d_model_64_final.torch
HENet2D(
  (conv1): PrunerModuleWrapper(
    (module): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )
  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (act1): polynom_act()
  (avg_pool1): AvgPool2d(kernel_size=2, stride=2, padding=0)
  (conv2): PrunerModuleWrapper(
    (module): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )
  (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (act2): polynom_act()
  (avg_pool2): AvgPool2d(kernel_size=2, stride=2, padding=0)
  (conv3): PrunerModuleWrapper(
    (module): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )
  (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (act3): polynom_act()
  (avg_pool): AdaptiveAvgPool2d(output_size=(1, 1))
  (linear): Linear(in_features=256, out_features=10, bias=True)
)
[2023-04-12 23:44:39] start to speedup the model
[2023-04-12 23:44:39] infer module masks...
[2023-04-12 23:44:39] Update mask for conv1
[2023-04-12 23:44:39] Update mask for bn1
[2023-04-12 23:44:39] Update mask for act1
Traceback (most recent call last):
  File "train.py", line 149, in <module>
    main(filters=args.filters, use2d=True if args.type == '2d' else False, name=f"{args.type}_model_{args.filters}")
  File "train.py", line 139, in main
    ModelSpeedup(Encrypted_model, torch.rand(3, 3, 32, 15).to(device), masks)\
  File "/Users/keyesgheza/信息安全竞赛/training代码/NNI/nni/nni/compression/pytorch/speedup/compressor.py", line 488, in speedup_model
    self.infer_modules_masks()
  File "/Users/keyesgheza/信息安全竞赛/training代码/NNI/nni/nni/compression/pytorch/speedup/compressor.py", line 398, in infer_modules_masks
    self.update_direct_sparsity(curnode)
  File "/Users/keyesgheza/信息安全竞赛/training代码/NNI/nni/nni/compression/pytorch/speedup/compressor.py", line 259, in update_direct_sparsity
    _auto_infer = AutoMaskInference(
  File "/Users/keyesgheza/信息安全竞赛/training代码/NNI/nni/nni/compression/pytorch/speedup/infer_mask.py", line 80, in __init__
    self.output = self.module(*dummy_input)
  File "/Users/keyesgheza/opt/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward() takes 2 positional arguments but 3 were given

Environment:

NNI version: 2.0( build from source)
Training service (local|remote|pai|aml|etc): local
Python version: 3.8.16
PyTorch version: 1.13.1
Cpu or cuda version: cpu

just suggest to use nni master branch to test it.

Hihi, would you please pull the latest NNI branch and use SpeedupV2 instead?
classifier = HENet2D(10, filters=128)
config_list = [{
    'sparsity_per_layer': 0.5,
    'op_types': ['Conv2d']
}, {
    'exclude': False,
    'op_names': ['bn1']
}]

from nni.algorithms.compression.v2.pytorch.pruning import L1NormPruner
from nni.compression.pytorch.speedup.v2 import ModelSpeedup

pruner = L1NormPruner(classifier, config_list)
_, masks = pruner.compress()

pruner._unwrap_model()

ModelSpeedup(classifier, torch.rand(3, 3, 32, 15), masks).speedup_model()

classifier(torch.rand(3, 3, 32, 15))

A gentle reminder is that you might use nn.init.xavier_uniform_(self.conv1.weight) instead of nn.init.xavier_uniform(self.conv1.weight), since the later one is depreciated by PyTorch

just suggest to use nni master branch to test it.

I tried the uni master branch these days, and I success! Thanks for your help!

Hihi, would you please pull the latest NNI branch and use SpeedupV2 instead?
classifier = HENet2D(10, filters=128)
config_list = [{
    'sparsity_per_layer': 0.5,
    'op_types': ['Conv2d']
}, {
    'exclude': False,
    'op_names': ['bn1']
}]

from nni.algorithms.compression.v2.pytorch.pruning import L1NormPruner
from nni.compression.pytorch.speedup.v2 import ModelSpeedup

pruner = L1NormPruner(classifier, config_list)
_, masks = pruner.compress()

pruner._unwrap_model()

ModelSpeedup(classifier, torch.rand(3, 3, 32, 15), masks).speedup_model()

classifier(torch.rand(3, 3, 32, 15))

A gentle reminder is that you might use nn.init.xavier_uniform_(self.conv1.weight) instead of nn.init.xavier_uniform(self.conv1.weight), since the later one is depreciated by PyTorch

Thanks for your help and code. Code run successfully!

I am very glad your problem has been solved, could you close this issue?

I am very glad your problem has been solved, could you close this issue?

yes，sorry for that, I will close it right now!


Stormblessed: Blacklist smoreworld.com

Stormblessed requests the blacklist of the website smoreworld\.com. See the MS search here and the Stack Exchange search here.

smoreworld\.com has been seen in 5 true positives, 0 false positives, and 0 NAAs.

Approved by Makyen in Charcoal HQ