Extraneous warnings: WARNING Retrying ... after connection broken by 'SSLError...'

I installed microscope via pip.
When I tried to run a command on each cilium instance, these warning messages came up:
$ microscope --send-command "cilium endpoint list"
2018-08-29 08:42:41,620 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-6s967
2018-08-29 08:42:41,620 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-28nh7
2018-08-29 08:42:41,620 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-2cr4d
2018-08-29 08:42:41,620 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-7l6xl
2018-08-29 08:42:41,622 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-ck24m
2018-08-29 08:42:41,624 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMB
ER] wrong version number (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-5mjzg
2018-08-29 08:42:41,624 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: DECRYPTION_FAILED_
OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2273)'),)': /api/v1/namespaces/kube-system/pods/cilium-xggcw

Afterwards, microscope successfully executed the command on each Cilium instance.
These warnings seem like they are overly verbose considering that the command actually succeeded.

I haven't looked at the code but this might come directly from the SSL library and doesn't trigger an exception that we catch. It might be some kind of upgrade/downgrade, since they all succeeded on the second try (unless total is the total of retries instead of tries). If it is an upgrade thing that suggests one side of the SSL connection is old. I think more recent protocol versions have more seamless ways to do it.

another thing that comes to mind: Is this even coming from microscope or is it kubectl? microscope runs only execs so it would be odd for it to do any SSL negotiation.


Language detection based on kakoune filetype option

At the moment both kakoune and kak-lsp detect the filetype of a file.
I think it would be better (if it is possible) to let kakoune define which language a given file is in based on the filetype option.

Yes, it's totally possible. We can introduce an extra field in request [meta] section. I just thought that it's redundant information as it can be inferred from buffile.

In most cases we can indeed infer it from the buffer file, however some languages use the same extension.
In short, extension to filetype is not a one-to-one map.

Nice implementation, I have been testing and I ran into 2 (semi) problems.

for different filetypes that use the same server (c,cpp), the entry should be duplicated for these filesystems, maybe there should be a way to match them with an or statement.
The extensions field in the toml file is still required.

Good catch! I think problem could be solved by replacing extensions with filetypes and making entry key arbitrary again.

Now it seems to work, but just realized that it's a breaking change (in case someone uses custom config). I should pack few more before release then =)


Sending data to a room always causes an exception to fire

When using io.livekit.server.RoomService.sendData to send any data to a exisintg room an exception is raised.
The data does however end up being sent, meaning that members of the room receive the data.
The exception being:
kotlin.KotlinNullPointerException: Response from io.livekit.server.RoomService.sendData was null but response body type was declared as non-null
at retrofit2.KotlinExtensions$await$2$2.onResponse(KotlinExtensions.kt:43)
at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:161)
at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)

Steps to reproduce the behavior:

Use io.livekit.server.RoomService.sendData to send any data

Expected behavior
No exception is thrown if the request was a success
Device Info:

Java Version: Java 21
LiveKit Server SDK version: 0.5.10
LiveKit Server: 1.5.3

May be related to https://github.com/square/retrofit/issues/3595

Can you turn on logging in your client and post the logs?

If needed we can try to dive deeper, hope this helps...
We call this via:
  logger.debug { "Sending call info to LiveKit: $data" }
  roomService.sendData(id.toString(), data.encodeToByteArray(), LivekitModels.DataPacket.Kind.RELIABLE)
    .await()

and this is the output
09:40:06.806 [DefaultDispatcher-worker-3] DEBUG at.xxx.telephony.call.Call session= - Sending call info to LiveKit: {"type":"newCallInfo","info":{"id":"c9574502-a945-4b70-9665-144a1d45d7ec","owner":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","participants":[{"id":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","state":"ACTIVE"},{"id":"9c716d48-8916-42d5-a840-2c66789946c0","state":"INVITED"}],"state":"WAITING","startTime":1718790005,"participantEvents":[{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"created","timeStamp":1718790005000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"},{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"joined","timeStamp":1718790006000,"initiator":null},{"userId":"9c716d48-8916-42d5-a840-2c66789946c0","action":"invited","timeStamp":1718790006000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"}]}}
09:40:06.811 [DefaultDispatcher-worker-3] ERROR at.xxx.telephony.call.Call session= - Could not send call info to LiveKit: Response from io.livekit.server.RoomService.sendData was null but response body type was declared as non-null
kotlin.KotlinNullPointerException: Response from io.livekit.server.RoomService.sendData was null but response body type was declared as non-null
	at retrofit2.KotlinExtensions$await$2$2.onResponse(KotlinExtensions.kt:43)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:161)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Turn on logging by passing true to the logging parameter when creating the RoomServiceClient.

Thanks for your understanding. I have taken over from my collegue and I have not so much knowledge on livekit so far....
Here are the logs:
06:01:51.862 [DefaultDispatcher-worker-3] DEBUG at.xxx.telephony.call.Call session= - Sending call info to LiveKit: {"type":"newCallInfo","info":{"id":"458b5e6c-3146-4506-b3ec-a3116a4444ef","owner":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","participants":[{"id":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","state":"ACTIVE"},{"id":"9c716d48-8916-42d5-a840-2c66789946c0","state":"ACCEPTED"}],"state":"STARTED","startTime":1718863307,"participantEvents":[{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"CREATED","timeStamp":1718863307000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"},{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"JOINED","timeStamp":1718863309000,"initiator":null},{"userId":"9c716d48-8916-42d5-a840-2c66789946c0","action":"INVITED","timeStamp":1718863309000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"},{"userId":"9c716d48-8916-42d5-a840-2c66789946c0","action":"ACCEPTED","timeStamp":1718863311000,"initiator":null}]}}
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: --> POST https://lk.exp.xxx.at/twirp/livekit.RoomService/SendData
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/protobuf
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Content-Length: 887
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Authorization: Bearer XXX
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO:
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO:
$458b5e6c-3146-4506-b3ec-a3116a4444ef�{"type":"newCallInfo","info":{"id":"458b5e6c-3146-4506-b3ec-a3116a4444ef","owner":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","participants":[{"id":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","state":"ACTIVE"},{"id":"9c716d48-8916-42d5-a840-2c66789946c0","state":"ACCEPTED"}],"state":"STARTED","startTime":1718863307,"participantEvents":[{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"CREATED","timeStamp":1718863307000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"},{"userId":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc","action":"JOINED","timeStamp":1718863309000,"initiator":null},{"userId":"9c716d48-8916-42d5-a840-2c66789946c0","action":"INVITED","timeStamp":1718863309000,"initiator":"6cfaaa4c-eac7-4858-b968-c468a71cb2cc"},{"userId":"9c716d48-8916-42d5-a840-2c66789946c0","action":"ACCEPTED","timeStamp":1718863311000,"initiator":null}]}}
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: --> END POST (887-byte body)
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: <-- 200 OK https://lk.exp.xxx.at/twirp/livekit.RoomService/SendData (3ms)
06:01:51.870 [DefaultDispatcher-worker-3] ERROR at.xxx.telephony.call.Call session= - Could not send call info to LiveKit: Response from io.livekit.server.RoomService.sendData was null but response body type was declared as non-null
kotlin.KotlinNullPointerException: Response from io.livekit.server.RoomService.sendData was null but response body type was declared as non-null
	at retrofit2.KotlinExtensions$await$2$2.onResponse(KotlinExtensions.kt:43)
	at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:161)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Content-Length: 0
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Content-Type: application/protobuf
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Vary: Origin
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: Date: Thu, 20 Jun 2024 06:01:51 GMT
Jun 20, 2024 6:01:51 AM okhttp3.internal.platform.Platform log
INFO: <-- END HTTP (0-byte body)

We had the implementation as follows:
                roomService.sendData(id.toString(), data.encodeToByteArray(), LivekitModels.DataPacket.Kind.RELIABLE)
                    .await()

After looking at the server-sdk-kotlin and the livekit server I saw that the call returns nothing (https://github.com/livekit/livekit/blob/master/pkg/service/roommanager.go#L778).
Now I removed the .await() from the call and at first glance I don't get the exception anymore.
Is this the proper usage?

Calling await (or execute if running in a non-coroutine world) is required to actually make the call to the server.
Looks like the problem here is using await though. Since this is a void call, await can't return void. Use awaitResponse instead.

Thanks a lot for clearifing the issue for me. Seems it works now with awaitResponse.
Would it be possible to change https://github.com/livekit/server-sdk-kotlin/blob/c893a050650aefe9d867acec768616e539454c17/src/main/kotlin/io/livekit/server/RoomService.kt#L94
from
    fun sendData(@Body request: LivekitRoom.SendDataRequest, @Header("Authorization") authorization: String): Call<Void>
to
    fun sendData(@Body request: LivekitRoom.SendDataRequest, @Header("Authorization") authorization: String): Call<Void?> (return Call<Void?>)
so it would trigger https://github.com/square/retrofit/blob/ed2c6c9f1840ace8b2f49bc6de51a099ac2b5059/retrofit/src/main/java/retrofit2/KotlinExtensions.kt#L65
Thanks


Update Django dependency

See https://docs.djangoproject.com/en/5.0/releases/3.2.24/

Closing because dependabot finally opened it's own PR


Design Signup Screen

Refer to #29 for an overview of all design related tasks

The signup screen has been created successfully
Please review the design

Signup and Login screens for review

Design reviewed and accepted


Update README.md

Translated into Korean.

@Kim-Su-Gyeong Can you please sign CLA. Thanks!

Please direct the PR to https://github.com/keras-team/keras-docs-ko. This is the wrong repo!


Fix CI lint failure by pinning to protobuf-4

Pin protobuf to version 4 to fix "Lint check" CI failure such as
https://github.com/quantumlib/Cirq/actions/runs/9654153610/job/26627844188

Also bump types-protobuf mypy-protobuf for consistency with protobuf-4.x

Related to https://github.com/protocolbuffers/protobuf/commit/26995798757fbfef5cf6648610848e389db1fecf which was released in protobuf-5.27.0.


How to interpret the cumulative return for N forward period in cumulative factor return plots?

In plot_cumulative_returns_by_quantile, cumulative returns for periods more than 1 is computed as:
 if period > 1:
        ret_wide = pd.rolling_apply(
            ret_wide,
            period,
            # compound returns
            lambda ret, period: ((np.nanmean(ret) + 1)**(1. / period)) - 1,
            min_periods=1,
            args=(period,))

    cum_ret = ret_wide.add(1).cumprod()

It converts the N-period return to one-period return by taking geometric mean. But I don't know how to make sense of this "derived" return, because:

Numerically, this "derived" one-period return is not equal to the exact one-period return computed as Price_t/Price_t-1 - 1. Maybe it can be seen as some kind of approximation, but that requires we make some assumption on the N-period return
Intuitively, cumulative return should be interpreted as generated by some continuously rebalanced portfolio. For instance, cumulative return using simple one-period returns can be interpreted as if the factor weighted portfolio is rebalanced every period. This is very intuitive. But if the cumulative return is computed using the "derived" one-period returns, the interpretation is not as intuitive.

It seems to me that the concept of cumulative return only applies to one-period returns. It does not make much sense to derive some kind of one-period returns from N-period returns just to compute the cumulative return. How much value does it provide for assessing the factor?

The "derived" one-period return is what is called the one-period rate of return:  the returns the factor quantile would have had every 1-period if it had grown at a steady rate.
But why does Alphalens computes the 1-period rate of return instead of computing the cumulative returns from a N-periods returns? Alphalens makes an assumption on how the factor should be used, that is how and how often a portfolio should be  rebalanced.
If the factor has the best performance at N periods (N >1) then a naive approach of using that factor would be to rebalance the portfolio every N periods. That would work but also it would make the portfolio performance dependent on the specific starting point. Also, Alphalens statistics are calculated considering the factor performance at each period, but if the factor is traded every N periods (rebalancing every N periods) we cannot expect to achieve the same performance.
So, the cumulative returns computed from N periods returns, are calculated assuming an algorithm that divides the initial capital by N and trade each 1/N sub-capital at subsequent periods and each sub-capital is then rebalanced every N periods. This means the algorithm would trade every period (one of the sub-capitals need to be rebalanced  every period) instead of every N-periods and this would mimic the cumulative returns you can see on Alphalens. Also, this kind of approach would produce lower volatility, better leverage/exposure control than an algorithm that rebalanced every N periods

feel free to reopen this if required

Copy-pasting @HereticSK comment here:

If we are to mimic the cumulative return for the algorithm that holding N equal weighted sub-portfolios and rebalancing them every N period, I wonder if it would make more sense to compute the one-period return as np.nanmean((1 + ret) ** (1. / period) - 1)?
Compared to the original algorithm, it computes geometric mean before arithmetic mean. For a given period, the one-period return for the whole portfolio equals the average of one-period returns for N sub-portfolios. On the other hand, assuming each sub-portfolio grows at a steady rate within each rebalancing cycle, the one-period return for each sub-portfolio is given by the geometric mean of the corresponding N-period return, i.e. (1 + ret)**(1. / period) - 1). Combining these two steps, gives the one-period return of the whole portfolio equals np.nanmean((1 + ret) ** (1. / period) - 1).
Moreover, in the original algorithm, ((np.nanmean(ret) + 1) ** (1. / period)) - 1, the N-period returns for each sub-portfolio is averaged first. Somehow that seems like all these N-period returns are realized at this particular period, while they are actually realized at subsequent N periods. So I think it would make more sense to first compute (1 + ret) ** (1. / period) - 1), as an estimation of the realized return for a sub-portfolio on the given period, and then to average over all the sub-portfolios.

@HereticSK

For a given period, the one-period return for the whole portfolio equals the average of one-period returns for N sub-portfolios

if you average the sub-portfolios one-period returns you end up with a small error that in a cumulative returns might scale up and become a big error.
period = 24

# we have 3 sub-portofolios with the following returns after 8 periods
r1 = 0.05
r2 = 0.03
r3 = -0.05

portfolio_returns = (r1 + r2 + r3) / 3.

one_period_returns = lambda period, n_periods_returns: (n_periods_returns+1.)**(1./period) - 1

# compute one period returns for the full portfolio
oneP_portfolio = one_period_returns(period, portfolio_returns)

# compute one period returns for each sub portfolio
oneP_r1 = one_period_returns(period, r1)
oneP_r2 = one_period_returns(period, r2)
oneP_r3 = one_period_returns(period, r3)
oneP_avg = (oneP_r1 + oneP_r2 + oneP_r3) / 3.

print "full_portfolio_returns", portfolio_returns
print "returns after",period ,"periods of oneP_portfolio", (oneP_portfolio+1)**period
print "returns after",period ,"periods of oneP_avg", (oneP_avg+1)**period`

full_portfolio_returns 0.01
returns after 24 periods of oneP_portfolio 1.01
returns after 24 periods of oneP_avg 1.00909886334

@luca-s

portfolo_returns = (r1 + r2 + r3) / 3.
If I understand correctly, that means r1, r2, r3 are realized at the same time. That is, all these three sub-portfolios are simultaneously bought 24 periods ago and sold at the same period. This is not consistent with "having N algorithms each rebalancing every N periods".

Under an algorithm holding N sub-portfolios and reblancing every N periods, for a given period, only one of the N sub-portfolios is rebalanced, whose N-period return is realized, while others are not realized until they are rebalanced. In other words, for a given period, each of the past N-period return corresponds to a different time span. I don't know if it is appropriate to average over them.
In terms of the code, each element of ret corresponds to a different interval, for instance, ret[1] corresponds to periods from 1 to 3, ret[2] corresponds to periods from 2 to 4......
 lambda ret, period: ((np.nanmean(ret) + 1)**(1. / period)) - 1,

I do agree that there is an error between oneP_portfolio and oneP_avg. But I think only things like oneP_r1, oneP_r2, oneP_r3 can be addable since they can be seen as realized at the same time, while r1, r2, r3 may not be directly addable because they are not simultaneously realized and covers different time interval, not, at least, without some additional assumption.

I understand your doubt about the sub-portofolios not rebalancing the same period, but I believe the current implementation is correct. The rate of returns of the averaged portfolio is computed inside pd.rolling_apply for every single period, so every period the set of N-periods return (each one corresponding to a specific sub-portfolio) over which the mean is computed changes: a new N-periods returns exit and a new one enter in the set. Still, each N-periods returns is considered N times.
I suggest to test the current behaviour in this way: run current implementation of plot_cumulative_returns_by_quantile  and a new one where the sub-portfolios cumulative returns are computed separately and averaged at the end. The two different implementations have to generate the same results.

@HereticSK I thought again about the way the sub-portfolios are averaged together and you might be right, it could be incorrect if they don't rebalance the same period and they are started in subsequent periods like it happens in Alphalens. I will have to double check and in case it is wrong we can simply compute the sub-portfolios cumulative returns separately and average them at the end.
Please, let me know if you come up with a different idea/conclusion.

@luca-s Even though sub-portfolios are averaged on a rolling basis, each N-periods return within the rolling window still corresponds to different time spans. It does not seem to make sense to average returns that do not cover the same time span.

in case it is wrong we can simply compute the sub-portfolios cumulative returns separately and average them at the end.
If I understand correctly, this implementation can only give a cumulative return series of a lower frequency, since the cumulative returns are constructed from N-period returns, which should be a time series of frequency N. If we want to have a cumulative return series of the same frequency as the original N-period return series, we may have to make some assumption in order to recover the one-period returns from the N-period returns. Is that correct?

If we want to have a cumulative return series of the same frequency as the original N-period return series, we may have to make some assumption in order to recover the one-period returns from the N-period returns. Is that correct?

Yes and we do it computing the 1-period rate of return for each N-periods return, so that we end up with N sub-portfolio cumulative returns time series if frequency 1 period. Then we can average them.
This is very similar to the current behaviour except now Alphalens averages the N-periods first and then  computes the 1 period rate of returns.

👍 I see. That's a nice solution.

Suppose you have 2 portfolios with returns 90% and -80% after 60 periods. If you want to simulate the cumulative returns of 1 period you have to compute the rate of returns.
def one_period_returns_series(period, returns):
     daily_returns = (returns+1.)**(1./period) - 1
     return pd.Series([daily_returns] * period)

period = 60

s1 = one_period_returns_series(period, returns=0.90)
s2 = one_period_returns_series(period, returns=-0.80)
s1cr = s1.add(1).cumprod()
s2cr = s2.add(1).cumprod()

s1cr.name = 's1cr'
s2cr.name = 's2cr'
s1cr.plot(legend=True)
s2cr.plot(legend=True)

Then, if you like to plot the full portfolio cumulative returns you have 2 ways (avgcr1 and avgcr3):
# average the 1 period comulative return time series
avgcr1 = (s1cr+s2cr)/2

# compute the final portfolio returns and build the 1 period rate of returns from that
s12_avg = (0.90-0.80) / 2.
avgcr3 = one_period_returns_series(period, s12_avg).add(1).cumprod()

avgcr2 = ((s1+s2)/2).add(1).cumprod() # this is completely wrong

avgcr1.name = 'avgcr1'
avgcr3.name = 'avgcr3'
s1cr.plot(legend=True)
s2cr.plot(legend=True)
avgcr1.plot(legend=True)
avgcr3.plot(legend=True)

They are slightly different, but both final values equal the value of the portfolio at the end of the 60 periods. The intermediate values are approximation so you might prefer one or the other.
Alphalens uses the avgcr3 approach and it applies it at every periods, using a rolling window of N-periods returns as input. Here is where the bug might be: compared to the example above, the N-portfolios are shifted by 1 period from each other so the cumulative returns build from that could be slightly wrong, I need to double check. The other approach, avgcr1,  is fine anyway.


Add details regarding QA automation in CONTRIBUTING.md file.

Fixes #123

@jrfnl or @mjrider Does either of you have time to review my prose?

@Potherca Sorry that it's taken such a long time to get to this. I'd like to get this merged in before I pull the initial test setup PR.
Looking at it, it looks like some of the information is slightly out of date by now (Travis vs GH Actions and such). Have you got time to review and update it yourself ? Or would you like me to propose an update by adding a commit ?

If you've got time, feel free to add. I'll move future branches from my fork to here so they can be more easily edited.

If you've got time, feel free to add. I'll move future branches from my fork to here so they can be more easily edited.

I've created the update, but looks like the checkbox to allow maintainers to edit the PR has not been checked ?
A push to your fork on this PR branch is being rejected.
I'll send you a PR to update this PR 🙃

See: https://github.com/potherca-contrib/phpcodesniffer-composer-installer/pull/2

@jrfnl I've merged your MR so you can merge my MR. 😬


profile_this segfaults on 8 chip tests

Describe the bug
Running a pytest with profile_this led to a segfault while running the test.
To Reproduce
Steps to reproduce the behavior:

git checkout model-team/llama2-tunneling
./tt_metal/tools/profiler/profile_this.py -c  "pytest -svv models/demos/llama2_70b/tests/test_llama_model.py::test_LlamaModel_inference[BFLOAT16-DRAM-32-1-True-8-False-0.999-1]"

Expected behavior
Segfault while profiling.
Screenshots
models/demos/llama2_70b/tests/test_llama_model.py::test_LlamaModel_inference[BFLOAT16-DRAM-32-1-True-8-True-0.999-1]                  Device | INFO     | Opening user mode device driver
CHECKING: [1, 0, 0, 0]
CHECKING: [2, 0, 0, 0]
CHECKING: [3, 0, 0, 0]
CHECKING: [3, 1, 0, 0]
CHECKING: [2, 1, 0, 0]
CHECKING: [1, 1, 0, 0]
CHECKING: [0, 1, 0, 0]
CHECKING: [0, 0, 0, 0]
CHECKING: [1, 1, 0, 0]
CHECKING: [2, 1, 0, 0]
                                                                                [1023/1914]
Current thread 0x00007f5dacf0a740 (most recent call first):
  File "/home/cglagovich/tt-metal/models/demos/llama2_70b/tt/llama_decoder_optimized.py", line 257 in __call__
  File "/home/cglagovich/tt-metal/models/demos/llama2_70b/tt/llama_model_optimized.py", line 272 in forward
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1520 in _call_impl
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1511 in _wrapped_call_impl
  File "/home/cglagovich/tt-metal/models/demos/llama2_70b/tests/test_llama_model.py", line 144 in run_test_LlamaModel_inference
  File "/home/cglagovich/tt-metal/models/demos/llama2_70b/tests/test_llama_model.py", line 279 in test_LlamaModel_inference
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/python.py", line 195 in pytest_pyfunc_call
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/python.py", line 1789 in runtest
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 167 in pytest_runtest_call
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 260 in <lambda>
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 339 in from_call
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 259 in call_runtest_hook
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 220 in call_and_report
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 131 in runtestprotocol
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/runner.py", line 112 in pytest_runtest_protocol
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/main.py", line 324 in _main
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 167 in main
  File "/home/cglagovich/tt-metal/build/python_env/lib/python3.8/site-packages/_pytest/config/__init__.py", line 190 in console_main
  File "/home/cglagovich/tt-metal/build/python_env/bin/pytest", line 8 in <module>
Segmentation fault (core dumped)

Please complete the following environment information:

machine sjc-snva-t3002

Tried to reproduce, I don't see the seg fault
(python_env) ✔ ~/bug_fix/tt-metal [model-team/llama2-tunneling|✔]
18:59 $ ./tt_metal/tools/profiler/profile_this.py -c  "pytest -svv models/demos/llama2_70b/tests/test_llama_model.py::test_LlamaModel_inference[BFLOAT16-DRAM-32-1-True-8-False-0.999-1]"
                 Always | INFO     | Device profiling detected, logs folder location changed to generated/profiler/.logs//ops_device
2024-03-07 18:59:13.250 | INFO     | __main__:main:68 - Profiler build flag is set
                 Always | INFO     | Device profiling detected, logs folder location changed to generated/profiler/.logs//ops_device
=========================================================================================================================================================================================================== test session starts ============================================================================================================================================================================================================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.4.0 -- /home/mmemarian/bug_fix/tt-metal/build/python_env/bin/python
cachedir: .pytest_cache
rootdir: /home/mmemarian/bug_fix/tt-metal, configfile: pytest.ini
plugins: timeout-2.2.0, dash-2.8.1, anyio-4.3.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 1 item

models/demos/llama2_70b/tests/test_llama_model.py::test_LlamaModel_inference[BFLOAT16-DRAM-32-1-True-8-False-0.999-1]                  Device | INFO     | Opening user mode device driver
CHECKING: [1, 0, 0, 0]
CHECKING: [2, 0, 0, 0]
CHECKING: [3, 0, 0, 0]
CHECKING: [3, 1, 0, 0]
CHECKING: [2, 1, 0, 0]
CHECKING: [1, 1, 0, 0]
CHECKING: [0, 1, 0, 0]
CHECKING: [0, 0, 0, 0]
CHECKING: [1, 1, 0, 0]
CHECKING: [2, 1, 0, 0]
CHECKING: [2, 0, 0, 0]
2024-03-07 18:59:19.423 | INFO     | SiliconDriver   - Detected 4 PCI devices : {0, 1, 2, 3}
2024-03-07 18:59:19.560 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 3)
2024-03-07 18:59:19.560 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 4)
2024-03-07 18:59:19.606 | INFO     | SiliconDriver   - Detected 4 PCI devices : {0, 1, 2, 3}
2024-03-07 18:59:19.674 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 2)
2024-03-07 18:59:19.674 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 5)
2024-03-07 18:59:19.722 | INFO     | SiliconDriver   - Detected 4 PCI devices : {0, 1, 2, 3}
2024-03-07 18:59:19.790 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 1)
2024-03-07 18:59:19.791 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 6)
2024-03-07 18:59:19.838 | INFO     | SiliconDriver   - Detected 4 PCI devices : {0, 1, 2, 3}
2024-03-07 18:59:19.907 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 0)
2024-03-07 18:59:19.908 | INFO     | SiliconDriver   - Software version 6.0.0, Ethernet FW version 6.3.0 (Device 7)
                  Metal | INFO     | Initializing device 0
                  Metal | INFO     | AI CLK for device 0 is:   1000 MHz
                  Metal | INFO     | Initializing device 7
                  Metal | INFO     | AI CLK for device 7 is:   1000 MHz
                  Metal | INFO     | Initializing device 1
                  Metal | INFO     | AI CLK for device 1 is:   1000 MHz
                  Metal | INFO     | Initializing device 6
                  Metal | INFO     | AI CLK for device 6 is:   1000 MHz
                  Metal | INFO     | Initializing device 2
                  Metal | INFO     | AI CLK for device 2 is:   1000 MHz
                  Metal | INFO     | Initializing device 5
                  Metal | INFO     | AI CLK for device 5 is:   1000 MHz
                  Metal | INFO     | Initializing device 3
                  Metal | INFO     | AI CLK for device 3 is:   1000 MHz
                  Metal | INFO     | Initializing device 4
                  Metal | INFO     | AI CLK for device 4 is:   1000 MHz
FAILED                  Metal | INFO     | Closing device 0
                  Metal | INFO     | Closing device 1
                  Metal | INFO     | Closing device 2
                  Metal | INFO     | Closing device 3
                  Metal | INFO     | Closing device 4
                  Metal | INFO     | Closing device 5
                  Metal | INFO     | Closing device 6
                  Metal | INFO     | Closing device 7
                     Op | INFO     | Program Cache: disabled and cleared.

However later into the test I do run into
FileNotFoundError: [Errno 2] No such file or directory: '/home/llama-data-repacked-2/llama-2-70b/params.json'

Anything needs to run before the test?

I realize this test is difficult to repro because it depends on model weights which are only on one machine.
Do you know of another 8-chip test which passes when profiling?

Further investigating the bug, it is suggesting segfault on finish call.
gdb backtrace:
#0  0x00007f529e9bb3e5 in tt::tt_metal::HWCommandQueue::finish (this=0x83df090) at tt_metal/impl/dispatch/command_queue.cpp:1196
#1  0x00007f529e9c2d3c in tt::tt_metal::CommandQueue::run_command_impl (this=0x83d9b00, command=...) at tt_metal/impl/dispatch/command_queue.cpp:1771
#2  0x00007f529e9c38df in tt::tt_metal::CommandQueue::run_command (this=0x83d9b00, command=...) at tt_metal/impl/dispatch/command_queue.cpp:1706
#3  0x00007f529e9c5f04 in tt::tt_metal::Finish (cq=...) at tt_metal/impl/dispatch/command_queue.cpp:1521
#4  0x00007f529eecd08b in pybind11::detail::argument_loader<tt::tt_metal::Device*>::call_impl<void, void (*&)(tt::tt_metal::Device*), 0ul, pybind11::detail::void_type>(void (*&)(tt::tt_metal::Device*), std::integer_sequence<unsigned long, 0ul>, pybind11::detail::void_type&&) && (f=<optimized out>, this=<optimized out>) at tt_metal/third_party/pybind11/include/pybind11/detail/../detail/type_caster_base.h:506
#5  pybind11::detail::argument_loader<tt::tt_metal::Device*>::call<void, pybind11::detail::void_type, void (*&)(tt::tt_metal::Device*)>(void (*&)(tt::tt_metal::Device*)) && (f=<optimized out>, this=<optimized out>) at tt_metal/third_party/pybind11/include/pybind11/detail/../cast.h:1417
#6  pybind11::cpp_function::initialize<void (*&)(tt::tt_metal::Device*), void, tt::tt_metal::Device*, pybind11::name, pybind11::scope, pybind11::sibling, char [61]>(void (*&)(tt::tt_metal::Device*), void (*)(tt::tt_metal::Device*), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [61])::{lambda(pybind11::detail::function_call&)#3}::operator()(pybind11::detail::function_call&) const (
    call=..., this=<optimized out>) at tt_metal/third_party/pybind11/include/pybind11/pybind11.h:248
#7  pybind11::cpp_function::initialize<void (*&)(tt::tt_metal::Device*), void, tt::tt_metal::Device*, pybind11::name, pybind11::scope, pybind11::sibling, char [61]>(void (*&)(tt::tt_metal::Device*), void (*)(tt::tt_metal::Device*), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [61])::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) ()
    at tt_metal/third_party/pybind11/include/pybind11/pybind11.h:223
#8  0x00007f529eed41a6 in pybind11::cpp_function::dispatcher (self=0x7f529eb6b760 <tt::Logger::get()::logger>, args_in=0x7f528f467b80, kwargs_in=0xb) at tt_metal/third_party/pybind11/include/pybind11/pybind11.h:939
#9  0x00000000005d5499 in PyCFunction_Call ()
#10 0x000000000054d9f0 in _PyEval_EvalFrameDefault ()
#11 0x000000000054552a in _PyEval_EvalCodeWithName ()
#12 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#13 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#14 0x000000000054552a in _PyEval_EvalCodeWithName ()
#15 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#16 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#17 0x000000000054552a in _PyEval_EvalCodeWithName ()
#18 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#19 0x00000000005d4c12 in PyObject_Call ()
#20 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#21 0x000000000054552a in _PyEval_EvalCodeWithName ()
#22 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#23 0x00000000005d4c12 in PyObject_Call ()
#24 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#25 0x000000000054552a in _PyEval_EvalCodeWithName ()
#26 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#27 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#28 0x00000000004e1a5e in ?? ()
#29 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#30 0x000000000054552a in _PyEval_EvalCodeWithName ()
#31 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#32 0x000000000057a4af in ?? ()
#33 0x00000000005d6066 in _PyObject_MakeTpCall ()
#34 0x000000000054d44a in _PyEval_EvalFrameDefault ()
#35 0x00000000005d5846 in _PyFunction_Vectorcall ()
#36 0x0000000000547447 in _PyEval_EvalFrameDefault ()
#37 0x00000000005d5846 in _PyFunction_Vectorcall ()
#38 0x00000000005d4c12 in PyObject_Call ()
#39 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#40 0x000000000054552a in _PyEval_EvalCodeWithName ()
#41 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#42 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#43 0x00000000004e1a5e in ?? ()
#44 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#45 0x000000000054552a in _PyEval_EvalCodeWithName ()
#46 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#47 0x000000000057a4af in ?? ()
#48 0x00000000005d4e0e in PyObject_Call ()
#49 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#50 0x000000000054552a in _PyEval_EvalCodeWithName ()
#51 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#52 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#53 0x000000000054552a in _PyEval_EvalCodeWithName ()
#54 0x00000000004e1bd0 in ?? ()
#55 0x00000000005483b6 in _PyEval_EvalFrameDefault ()
#56 0x000000000054552a in _PyEval_EvalCodeWithName ()
#57 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#58 0x00000000005d4c12 in PyObject_Call ()
#59 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#60 0x000000000054552a in _PyEval_EvalCodeWithName ()
#61 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#62 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#63 0x000000000054552a in _PyEval_EvalCodeWithName ()
#64 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#65 0x00000000005483b6 in _PyEval_EvalFrameDefault ()
#66 0x00000000005d5846 in _PyFunction_Vectorcall ()
#67 0x00000000005d4c12 in PyObject_Call ()
#68 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#69 0x000000000054552a in _PyEval_EvalCodeWithName ()
#70 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#71 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#72 0x00000000004e1a5e in ?? ()
#73 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#74 0x000000000054552a in _PyEval_EvalCodeWithName ()
#75 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#76 0x000000000057a4af in ?? ()
#77 0x00000000005d6066 in _PyObject_MakeTpCall ()
#78 0x000000000054d44a in _PyEval_EvalFrameDefault ()
#79 0x00000000005d5846 in _PyFunction_Vectorcall ()
#80 0x00000000005d4c12 in PyObject_Call ()
#81 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#82 0x000000000054552a in _PyEval_EvalCodeWithName ()
#83 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#84 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#85 0x00000000004e1a5e in ?? ()
#86 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#87 0x000000000054552a in _PyEval_EvalCodeWithName ()
#88 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#89 0x000000000057a4af in ?? ()
#90 0x00000000005d6066 in _PyObject_MakeTpCall ()
#91 0x000000000054d44a in _PyEval_EvalFrameDefault ()
#92 0x00000000005d5846 in _PyFunction_Vectorcall ()
#93 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#94 0x00000000005d5846 in _PyFunction_Vectorcall ()
#95 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#96 0x00000000005d5846 in _PyFunction_Vectorcall ()
#97 0x00000000005d4c12 in PyObject_Call ()
#98 0x0000000000548a66 in _PyEval_EvalFrameDefault ()
#99 0x000000000054552a in _PyEval_EvalCodeWithName ()
#100 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#101 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#102 0x00000000004e1a5e in ?? ()
#103 0x000000000054c8a9 in _PyEval_EvalFrameDefault ()
#104 0x000000000054552a in _PyEval_EvalCodeWithName ()
#105 0x00000000005d5a23 in _PyFunction_Vectorcall ()
#106 0x000000000057a4af in ?? ()
#107 0x00000000005d6066 in _PyObject_MakeTpCall ()
#108 0x000000000054d44a in _PyEval_EvalFrameDefault ()
#109 0x00000000006af02c in ?? ()
#110 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#111 0x00000000005d5846 in _PyFunction_Vectorcall ()
#112 0x0000000000547265 in _PyEval_EvalFrameDefault ()
#113 0x000000000054552a in _PyEval_EvalCodeWithName ()
#114 0x0000000000684327 in PyEval_EvalCode ()
#115 0x0000000000673a41 in ?? ()
#116 0x0000000000673abb in ?? ()
#117 0x0000000000673b61 in ?? ()
#118 0x00000000006747e7 in PyRun_SimpleFileExFlags ()
#119 0x00000000006b4072 in Py_RunMain ()
#120 0x00000000006b43fd in Py_BytesMain ()
#121 0x00007f530c9bb083 in __libc_start_main (main=0x4c4510 <main>, argc=4, argv=0x7ffcf3a3b548, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffcf3a3b538) at ../csu/libc-start.c:308
#122 0x00000000005da67e in _start ()

Is there a minimal way to reproduce this without a t3000 machine?

Minimal repro: Update test_llama_model.py with a diff like this (line numbers may be off)
     else:
-        ckpt_dir = model_config["DEFAULT_CKPT_DIR"]
-        tokenizer_path = model_config["DEFAULT_TOKENIZER_PATH"]
+        # ckpt_dir = model_config["DEFAULT_CKPT_DIR"]
+        ckpt_dir = "Path/to/params.json"
+        # tokenizer_path = model_config["DEFAULT_TOKENIZER_PATH"]
+        tokenizer_path = "Path/to/tokenizer.model"
         cache_path = model_config["DEFAULT_CACHE_PATH"]

     max_seq_len = 4096
@@ -80,7 +85,7 @@ def run_test_LlamaModel_inference(
         max_seq_len=max_seq_len,
         max_batch_size=batch,
         n_layers=n_layers,
-        skip_model_load=False,
+        skip_model_load=True,
     ).model

Update these paths with the files uploaded to this comment. Remove the .txt from the tokenizer file.
params.json
tokenizer.model.txt

Some additional data:

Running with emulate=True does not segfault
Segfaults in all_gather op
Commenting out profiler code in all_gather_op.cpp also allows it to run and pass
Cannot yet reproduce segfault in a unit test

It appears that we are segfaulting on GetKernel, actually. I find it interesting that Mo observed this on Finish.
std::unordered_map<unsigned short, std::shared_ptr<tt::tt_metal::Kernel>, std::hash<unsigned --Type <RET> for more, q to quit, c to continue without paging--
short>, std::equal_to<unsigned short>, std::allocator<std::pair<unsigned short const, std::shared_ptr<tt::tt_metal::Kernel> > > >::find (__x=<synthetic pointer>: <optimized out>,
    this=0x83ecdf0) at /usr/include/c++/9/bits/unordered_map.h:926
#6  tt::tt_metal::Program::get_kernel (this=this@entry=0x7d72ef0, kernel_id=kernel_id@entry=0)
    at tt_metal/impl/program/program.cpp:126
#7  0x00007fe1e3ec250b in tt::tt_metal::detail::GetKernel (kernel_id=0, program=...)
    at ./tt_metal/detail/program.hpp:23
#8  tt::tt_metal::op_profiler::append_kernel_info (program=...)
    at ./tt_metal/tools/profiler/op_profiler.hpp:560
#9  tt::tt_metal::all_gather (input_tensors=std::vector of length 8, capacity 8 = {...},

This does not appear to be a command queue error, and I've only observed this on the hacky profile block within the all-gather op. I am unassigning myself from this issue.

@mo-tenstorrent For me it would be alright to wait until your DRAM profiling changes are in main. Thanks for the support. I'll downgrade the priority of this issue since you have a workaround

Resolved in main


gccgo: bogus warning "control reaches end of non-void function"

What version of Go are you using (go version)?
go version go1.9 gccgo (GCC) 8.0.0 20171114 (experimental) linux/amd64
Does this issue reproduce with the latest release?
Yes.  This is gccgo in gcc trunk.
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xry111/go"
GORACE=""
GOROOT="/opt/go"
GOTOOLDIR="/usr/lib64/gcc/x86_64-pc-linux-gnu/8.0.0"
GCCGO="/usr/bin/gccgo-8.0"
CC="gcc-8.0"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build037067240=/tmp/go-build -gno-record-gcc-switches -funwind-tables"
CXX="g++-8.0"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?
package main

import (
	"fmt"
	"os"
)

func foo() int {
	switch os.Args[0] {
	case "cat":
		return 0
	case "ls":
		return 1
	}
	panic("wtf")
}

func main() {
	fmt.Println(foo())
}

And build it with LC_ALL=C go-8.0 build.
What did you expect to see?
No warnings and no errors.
What did you see instead?
# hw
./main.go: In function 'main.foo':
go1: warning: control reaches end of non-void function [-Wreturn-type]

I think panic() should have something like __attribute__((noreturn)) in GNU C, to suppress the warning.

By the way, this bogus warning spamed my terminal compiling Go standard library.

With -gccgoflags "-O" the warning gone.

Thanks, I have seen this too.  Seems to be an annoying change in the GCC middle-end.  Since the Go frontend provides its own "missing return" warning anyhow, the simplest approach may be for the frontend to disable the middle-end warning.


Add: chinese trans in send and confirm page

This change is

Builds ready [682f8d2]builds: chrome, firefox, opera
bundle viz: background, ui, inpage, contentscript, ui-libs, bg-libs, phishing-detect
dep viz: background
all artifacts


Migrate ecmarkup to TypeScript

Migrates ecmarkup to TypeScript. I've set up the tsconfig.json to use some fairly strict settings (noImplicitAny, strictNullChecks, etc.), though they can be relaxed if necessary.

Let's put @internal on its own line (when used before class methods, at least)

Why change from 2-space indent to 4-space indent? It makes the diffs useless and breaks blame.

The TypeScript compiler emits 4 space indentation. There's no setting to override it at present.

I'm considering adding lib to .gitignore, since it now only contains build outputs.

I see, yes, build output should not be included in the repo... it will just confuse people like it confused me.

Agreed, let's not check in the built output.

Looks good with the latest commits! Thanks @rbuckton!


update fork

update

attempting to update fork

can't seem to switch it so that my fork is update. Not sure what to do here


Crash when the message doesn't have an imageUrl

I completed both tutorials for the FriendlyChat (Android/iOS swift), when you add pictures in your swift application, if you are using the same database the android app will crash trying to load the messages because in the swift app "imageUrl" is stored as "imageURL" and "photoUrl" is "photoURL" so the app crashes at
if (imageUrl.startsWith("gs://")) { (line 229 I believe)
because imageUrl is actually stored as imageURL.

@madmax330 thanks for doing both codelabs and for finding this issue!  As I said on the iOS repo, we'll get this fixed.

No problem! 👍🏽


feat: bot now notifies the respective channel in github

Fixes #7

/bounty $100

Contgraions!!! Claim your bounty https://bounty.100xdevs.com


Update diffx-scalatest to 0.4.4

Updates com.softwaremill.diffx:diffx-scalatest from 0.4.1 to 0.4.4.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.softwaremill.diffx", artifactId = "diffx-scalatest" } ]

labels: test-library-update, semver-patch

Superseded by #264.


Revert "Setting containerd shim to Pod cgroup"

Reverts kubernetes-incubator/cri-containerd#193
@abhinandanpb I have to revert this. This is breaking our node e2e test. We need to debug it.
https://travis-ci.org/kubernetes-incubator/cri-containerd/builds/270726614
I verified this myself, and kept getting failed to load cgroup error, seems like that pod cgroup is not ready when creating pod sandbox.

With the PR reverted, I could get better node e2e test result:
Summarizing 2 Failures:

[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume as non-root [Conformance] [sig-storage]
/home/lantaol/workspace/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:2165

[Fail] [k8s.io] AppArmor [Feature:AppArmor] when running with AppArmor [It] should enforce a permissive profile
/home/lantaol/workspace/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e_node/apparmor_test.go:75

Ran 182 of 248 Specs in 1211.774 seconds
FAIL! -- 180 Passed | 2 Failed | 0 Pending | 66 Skipped

Ginkgo ran 1 suite in 20m12.080807766s
Test Suite Failed
make[1]: Leaving directory '/home/lantaol/workspace/src/k8s.io/kubernetes'


esp_http_client fails due to base_poll_read() select() usage (IDFGH-13821)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.3.1
Espressif SoC revision.
ESP32-S3
Operating System used.
Linux
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
None
Development Kit.
Custom board
Power Supply used.
External 3.3V
What is the expected behavior?
Reliably download files with esp_http_client() even when the last IP packets arrive quickly.
What is the actual behavior?
esp_http_client_read() return an error instead of the last data.
Steps to reproduce.

Use a slow network uplink. I'm using PPP over an uart.
Try to use esp_http_client_open() and esp_http_client_read() to download files.
Sometimes, depending on the file size and buffer size to esp_http_client_read() the call fails.

Debug Logs.
D (15:15:01.618) HTTP_CLIENT: Begin connect to: http://<IP_ADDRESS>:41351
D (15:15:01.633) HTTP_CLIENT: Write header[3]: GET /foo.bin HTTP/1.1
User-Agent: ESP32 HTTP Client/1.0
Host: <IP_ADDRESS>:41351
Content-Length: 0

D (15:15:01.735) HTTP_CLIENT: on_message_begin
D (15:15:01.736) HTTP_CLIENT: HEADER=Server:SimpleHTTP/0.6 Python/3.12.6
D (15:15:01.737) HTTP_CLIENT: HEADER=Date:Fri, 27 Sep 2024 15:15:01 GMT
D (15:15:01.737) HTTP_CLIENT: HEADER=Content-type:application/octet-stream
D (15:15:01.738) HTTP_CLIENT: HEADER=Content-Length:129792
D (15:15:01.739) HTTP_CLIENT: HEADER=Last-Modified:Fri, 27 Sep 2024 14:56:50 GMT
D (15:15:01.739) HTTP_CLIENT: http_on_headers_complete, status=200, offset=204, nread=204
D (15:15:01.740) HTTP_CLIENT: http_on_body 308
D (15:15:01.740) HTTP_CLIENT: Body received in fetch header state, 0x3fcebb9c, 308
D (15:15:01.741) HTTP_CLIENT: content_length = 129792
D (15:15:01.742) HTTP_CLIENT: Data processed 308 != Data specified in content length 129792
D (15:15:01.742) HTTP_CLIENT: Data processed 308 != Data specified in content length 129792
D (15:15:01.743) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:01.744) HTTP_CLIENT: need_read=76, byte_to_read=76, rlen=76, ridx=52
D (15:15:01.745) HTTP_CLIENT: http_on_body 76
D (15:15:01.894) HTTP_CLIENT: Data processed 384 != Data specified in content length 129792
D (15:15:01.895) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:01.896) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:01.897) HTTP_CLIENT: http_on_body 128
D (15:15:01.898) HTTP_CLIENT: Data processed 512 != Data specified in content length 129792
D (15:15:01.898) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:01.900) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:01.900) HTTP_CLIENT: http_on_body 128
D (15:15:01.904) HTTP_CLIENT: Data processed 640 != Data specified in content length 129792
D (15:15:01.905) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:01.906) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:01.907) HTTP_CLIENT: http_on_body 128

.... snip ...

D (15:15:20.333) HTTP_CLIENT: Data processed 124416 != Data specified in content length 129792
D (15:15:20.333) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:20.335) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:20.335) HTTP_CLIENT: http_on_body 128
D (15:15:20.340) HTTP_CLIENT: Data processed 124544 != Data specified in content length 129792
D (15:15:20.341) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:20.342) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:20.343) HTTP_CLIENT: http_on_body 128
D (15:15:20.344) HTTP_CLIENT: Data processed 124672 != Data specified in content length 129792
D (15:15:20.344) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
D (15:15:20.346) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=128, ridx=0
D (15:15:20.346) HTTP_CLIENT: http_on_body 128
D (15:15:20.446) HTTP_CLIENT: Data processed 124800 != Data specified in content length 129792
D (15:15:20.447) HTTP_CLIENT: is_data_remain=1, is_chunked=0, content_length=129792
E (15:15:20.449) transport_base: poll_read select error 128, errno = Socket is not connected, fd = 57
D (15:15:20.450) HTTP_CLIENT: need_read=128, byte_to_read=128, rlen=-2, ridx=0
E (15:15:20.451) HTTP_CLIENT: transport_read: error - 57347 | ERROR
D (15:15:20.451) HTTP_CLIENT: Data processed 124800 != Data specified in content length 129792
D (15:15:20.452) HTTP_CLIENT: Data processed 124800 != Data specified in content length 129792

More Information.
The select() call might return 2. That means two bits are set in total
in the sets used. As there is only one fd it means that the fd both has
data and an error.
This happens when the last data is received and the peer has notified
that it won't send any more data.
If this happens one should use the information in the readset first
before handling any exceptions.
This caused the http_client to fail at the last packet depending on the
timing.
Change this so that it only considers the errorset if it's only one bit
set in the returned sets.
--- a/components/tcp_transport/transport_ssl.c
+++ b/components/tcp_transport/transport_ssl.c
@@ -169,7 +169,7 @@ static int base_poll_read(esp_transport_handle_t t, int timeout_ms)
         return remain;
     }
     ret = select(ssl->sockfd + 1, &readset, NULL, &errset, esp_transport_utils_ms_to_timeval(timeout_ms, &timeout));
-    if (ret > 0 && FD_ISSET(ssl->sockfd, &errset)) {
+    if (ret == 1 && FD_ISSET(ssl->sockfd, &errset)) {
         int sock_errno = 0;
         uint32_t optlen = sizeof(sock_errno);
         getsockopt(ssl->sockfd, SOL_SOCKET, SO_ERROR, &sock_errno, &optlen);

@david-cermak
Any chance to take a look? Thanks.

@AxelLin the fix is already under review.

@AxelLin the fix is already under review. Should be on master soon.

@euripedesrocha
So you confirm the bug is real and you have a fix for it?
Could you share the patch and test code?
(The fix is not yet in master, it's so slow)

Hello @AxelLin ,
We've already submitted a merge request for this fix, and we're currently testing it across additional scenarios to ensure stability. It should be merged soon.

@nileshkale123 @euripedesrocha
I'm testing on v5.2 branch.
I just realize this is a big issue because now my device cannot update
firmware via PPP due to this issue.
Can you fix this ASAP?

Hello @AxelLin ,
We've already submitted a merge request for this fix, and we're currently testing it across additional scenarios to ensure stability. It should be merged soon.

Your reply gives people the expect that this will be fix soon.
However, having wait for 3 weeks, still cannot get the fix.

@nileshkale123  This issue was reported by @gonzzor, not me.
@gonzzor Could you verify if the patch from https://github.com/espressif/esp-idf/issues/14673#issuecomment-2499797262 works?

@AxelLin I'm using my suggested change in transport_ssl.c in production and it for sure improved things, haven't seen any more problems with it.
The second part mentioned in https://github.com/espressif/esp-idf/issues/14673#issuecomment-2499797262 seem to have been merged already in https://github.com/espressif/esp-idf/issues/14724 so I don't see any problems with that change.
In addition, when testing this I'm using the python http.server module to simply serve files locally to my board. I didn't have problems all the time and it seem to be timing related as the size of file impacted things. I could see the issue when connected via uart and pppd running on Linux.

@nileshkale123
So you said "already submitted a merge request for this fix, and we're currently testing it" on Nov 5 https://github.com/espressif/esp-idf/issues/14673#issuecomment-2456330842.
But in the follow up you said "I have not been able to reproduce the issue" on Nov 26 https://github.com/espressif/esp-idf/issues/14673#issuecomment-2499797262.
And if look back the ealier response on Oct 16 https://github.com/espressif/esp-idf/issues/14673#issuecomment-2415839814, it does not make sense at all.
This is a very confusing situation.

Hello @AxelLin ,
Apologies for any confusion caused.
We have raised a merge request based on the user’s suggestion, as it seems logically correct and was agreed upon by the team. However, we suspect that this change might introduce another scenario that could potentially fail. To address this, we’ve been actively investigating further.
Additionally, the specific issue reported by the user is not being directly reproduced on our end. We are exploring more robust methods to accurately replicate the scenario. We kindly request some additional time to thoroughly analyze and resolve this. Thank you for your patience.

@nileshkale123
I thoutht you will add a test case for this, but the commit does not include a test case.
The fix is committed in master now, so could you confirm you do reproduce this issue?
BTW, the stable branches still need fix.


merge from arlac77/template-github-action,arlac77/template-kronos-component

README.md

docs(README): update from template

Codecov Report

Merging #999 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #999   +/-   ##
=======================================
  Coverage   92.24%   92.24%
=======================================
  Files          10       10
  Lines        1483     1483
  Branches      169      169
=======================================
  Hits         1368     1368
  Misses        115      115

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4a2678...8fe182f. Read the comment docs.


Update links.json

Cs added

👷 Deploy Preview for sensational-longma-f9b5ec processing.

Name
Link

🔨 Latest commit
4f4a4abfe34b4ac19ed9b5cd5995ddb56afc4f49

🔍 Latest deploy log
https://app.netlify.com/sites/sensational-longma-f9b5ec/deploys/64f34228651aae0008ba749c


support for Radix primitives

Pagoda is working on a design system to offer a set of polished components for BOS developers to leverage. Radix can be seen here as the missing set of primitives for modern web development. The unstyled primitives will be provided by the VM much like other primitives are provided by HTML(buttons, checkboxes, etc). The design system will then be built from primitives within Discovery, supporting our ideals of forking and remixing. The primitives will also be available to the ecosystem at large to build their own design systems.
https://www.radix-ui.com/docs/primitives/overview/introduction
VM Changes
forwardRef
We now create Widget with React.forwardRef to support Radix's asChild prop, which is necessary to use BOS components for things like <Accordion.Trigger>.
Pass down non-BOS props on Widget
Previously, props could only be passed in an attribute named props e.g. <Widget ... props={{text:'hello'}} />. Since Radix handles setting some props, and didn't set them in that way, they were not making it through to the rendered element. Now, any props placed directly on Widget which are not meant for the VM are placed onto the rendered element.
e.g. The follow element will have props.label available in component-land code for the widget, and onClick will be applied to the root element.
<Widget src='michaelpeter.near/widget/SomeComponent' props={{label:'foo'}} onClick={() => console.log(bar)}

Test Components
http://localhost:3000/#/calebjacob.near/widget/RadixFunctionTest
http://localhost:3000/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixFunctionTest&tab=source
http://localhost:3000/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixAlertDialogTest&tab=source
http://localhost:3000/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixAlertDialogTest2&tab=source
http://localhost:3000/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixAlertDialogTest3&tab=source

Radix UI Docs: https://www.radix-ui.com/docs/primitives/overview/introduction
Here's a screenshot of applying custom (and rough) styles to the Radix DropdownMenu:

And here's a super crude Radix Accordion example:

You can view the source for each, but the previews won't work due to the unpublished VM changes required to support Radix:
https://alpha.near.org/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixDropdownMenuTest&tab=source
https://alpha.near.org/#/adminalpha.near/widget/ComponentDetailsPage?src=calebjacob.near/widget/RadixAccordionTest&tab=source

@evgenykuzyakov Do you have any concerns with this implementation? Right now this only exposes three Radix primitives but Caleb is testing more and the goal would be to expose all of the primitives that are functional when included in this way
https://www.radix-ui.com/docs/primitives/overview/introduction

@mpeterdev @evgenykuzyakov All Radix primitives have been added and tested within the VM. The only primitive we're excluding for now is Form due to it relying on the <form> tag (we can come back to this later): https://www.radix-ui.com/docs/primitives/components/form
We're ready for a final review on the PR now.

Thanks for all the great feedback @evgenykuzyakov. I pushed a commit that removes the forwarded props from vm.state: https://github.com/NearSocial/VM/commit/f2a5dcccf29e3cc6e629c71e4077bee5f430578a
Any other concerns I can address?


feat: add maxRetry and patch option

ref: https://github.com/volcano-sh/volcano/pull/3846

It doesn't look good. How about just adding a maxRetry field?

type Flow struct {
	// +kubebuilder:validation:MinLength=1
	// +required
	Name string `json:"name"`
	// +optional
	DependsOn *DependsOn `json:"dependsOn,omitempty"`
	// +optional
	MaxRetry *MaxRetry `json:"maxRetry,omitempty"`
}

It doesn't look good. How about just adding a maxRetry field?

Thanks @hwdef @googs1025
Got it. Just add maxRetry field done.


[TEST] Test generating content

Description
No upstreams

Upstream PR
The following PR deals with the following generated content for the following versions:

3.10:
3.11:
3.12:
devel:

/generate


Update repositories.txt

@ArduinoBot added the missing tag (oops)

@ArduinoBot update

Hello @abread. The automated system here only operates on open pull requests. Since your submission was already accepted and the pull request merged, the bot won't pay any more attention to this pull request.
But this human is happy to provide assistance. Was there something you needed from the bot?


Issue with k8s.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-14/

This is a Bug Report

Problem:
Proposed Solution:
Page to Update:
https://kubernetes.io/...

Hi @knbz
What's not right with that page?
/triage needs-information

@knbz Can you please provide bit more detail about your issue?

@knbz Any update on this else we can close this issue?

could be a miss click.
please re-open if needed.
/close


Etcd/move schemas to file

Moves the schemas (utility types) into a separate file so zod schemas for runtime type-checking can be added without cluttering up the client code.
(This is a draft PR, stacked on top of a parent that currently needs to be merged first! The diff will be smaller once the parent is merged. The GitHub UI doesn't seem to let me change the diffbase, as the parent branch lives on my fork. Before merging, first merge the parent and rebase this PR on main.)
Parent:

https://github.com/turbopuffer/turbopuffer-typescript/pull/9

Child:

https://github.com/turbopuffer/turbopuffer-typescript/pull/11

The whole stack (in order):

https://github.com/turbopuffer/turbopuffer-typescript/pull/8
https://github.com/turbopuffer/turbopuffer-typescript/pull/9
https://github.com/turbopuffer/turbopuffer-typescript/pull/10 (this)
https://github.com/turbopuffer/turbopuffer-typescript/pull/11

Gonna close this one due to drift from main - the changes are easy enough to redo from scratch off of main and put in a new branch.


YARN-11698. Store pending log aggregators in the NM State Store

Description of PR
Stores containers pending log aggregation in the NodeManager state store so logs can still be aggregated for complete containers after a Node Manager restart. This undoes and replaces https://issues.apache.org/jira/browse/YARN-4771 with a finer-grained approach that doesn't involve storing containers indefinitely until the application finishes.
The original approach has several issues, some of which were mentioned in the JIRA but decided it was ok:

Long running applications can lead to a large number of containers being stored indefinitely in the state store as well as in memory on the Node Manager
On restarts, the Node Manager has to do a lot of work fully recovering all of these complete containers just so they can be registered for log aggregation again
This leads to large heartbeat messages to the Resource Manager that can DoS or OOM it
This ignores the fact that users may not have log aggregation enabled or may have rolling log aggregation enabled, meaning containers are stored even after there is no need to worry about aggregating the logs in the future

Instead, this adds a new state store entry for containers pending log aggregation. This solves all the above issues, while still providing the same guarantees about logs being aggregated after a Node Manager restart.
How was this patch tested?
New UTs added
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 19s

trunk passed

+1 :green_heart:
compile
1m 25s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
1m 22s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 40s

trunk passed

+1 :green_heart:
mvnsite
0m 45s

trunk passed

+1 :green_heart:
javadoc
0m 44s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m 27s

trunk passed

+1 :green_heart:
shadedclient
34m 30s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 32s

the patch passed

+1 :green_heart:
compile
1m 21s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
1m 21s

the patch passed

+1 :green_heart:
compile
1m 18s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
1m 18s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
0m 30s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 13 new + 330 unchanged - 0 fixed = 343 total (was 330)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

-1 :x:
javadoc
0m 34s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt
hadoop-yarn-server-nodemanager in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.

-1 :x:
javadoc
0m 32s
/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt
hadoop-yarn-server-nodemanager in the patch failed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.

-1 :x:
spotbugs
1m 31s
/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.html
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
shadedclient
34m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
24m 44s

hadoop-yarn-server-nodemanager in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

157m 15s

Reason
Tests

SpotBugs
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager

Switch statement found in org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogHandlerEvent) where one case falls through to the next case  At LogAggregationService.java:where one case falls through to the next case  At LogAggregationService.java:[lines 404-406]

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6845

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 5b38c0b5239a 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 6aee095b556a555a180963e1395291403aea3807

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/1/testReport/

Max. process+thread count
699 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 39s

trunk passed

+1 :green_heart:
compile
1m 27s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
1m 24s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 39s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
0m 50s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m 26s

trunk passed

+1 :green_heart:
shadedclient
34m 15s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 32s

the patch passed

+1 :green_heart:
compile
1m 20s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
1m 20s

the patch passed

+1 :green_heart:
compile
1m 15s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
1m 15s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 30s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 14 new + 330 unchanged - 0 fixed = 344 total (was 330)

+1 :green_heart:
mvnsite
0m 35s

the patch passed

+1 :green_heart:
javadoc
0m 31s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 30s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m 27s

the patch passed

+1 :green_heart:
shadedclient
33m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
24m 41s

hadoop-yarn-server-nodemanager in the patch passed.

+1 :green_heart:
asflicense
0m 46s

The patch does not generate ASF License warnings.

156m 35s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6845

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux eae4a5f6e38c 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c497647f78e7ad8d901b2e1ab86aa2e41732d1ea

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/2/testReport/

Max. process+thread count
551 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 48s

trunk passed

+1 :green_heart:
compile
1m 28s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
1m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
0m 43s

trunk passed

+1 :green_heart:
mvnsite
0m 47s

trunk passed

+1 :green_heart:
javadoc
0m 50s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 42s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
1m 30s

trunk passed

+1 :green_heart:
shadedclient
35m 12s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 34s

the patch passed

+1 :green_heart:
compile
1m 18s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
1m 18s

the patch passed

+1 :green_heart:
compile
1m 16s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
1m 16s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 29s
/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 14 new + 330 unchanged - 0 fixed = 344 total (was 330)

+1 :green_heart:
mvnsite
0m 36s

the patch passed

+1 :green_heart:
javadoc
0m 34s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
0m 34s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
spotbugs
1m 25s

the patch passed

+1 :green_heart:
shadedclient
35m 11s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
24m 48s

hadoop-yarn-server-nodemanager in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

155m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6845

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux bdd3f2516208 5.15.0-122-generic #132-Ubuntu SMP Thu Aug 29 13:45:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 79e7bd949a24b9d4342a8d8f7bd165f96c91aa50

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/3/testReport/

Max. process+thread count
554 (vs. ulimit of 5500)

modules
C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6845/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


[MacOS] Photino's never exits upon closing window

When running Photino on MacOS, upon closing the window (or even the whole application using CMD+Q), the underlying process is not killed.
To reproduce:

Clone the samples repository (https://github.com/tryphotino/photino.Samples)
Run any project, e.g. Photino.HelloPhotino.NET
Start the application
Kill the foreground application
Observe the debugger is still active and the code is still in the window.WaitForClose(); loop
Kill the running dotnet process to actually kill the application

Good catch! Not easy to find when VS Code is running that spawns dotnet processes of its own.

@gerwim This has been rectified in the latest release v2.5.2 (https://github.com/tryphotino/photino.NET/releases/tag/v2.5.2)


added prerequisites for documentation styling conventions

Please consider this PR in any way yet.

Please do not consider this Draft in any way yet.

I converted this PR to a draft, I hope this was okay.

@Grashalmbeisser is this still active? Can you please resolve conflicts, and give an update?

Will close this for now due inactivity.


Index out of range when importing glTF

There seems to be a bug in the importer, that results in the following error when importing project_polly.gltf and other glTF files.
Traceback (most recent call last):
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\__init__.py", line 522, in execute
    return self.import_gltf2(context)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\__init__.py", line 542, in import_gltf2
    BlenderGlTF.create(self.gltf_importer)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_gltf.py", line 27, in create
    BlenderScene.create(gltf, scene_idx)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_scene.py", line 49, in create
    BlenderNode.create(gltf, node_idx, None) # None => No parent
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_node.py", line 100, in create
    BlenderNode.create(gltf, child_idx, node_idx)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_node.py", line 65, in create
    BlenderMesh.set_mesh(gltf, gltf.data.meshes[pynode.mesh], mesh, obj)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_mesh.py", line 54, in set_mesh
    offset = BlenderPrimitive.set_normals(gltf, prim, mesh, offset)
  File "C:\Users\Moritz Becher\Developer\UX3D\glTF-Blender-IO\addons\io_scene_gltf2\blender\imp\gltf2_blender_primitive.py", line 80, in set_normals
    vert0 = mesh.vertices[calc_norm_vertices[0]].co
IndexError: list index out of range

location: <unknown location>:-1

location: <unknown location>:-1

Fixed in 5d3c15819a3b6aae6bcd23ad5fc8a16623228f2d


Default to /bin/bash on exec

This enables my laziness so:
devcontainer exec monorepo

Drops you into a bash terminal. Instead of
devcontainer exec monorepo /bin/bash

I like the idea!
That might make the meaning of the positional arguments ambiguous since you can do `devcontainer exec dc-name my-app arg1 arg2 arg3" (at least I think it supports that!)
So with:
devcontainer exec myapp some-arg
vs
devcontainer exec dc-name myapp
I'm not clear how you'd do that reliably.
Maybe a breaking change is needed:
devcontainer exec --container=dc-name my-app arg1 ...
where --container is optional if you're in a folder for a dev container?

Thanks for this @lawrencegripper  - really nice addition :-)


Remove the unused method. & Modify doc.

Please answer these questions before submitting pull request

Why submit this pull request?
[x] Bug fix
[ ] New feature provided
[ ] Improve performance

Remove the unused method. & Modify doc.

Coverage increased (+0.005%) to 16.549% when pulling af93e2371b81f8044d23d0e38d76d3ef864361da on wayilau:fix_bug1 into b76839fe4ca7d02b3c09c7a66968c3d1e4323d71 on apache:master.


Widget-ID is always 0

The widget ID delivered through on-create is always 0 in my case. My ReCaptcha key is valid, I'm using it on a plain HTML page as well.
<div         class="g-recaptcha"         vc-recaptcha         key="'[mykey]'"         ng-model="formData.gRecaptchaResponse"         on-create="setWidgetId(widgetId)"         on-success="setResponse(response)"         on-expire="cbExpiration()">     </div>
$scope.setWidgetId = function (widgetId) {     console.log("widgetId: " + widgetId);     $scope.recaptchaWidgetId = widgetId; };
console output:
widgetId: 0
It was working like that for a while but with current AngularJS 1.6.3 and angular-recaptcha it doesn't work anymore for me when used like shown above.

0 is valid, that means it is working. If you got null, undefined, an object, a string, or literally anything that isn't a number/integer, then that would be a problem. widget ids comes from recaptcha itself, and are integers, starting at 0. For each instance you create, the number will go up.

Ah, ok. Then there must be another problem with my implementation, because I get the following error when trying to invoke reload() on the widget with id 0:
Error: Invalid ReCAPTCHA client id: 0

I'm running into this as well with version 4.1.0. The widget it is being set to zero, but a subsequently attempt to reset it fails, claiming that the identifier is invalid. If I change my dependency on angular-recaptcha back to 4.0.3 then the problem goes away and the reCaptcha gets reset correctly. Is there an API change between these versions that I need to take account of?

Yep, not working for me either. It's always 0

I also run into this problem, using build:2017-04-24 (it wouldn´t hurt to have the version number in the js-file).

Going back to build:2015-06-22 solves the problem. Then I can use  and in set_captcha_widget, I save the id to be used when I want to vcRecaptchaService.reload(id)

+1
Error: Invalid ReCAPTCHA client id: 0

Do we need to pass the client ID?
I see similar issue: Error: Invalid ReCAPTCHA client id: 0
I have only 1 reCaptcha in the view, so if I just reload without a client ID: vcRecaptchaService.reload();
reCaptch is reset and forces the user to answer the captch again.
Any issues if I don't pass a client ID?


Extension host terminated unexpectedly

From @cortesi on March 31, 2017 4:56

VSCode Version: 1.10.2
OS Version: macOS 10.12.3

I see this every few minutes while editing a Rust project, and vscode is essentially unusable while it is happening. I'm including a screenshot of output in the developer tools console. There doesn't seem to be anything relevant in the Output pane.

Copied from original issue: Microsoft/vscode#23721

From @aeschli on March 31, 2017 6:23
Can you check if it is related to an extension? Run VSCode without any extension with code --disable-extensions.

From @aeschli on March 31, 2017 13:41
It's likely a extension running havoc. If you can single it out and file an issue against it that would be good.

From @cortesi on March 31, 2017 9:24
I haven't been able to reproduce this without extensions enabled, but then also I don't (can't) use the editor extensively without extensions. From the location of the error, I'm assuming that this is an issue with managing spawned child processes within extensions.

Could you list the extensions you have installed with code --list-extensions?

From @cortesi on April 4, 2017 23:36
I've since seen a traceback that definitely links the issue to Rust autocompletion using racer in the vscode-rust addon (attached). I haven't had time to dig into this deeply, but it seems to me that the external process crashed, taking down the extension host with it. I think there may be two bugs here - one in racer causing a crash, and one in the vscode extension host.

The extension runs on both ends and there is currently no way for us to protect against misbehaving extensions on the extension host's side. Moving to the extension, thanks.

@cortesi,
You should be able to debug it using the page.

I am closing this issue.
If anyone experiences it reopen it please.


Gather all element metadata before upgrading scripts.

Element metadata includes stuff like attributes, host attributes,
new computed functions. We need this metadata to build out the Polymer()
declaration for the element.
Before we were upgrading scripts more or less as we saw them, and not
keeping track of all element metadata in one place. Now, we first gather
all of the metadata, so that we'll be able to upgrade any Polymer() call
for any element in any

LGTM


Enabling ForceReconnect with StackExchange.Redis

In the https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-best-practices-connection#using-forcereconnect-with-stackexchangeredis section, use of the ForceReconnect pattern is recommended when using StackExchange.Redis.
No mention is made of the integrated implementation in the Microsoft.Extensions.Caching.StackExchangeRedis package, even though this is certainly the most common use of the StackExchange.Redis package.
ForceReconnect can be activated via the "Microsoft.AspNetCore.Caching.StackExchangeRedis.UseForceReconnect" switch.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d0c4a176-bed8-4879-8aa7-5113610710ec
Version Independent ID: 39180666-8539-4884-8a55-4a2bc6425bf3
Content: Best practices for connection resilience - Azure Cache for Redis
Content Source: articles/azure-cache-for-redis/cache-best-practices-connection.md
Service: cache
GitHub Login: @flang-msft
Microsoft Alias: franlanglois

@FPierrel
Thanks for your feedback! We will investigate and update as appropriate.

@philon-msft Could you please share your inputs on the above?

Agreed, we could add a paragraph to the "Using ForceReconnect with StackExchange.Redis" section like:
"If you're using Microsoft.Extensions.Caching.StackExchangeRedis in an ASP.NET application rather than using StackExchange.Redis directly, you can simply set the property Microsoft.AspNetCore.Caching.StackExchangeRedis.UseForceReconnect = true.

I have added this to the documentation backlog. I'll keep this open until we commit to working on it in a given sprint. Thanks, @FPierrel, @philon-msft

Thanks for the doc suggestion! I've noted it for a future update of this article where we might be able to include it. According to our usual policy, I'll now proceed to close this issue. If you have further information or would like to continue the discussion, please tag me and or reopen the issue. Thank you!
#please-close

Hello,
I think there is a small mistake here, the property  Microsoft.AspNetCore.Caching.StackExchangeRedis.UseForceReconnect does not exist when using the Microsoft.Extensions.Caching.StackExchangeRedis nuget. Instead what must be done is activating the AppContext switch called Microsoft.AspNetCore.Caching.StackExchangeRedis.UseForceReconnect  like this:
AppContext.SetSwitch("Microsoft.AspNetCore.Caching.StackExchangeRedis.UseForceReconnect", true);

@canu1337 What version of the Microsoft.Extensions.Caching.StackExchangeRedis package are you using?

I'm using 8.0.1

I can confirm that the guidance in the docs is misleading. https://source.dot.net/#Microsoft.Extensions.Caching.StackExchangeRedis/RedisCache.cs has a note at the top saying it needs the AppContext switch