[low-power] add csl feature for Thread 1.2

This PR implements the CSL feature in Thread 1.2.
It is based on enhanced keep alive(#4325) and enhanced frame pending(#4334).
In more detail, this PR does these things:

Add macro definitions for low power to control the compiling of source code
Add data and methods for running CSL in Mac and SubMac. This mainly includes setting CSL parameters, starting/stopping CSL, and the timer handling process.
Add otPlatTimeGetAPI and implementation.
Add CSL transmission implementation. CSL transmission is a new kind of transmission, the related definition and implementaion for the whole transmitting process is added.
Add calling of start/stop CSL in certain cases.
Implement CSL synchronization maintainence. If a CSL cordinator didn't get a frame containing CSL IE for CSLTimeout, the CSL receiver is regarded as de-synchronized.
Add Cli interface for using CSL.
Implement enhanced Ack with IE. The original code can only generate auto ack for Imm-Ack. As CSL requires CSL IE included in enhanced ack. This PR implements it.
Add basic functional test for CSL transmission. More tests corresponding to test plan would be added later.

The above items are corresponded to the commits in this PR.

Parts of this PR have been moved into #4884. Will rebase this PR after it gets merged.

Still work in progress. There are some problems after merging with master. I'm fixing it.

Just merged with master. Please review when you have a chance. @edmont @bukepo @brianljt @librasungirl

@Irving-cl, here is my suggestion/thoughts on CSL implementation in SubMac:

SubMac basically abstracts radio operation and provides some main methods to next layer (Mac) to control the radio: Send() to send a frame and put radio in tx mode, Receive() to put radio in rx mode, and Sleep() to put it to sleep.

We want to add CSL sampling logic to periodically do sampling, i.e. put radio in rx model for some time, then switch it to sleep for some time.

I think only after the next layer calls SubMac::Sleep() (i.e. it is done with radio, and SubMac's mState is kStateSleep) the CSL sampling code should control the radio (i.e., put it to sleep and rx periodically).
If the next layer calls SubMac::Receive() (to put radio in receive mode on a specific channel) or SubMac::Send() (to send a frame) we should honor the request and don't let the CSL sampling logic to control/change radio mode.

I have been thinking about the CSL code. Here are my thoughts/suggestions on how to architect the design.
This is all related to Tx of CSL frames.
SubMac layer

We use the SubMac::Send(aFrame) for sending frame.  The frame would have parameters/fields to indicate whether it is CSL tx or regular
If regular, use existing flow/code.
If it is CSL, the frame includes info to provide the requested tx time.

If the request time is already passed or if it is too far in future (using some fixed threshold), we fail and return an error (e.g. invalid arg or another error) - This situation should not happen often, and when it happens the next layer(s) need to try again later.
If the requested time is within a given window, we use the existing CSMA timer to schedule the tx time and when timer fires we issue BeginTransmit() (this should be almost the same as the existing code handling CSMA logic).
One thing to consider (for future) is what mode to keep the radio in, during this wait time:

We can keep it in rx mode - the downside is that radio may start receiving a frame during wait time and then when we want to send the CSL frame, it may fail (or be too late); or we can keep radio in sleep mode to not let it receive.
I suggest for now, we keep it in rx mode, later we can experiment and see its performance impact to see if using sleep mode (or any other solution) makes sense.

Mac layer
First let me add a quick overview of how Mac works:

Mac manages/schedules "operations" which basically control how (for what purpose) the radio is being used at any point of time.
We follow a multi (typically 3) call/callback model:

Next layer (a user of Mac) calls a Mac method to request/schedule an operation (e.g.,  ActiveScan(), EnergyScan(), RequestDirectFrameTransmission(), etc.)
Mac itself manages the state and the current operation (only one operation is active at a time) - It determines which requested operation is higher propriety and the order in which they should be performed.
When Mac is ready to start a requested operation, it (can) use a callback to ask for additional input.

For example, for a direct tx, when Mac is ready,  it would use a callback to ask the associated next layer to prepare the frame (HandleFrameRequest() callback to MeshForwarder)

When the operation finishes, Mac uses another callback to signal it to the next layer along with the status (error) and any other info.

CSL Tx

I suggest adding/changing the method RequestCslFrameTransmission() to include parameters to indicate a requested CSL tx time (a tx time along with a guard interval before it).
Mac code would remember the pending requested CSL tx and its time window. When the time comes it would start the CSL tx operation and uses a callback to the next layer to ask it to prepare the frame.
Since Mac is already scheduling and prioritizing different requested operation, it is well-suited to manage the CSL tx timing as well.

Some notes about RequestCslFrameTransmission()

We should allow next layer to call this method again to change the requested CSL tx time (this is unlike other Mac request methods which would fail when a previous request is not yet finished).

Basically next layer can call this method again to cancel a previous request and request a new CSL tx time window.

We need a mechanism to inform next layer from Mac if/when for whatever reason the CSL tx time window expires and MAC layer can not start CSL tx operation in time (e.g., a previous operation takes longer or ...).

I think best approach is to add a new callback to the next layer to indicate this situation (in this case, next layer can request again - maybe with longer guard window)

Regarding the next layer (which interacts with Mac for CSL tx) we would need a module to keep track of the next CSL tx time (going through all sleepy children and queued messages, etc). I think some this code in this PR is present in DataPollHandler and IndirectSendeer. I suggest adding this code in a new module (e.g. CslSender) specific for this. This class would then interact with Mac (request the CSL tx and implement the callbacks from Mac).

@Irving-cl, here is my suggestion/thoughts on CSL implementation in SubMac:

SubMac basically abstracts radio operation and provides some main methods to next layer (Mac) to control the radio: Send() to send a frame and put radio in tx mode, Receive() to put radio in rx mode, and Sleep() to put it to sleep.

We want to add CSL sampling logic to periodically do sampling, i.e. put radio in rx model for some time, then switch it to sleep for some time.

I think only after the next layer calls SubMac::Sleep() (i.e. it is done with radio, and SubMac's mState is kStateSleep) the CSL sampling code should control the radio (i.e., put it to sleep and rx periodically).
If the next layer calls SubMac::Receive() (to put radio in receive mode on a specific channel) or SubMac::Send() (to send a frame) we should honor the request and don't let the CSL sampling logic to control/change radio mode.

@abtink
Thanks for the kind suggestions. I also agree to honor the request from next layer.
I'm thinking such a case for SSED: When the next layer is idle for some time and the control is passed to CSL currently, and it is doing sampling(receiving). However at this time, the next layer request a Tx. How should we handle this case? It would probably break the receiving, should we allow this happen?

I made a graph to show the design for the interactions between Mac and SubMac for SSED:

Basically, Mac layer is always prioritized. Mac layer could interrupt the state from CSL but CSL couldn't break Mac layer's operation. And when there's no more other operations for Mac, Mac would do updateIdleMode and change SubMac's state(Currently it calls sleep). I would like to, for example, add a new method Idle in SubMac, and when CSL is started, it would change the state which CSL expects(Handle control over CSL).
Thoughts? @abtink @bukepo @brianljt

@abtink
Thanks for the kind suggestions. I also agree to honor the request from next layer.
I'm thinking such a case for SSED: When the next layer is idle for some time and the control is passed to CSL currently, and it is doing sampling(receiving). However at this time, the next layer request a Tx. How should we handle this case? It would probably break the receiving, should we allow this happen?

I would suggest checking the spec to see if it provides any details on this.
But from what I recall and expect, I think tx request on SSED should be handled when requested. The process is "CSL idle sampling" so should be used when in idle mode.
Thinking practically about this, the main point of CSL sampling on a SSED is to check if parent has any pending frame that wants to send to the child. If SSED child has a frame to send to its parent, the frame itself acts as a data poll and parent would indicate in its ack frame whether there is any pending frame for the child.

I'm thinking such a case for SSED: When the next layer is idle for some time and the control is passed to CSL currently, and it is doing sampling(receiving). However at this time, the next layer request a Tx. How should we handle this case? It would probably break the receiving, should we allow this happen?

I don't think the spec provides any guidance on whether TX can occur during a CSL window and is left up to the implementation.
At first glance, it may seem better to avoid TX during a CSL sample window because that is the only time a parent can transmit to an SSED most of the time. On one hand, the Enhanced Frame Pending feature should inform the SSED that the parent has packets queued, allowing the SSED to quickly issue a MAC Data Request immediately after. On the other hand, if the SSED had waited to TX until after the CSL sample window, it could have saved the cost of sending the MAC Data Request. Also, avoiding TX during a CSL sample window helps avoid potential collisions between the SSED and parent.
In the end, I don't know if it matters all that much. SSEDs should not transmit all that often (otherwise they are not really low power devices). Avoiding TX during CSL sample window potentially saves power (avoids additional MAC Data Request and potential collisions), but can also increase latency.
I would say we do what is easier/quicker to implement and optimize later if needed when we have more data and experience.
@Irving-cl @abtink @brianljt

@jwhui
That makes sense. I think I can file a JIRA for a discussion on this.
If it's not mandotory that a Tx has to wait until CSL sample finishes, I prefer that we can first prioritize a Tx. In the normal cases, it's always like, a parent sends something to a SSED and SSED responds. And SSED is unlikely to initiate many transmissions. Thoughts? @abtink @bukepo @brianljt

That makes sense. I think I can file a JIRA for a discussion on this.

I don't think we need to file a JIRA yet. Whatever an implementation chooses does not harm interoperability. Furthermore, I think the discussion would require collecting some data and experience.

If it's not mandotory that a Tx has to wait until CSL sample finishes, I prefer that we can first prioritize a Tx. In the normal cases, it's always like, a parent sends something to a SSED and SSED responds. And SSED is unlikely to initiate many transmissions.

It's not obvious to me which occurs more. I expect many use cases where the SSED still needs to periodically initiate communication primarily to assert liveness and connectivity. Parent will only initiate communication for issues commands or new configuration, which could be relatively rare compare to a liveness "heartbeat" coming from the SSED.

It's not obvious to me which occurs more. I expect many use cases where the SSED still needs to periodically initiate communication primarily to assert liveness and connectivity. Parent will only initiate communication for issues commands or new configuration, which could be relatively rare compare to a liveness "heartbeat" coming from the SSED.

Got it. So I think we can first do what is easier and then optimize it according to its performance.

Mac code would remember the pending requested CSL tx and its time window. When the time comes it would start the CSL tx operation and uses a callback to the next layer to ask it to prepare the frame.

Does it mean to move the timer from next layer(currently data_poll_handler) to Mac? This timer first arranges a time a little bit ahead the CSL tx time, then request CSL tx from Mac and finally SubMac schedules the tx at an accurate time. Do you mean to put this timer(which arranges a time a little bit ahead) into Mac layer?(you mentioned that a tx time would also be passed to Mac via requestCslFrameTransmission) In this way, the next layer isn't responsible for scheduling a time to request CSL Tx from Mac. Instead it directly call requestCslFrameTransmission with a parameter(expected Csl Tx time) and Mac layer would make sure to schedule it a little bit ahead of the expected time and then pass to SubMac. However in this way, when the next layer requests a CSL tx and Mac starts a operation and this CSL tx may need some time to complete since it hasn't been scheduled before. You mentioned that:

We should allow next layer to call this method again to change the requested CSL tx time

What about other requests? If during this CSL tx process, there are some other requests. Should they be allowed?
Or you mean to keep the timer in next layer while letting Mac layer use another timer.(In this way we have 3 timers in total in this process: next layer, Mac, SubMac).
@abtink

I suggest adding/changing the method RequestCslFrameTransmission() to include parameters to indicate a requested CSL tx time (a tx time along with a guard interval before it).
Mac code would remember the pending requested CSL tx and its time window. When the time comes it would start the CSL tx operation and uses a callback to the next layer to ask it to prepare the frame.
Since Mac is already scheduling and prioritizing different requested operations, it is well-suited to manage the CSL tx timing as well.

Let me repeat the process with a little more details. Let's refer to the 'next layer' which would call RequestCslFrameTransmission as  CslSender here.

First, CslSender calls RequestCslFrameTransmission with a expected Tx time as parameter.
Mac layer gets this request, sets PendingCsl as true and records this expected Tx time.
And Mac layer decides which operations are prioritized. When the recorded Tx time is nearby(we can set a threshold value for it), then it would prioritize the CSL Tx opeartions. Otherwise, a CSL tx operation wouldn't be started.
Then, when recorded Tx time comes, Mac layer starts the CSL Tx operation and pass it to SubMac with the expected Tx time.
Next, SubMac check this Tx time

If it already passed or it's too far from now, it returns an error to Mac. Mac should tell this to CslSender and CslSender would make another request to Mac.
If the Tx time is good, SubMac would use the existing csma timer to schedule the CSL Tx. During this period, before do the actual Tx, we let SubMac receive(or sleep, may change later).

SubMac finally completes the Tx. No matter it succeeded or not, it would call TransmitDone to Mac. Mac passes it to CslSender. If it fails, CslSender would call RequestCslFrameTransmission again.

Am I right about this process? It overall looks good to me. I just have some questions:

It seems that CslSender doesn't need a timer anymore. It only needs to calculate the Tx time. It can directly call RequestCslFrameTransmission with that Tx time, right?
Mac neither uses a timer to start the Csl Tx operation, it just checks time in PerformNextOpeartion. If the time is close to the recorded tx time(within a range), it starts the Csl Tx operation. Will it brings the result that, when PerformNextOperation is called, the time is too late since we don't know when PerformNextOperation would be called. And Mac can only return an error to CslSender.  I'm not sure about this.
When Submac returns an error because the expected Tx time provided by Mac is too early or too late. Shall we let Mac handle it silently or pass it back to CslSender and let CslSender call RequestCslFrameTransmission to try it again?
In Submac, when it tries to do the CSL Tx, it would use the csma timer to schedule the Tx accurately. In a normal transmission, if software csma is enabled, it would generate a random value as backoff and wait such time by using timer. When it comes to a csl tx, the timer should wait to the accurate tx time. If software csma is enabled, how should we handle the time for tx?

@abtink

I would say we do what is easier/quicker to implement and optimize later if needed when we have more data and experience.

👍  on this.
I think allowing CSL sample only during Mac idle time would be easier to implement (at first).

From https://github.com/openthread/openthread/pull/4557#issuecomment-656992298

Basically, Mac layer is always prioritized. Mac layer could interrupt the state from CSL but CSL couldn't break Mac layer's operation. And when there's no more other operations for Mac, Mac would do updateIdleMode and change SubMac's state(Currently it calls sleep). I would like to, for example, add a new method Idle in SubMac, and when CSL is started, it would change the state which CSL expects(Handle control over CSL).

Here is my suggestion on the API/methods in Mac/SubMac to implement CSL listening/sampling behavior:

I suggest to remove StartCsl and StopCsl() from SubMac class.
Instead, we add a new method CslListen() which asks SubMac to enter CSL idle listening mode/state (while in this state it performs  channel sample periodically).

This aligns well with other SubMac methods Send(), Receive(), Sleep() which request it to enter a specific state/mode (tx, rx, or sleep).

The Mac would track whether the device needs to do CSL idle listening or not.
We change Mac::UpdateIdleMode() method (which is called when Mac is idle, i.e., in kOperationIdle) to call the proper SubMac method to put it proper state, i.e. in rx (e.g., when mRxOnWhenIdle is true), or sleep, or CSL listening state.

@Irving-cl , on the steps in https://github.com/openthread/openthread/pull/4557#issuecomment-657174991, yes it all sounds good/correct to me

It seems that CslSender doesn't need a timer anymore. It only needs to calculate the Tx time. It can directly call RequestCslFrameTransmission with that Tx time, right?

Yes. Exactly. No need for a timer in CslSender.

Mac neither uses a timer to start the Csl Tx operation, it just checks time in PerformNextOpeartion. If the time is close to the recorded tx time(within a range), it starts the Csl Tx operation. Will it brings the result that, when PerformNextOperation is called, the time is too late since we don't know when PerformNextOperation would be called. And Mac can only return an error to CslSender.  I'm not sure about this.

Mac would need to use a timer to be notified when it gets close to requested CSL tx time. We already have mTimer in Mac (used during different operations) which I think can be re-used for this purpose.
The high-level changes (algorithm) to implement this in Mac class:

In PerformNextOperation() (before checking any other pending operations), we check if there is a pending CSL tx request and that the current time is close to the requested tx time (i.e., within a guard interval of requested tx time). If so,  we then start the kOperationTxDataCsl operation.
We change UpdateIdleMode() after the calls to mSubMac,  we check if there is a pending CSL tx request and if so, we schedule mTimer to fire close to the tx time (i.e., guard interval before the tx time) using Timer::FireAt().
We change Mac::HandleTimer() to handle this case, i.e., if current operation (mOperation) is kOperationIdle (and  if there is a pending CSL tx request (though this check can be optimized out)) we call PerformNextOperation() (which would then start the CSL tx operation).
RequestCslFrameTransmission() would be simple. It just saves/updates the requested tx time and calls StartOperation()

The existing code/machinery would take care of the reset - we would post mOperationTask (if not in middle of an operation already) which would in turn call PerformNextOperation(). If near the requested tx time, it would handled then, otherwise we get to UpdateIdleMode() which schedules the timer for the requested CSL tx time.

When Submac returns an error because the expected Tx time provided by Mac is too early or too late. Shall we let Mac handle it silently or pass it back to CslSender and let CslSender call RequestCslFrameTransmission to try it again?

Yes. I think we need to pass the failure back to the CslSender for it to request for next tx time. CslSender is the class that is aware of all children and their CSL timings (Mac should not be aware of this), so CslSender should be the one to schedule a retry.

In Submac, when it tries to do the CSL Tx, it would use the csma timer to schedule the Tx accurately. In a normal transmission, if software csma is enabled, it would generate a random value as backoff and wait such time by using timer. When it comes to a csl tx, the timer should wait to the accurate tx time. If software csma is enabled, how should we handle the time for tx?

I am not sure if I understand this question. If you are asking how to schedule the timer for an exact time, we have Timer::FireAt()` method (which schedule the timer to fire at a given time).

I am not sure if I understand this question. If you are asking how to schedule the timer for an exact time, we have Timer::FireAt()` method (which schedule the timer to fire at a given time).

What I mean is, if sw csma is enabled, shall we take the backoff time into consideration when we schedules a CSL tx using the timer?

I suggest to remove StartCsl and StopCsl() from SubMac class.

Instead, we add a new method CslListen() which asks SubMac to enter CSL idle listening mode/state (while in this state it performs  channel sample periodically).

This aligns well with other SubMac methods Send(), Receive(), Sleep() which request it to enter a specific state/mode (tx, rx, or sleep).

The Mac would track whether the device needs to do CSL idle listening or not.

We change Mac::UpdateIdleMode() method (which is called when Mac is idle, i.e., in kOperationIdle) to call the proper SubMac method to put it proper state, i.e. in rx (e.g., when mRxOnWhenIdle is true), or sleep, or CSL listening state.

@abtink
Looks good to me.
I have one question though. Should the CSL timer in SubMac run all the time after CSL 'logically' starts(Mac thinks it needs CslListen when it's in idle mode)? Or the timer restarts every time when the Mac calls CslListen?
I think it should run all the time after CSL 'logically' starts. Thus when Mac calls CslListen, SubMac knows whether CslSample or CslSleep should the state be at the moment. However, in this way, if Mac calls CslListen and hand control over SubMac csl, when csl timer fires each time, how would it know that it can change the state as it wishes? If currently Mac is doing some operations, like tx, but the csl timer is also working and fires, how should Csl timer find this and not change the state as it want?
I think we can add a variable mFree inside SubMac to represent whether Mac has yielded the control to it. The variable would be set to true when CslListen is called. We may also need to add a method Occupy to set it to false and call this method before each operation in PerformNextOperation. This means Mac takes the control back. Thoughts?

CSL Tx
I suggest adding/changing the method RequestCslFrameTransmission() to include parameters to indicate a requested CSL tx time (a tx time along with a guard interval before it).
Mac code would remember the pending requested CSL tx and its time window. When the time comes it would start the CSL tx operation and uses a callback to the next layer to ask it to prepare the frame.
Since Mac is already scheduling and prioritizing different requested operations, it is well-suited to manage the CSL tx timing as well.

Looks like we want to move the 8ms to Mac layer. So Mac will send frame to SubMac just a little bit ahead of sample time. This approach works for NCP CSL transmitter but might not work for RCP CSL transmitter which is widely adopted in products. If we want to do it this way, we might need to queue this to RCP(mirrored SubMac) with some time in advance to cover the time needed on bus(SPI or UART). Some platform provides APIs which used hardware timer for precise transmission timing control(nrf_802154_transmit_raw_at). If radio provide such capability, we can queue packet to platform and let platform maintain the transmit timing.
Another problem is, when RequestCslFrameTransmission is called, how Mac schedule the direct sending packet and indirect sending packets? Should Mac start CSL transmission operation 8ms ahead of time to block any potential direct sending (8ms to cover the biggest frame transaction, including csma/ca, transmission, AIFS, Ack, pass TxDone all the way up to Mac).

Another problem is, when RequestCslFrameTransmission is called, how Mac schedule the direct sending packet and indirect sending packets? Should Mac start CSL transmission operation 8ms ahead of time to block any potential direct sending (8ms to cover the biggest frame transaction, including csma/ca, transmission, AIFS, Ack, pass TxDone all the way up to Mac).

@brianljt
Yes. Ideally, when Mac starts CSL transmission exactly 8ms ahead of the expected tx time, it passes the action to SubMac and its internal state becomes 'doing a CSL Tx operation' and hence block other operations.
However after SubMac receives this task, there are still 8ms before it should send the frame. So SubMac uses a timer to wait 8ms. During this 8ms, we can do receive or sleep(it's not decided yet). But during this 8ms, no other operations in Mac could be done.

Update
We found in spec <IP_ADDRESS>.4 CSL Phase Definition for Thread Interfaces:

The CSL Receiver SHOULD be ready to receive earlier than the time specified in
CSL Phase and SHOULD stay ready to receive until after the time specified in CSL
Phase, according to the implementation and accuracy expectations.

Does it mean we shouldn't let Tx break CSL sampling? @abtink @jwhui

We change Mac::HandleTimer() to handle this case, i.e., if current operation (mOperation) is kOperationIdle (and  if there is a pending CSL tx request (though this check can be optimized out)) we call PerformNextOperation() (which would then start the CSL tx operation).

What if in Mac::HandleTimer, the mOperation is not kOperationIdle? Shall we return an error to the upper layer? Since the timer is started in UpdateIdleMode, I think it's possible that other operations are handled after the timers starts but before the timer fires.
As an example:

after the timer is started in UpdateIdleMode, mOperation becomes kOperationIdle.
And at this moment, StartOperation(kOperationTransmitDataDirect) is called.
Then PerformNextOperationis called. However it finds that it's within the guard interval. So it does the TransmitDataDirect operation first. mOperation becomes kOperationTransmitDataDirect.
Before the direct transmission completes, the timer fires. And it finds the state is not kOperationIdle at this moment.
What should it do in this case?

@abtink

What if in Mac::HandleTimer, the mOperation is not kOperationIdle? Shall we return an error to the upper layer? Since the timer is started in UpdateIdleMode, I think it's possible that other operations are handled after the timer starts but before the timer fires.
As an example:

after the timer is started in UpdateIdleMode, mOperation becomes kOperationIdle.
And at this moment, StartOperation(kOperationTransmitDataDirect) is called.
Then PerformNextOperationis called. However it finds that it's within the guard interval. So it does the TransmitDataDirect operation first. mOperation becomes kOperationTransmitDataDirect.
Before the direct transmission completes, the timer fires. And it finds the state is not kOperationIdle at this moment.
What should it do in this case?

@abtink

mTimer is already used during different MAC operation (you can check the code to see how it's being used). The idea is to try to re-use the same timer for scheduling CSL tx as well.

What if in Mac::HandleTimer, the mOperation is not kOperationIdle? Shall we return an error to the upper layer?

No, absolutely not, otherwise, it will mess up other uses of the timer. We actually need to stop the timer whenever/before we start any other (non-idle) operation (in case it is already running for scheduling the next CSL tx time).

Then PerformNextOperationis called. However it finds that it's within the guard interval. So it does the TransmitDataDirect operation first. mOperation becomes kOperationTransmitDataDirect.
Before the direct transmission completes, the timer fires. And it finds the state is not kOperationIdle at this moment.
What should it do in this case?

At the end of any MAC operation, the code calls again PerformNextOperation() to check if there are any next/pending operations and start executing it (you can check the Mac code to see how it's done). So if at that point, it's time for the CSL tx, we will handle it from PefromNextOperation().
With regards to scheduling next CSL tx time, the only situation we need (and should be allowed) to use the mTimer is when mOperation is indeed idle (kOperationIdle) and then from HandleTimerwe callPerformNextOperation()` to start CSL tx.

Responding to @brianljt on https://github.com/openthread/openthread/pull/4557#issuecomment-657343786

Looks like we want to move the 8ms to Mac layer. So Mac will send frame to SubMac just a little bit ahead of sample time. ...
Another problem is, when RequestCslFrameTransmission is called, how Mac schedule the direct sending packet and indirect sending packets? Should Mac start CSL transmission operation 8ms ahead of time to block any potential direct sending (8ms to cover the biggest frame transaction, including csma/ca, transmission, AIFS, Ack, pass TxDone all the way up to Mac).

My proposal here is basically an improvement over what the implementation in this PR is currently doing. Basically move the logic that schedule the CSL tx from higher layer code to Mac  layer. Since Mac is aware of and is scheduling all other operations that can use the radio, it's well suited to coordinate the CSL tx time as well.

This approach works for NCP CSL transmitter but might not work for RCP CSL transmitter which is widely adopted in products. If we want to do it this way, we might need to queue this to RCP(mirrored SubMac) with some time in advance to cover the time needed on bus(SPI or UART). Some platform provides APIs which used hardware timer for precise transmission timing control(nrf_802154_transmit_raw_at). If radio provide such capability, we can queue packet to platform and let platform maintain the transmit timing.

I was thinking of RCP use-case and I think this model can work for RCP as well.
The idea is that Mac would prepare and send the frame to SubMac/RCP layer ahead of the actual tx time (a guard interval before it). The SubMac uses a timer to ensure the frame is sent in time (or if radio driver provides such function, it uses it).

This guard interval should take into account all the possible delays (preparing the frame, sending over the bus to RCP, processing time on RCP, etc).
We need to ensure the guard time is build-time configurable so it's easy to change and try out  different numbers per use.
I think this guard value is really platform/device dependent.

I think the goal of this PR/work should be to add CSL support and this approach is reasonable/fine solution for RCP.
In longer term (future work and PR), I think we should consider improving/re-designing the RCP model and add more functionality from Mac and other higher module to the RCP itself:

We can have RCP handle preparation and tx of beacon frames.
We can have RCP handle the ActiveScan operation (sending beacon request, etc) or perform full (multi-channel) energy scan.
RCP can send data polls (give it to the interval and have it manage when to send data polls) - not high priority IMO since most RCP use-case the device is router.
We can move handling of data polls and indirect tx to RCP (requires new interactions between host and RCP and that RCP maintains a queue/list of pending frames for different children)
The CSL tx model can be improved along with the above (to give frame way ahead of time to RCP to send it on time).

Update
We found in spec <IP_ADDRESS>.4 CSL Phase Definition for Thread Interfaces:

The CSL Receiver SHOULD be ready to receive earlier than the time specified in
CSL Phase and SHOULD stay ready to receive until after the time specified in CSL
Phase, according to the implementation and accuracy expectations.

Does it mean we shouldn't let Tx break CSL sampling? @abtink @jwhui

To me there is still room for different interpretations.
We can say, this is defining how the "CSL idle listening" should be implemented and Thread spec is ensuring that radio is in rx mode for entire duration (to ensure implementation does not do an energy detection or sampling (from 15.4 spec)).
And that this applies when device is doing  "CSL idle listening" and is in idle mode and when sending a frame, we are not idle so not in this mode :)

At the end, I think either way of implementing the code would be fine. And in both the SubMac code need to be updated.

Update
We found in spec <IP_ADDRESS>.4 CSL Phase Definition for Thread Interfaces:

The CSL Receiver SHOULD be ready to receive earlier than the time specified in
CSL Phase and SHOULD stay ready to receive until after the time specified in CSL
Phase, according to the implementation and accuracy expectations.

Does it mean we shouldn't let Tx break CSL sampling? @abtink @jwhui

Note that the wording is SHOULD, which leaves the behavior up to implementation. The intention of this text is more guidance to the implementor to help explain how CSL channel samples are supposed to work normally.

I was thinking of RCP use-case and I think this model can work for RCP as well.
The idea is that Mac would prepare and send the frame to SubMac/RCP layer ahead of the actual tx time (a guard interval before it). The SubMac uses a timer to ensure the frame is sent in time (or if radio driver provides such function, it uses it).

Thanks Abtin. We had a discussion about this new approach and you are right, it can also handle RCP case. RCP should use a longer guard interval compared to NCP case as bus transaction takes longer than just pass down the stack on same CPU. As long as the guard interval takes the bus transaction time into consideration, and make sure the packet arrives at RCP SubMac ahead of next sample window, then it is fine.
We can add a new capability like OT_RADIO_CAP_TRANSMIT_TIMING to indicate whether platform supports precise transmission timing or not. If so, SubMac will not hold the packet and directly handle it to radio. Otherwise, SubMac handle the transmission timing, just like what we have for CSMA, retries and tx security.

Regarding the problem of separating the logic of DataPollHandler and CslSender handlers, after another considering, I think they cannot be totally independent. Something has to be shared between them.
I think the essence of DataPollHandler and CslSender are both to send the messages in the queue. So at least, aChild.GetIndirectMessage() are needed by both of them. We can see the process as: The parent just try to send all the messages in a child's queue to that child one by one, by one of the two ways, no matter which one. The two ways can alternate, be used in any order.
And things like FrameCounter, KeyId, Dsn when a Tx fails are also required for both of them to do a retx.
mIndirectTxAttempts should be shared too: If we keep two TxAttempts, how should we handle it when one reaches max while the other doesn't? If we allow the frame in only one way whose TxAttempts doesn't reach max, it would be very weird. Say, DataPoll's counter reaches max, and CSL's doesn't. And a child sends a Data poll at the moment. What frame should be sent by the parent? If there's no CSL at all, the next frame should be sent to the child. But for CSL, since it hasn't reached max, it should still send that frame.
I think the logic of PurgeRequest and ReplaceRequest should also apply for CSL. Take ReplaceRequest as an example, it's initiated by upper calls like RemoveMessages. When the a message is removed, we should also let CSL stop sending that message.
I suggest put these things that shared by both ways into IndirectSender. And we always obey the norm that the shared things could only be changed by one of them in the same time.(when a new message is added, or in their callbacks when some message is removed, or a tx fails, or a tx completes).
Thoughts? @abtink

Regarding the problem of separating the logic of DataPollHandler and CslSender handlers, after another considering, I think they cannot be totally independent. Something has to be shared between them.
I think the essence of DataPollHandler and CslSender are both to send the messages in the queue. So at least, aChild.GetIndirectMessage() are needed by both of them. We can see the process as: The parent just try to send all the messages in a child's queue to that child one by one, via one of the two ways, no matter which one. The two ways can alternate, be used in any order.
And things like FrameCounter, KeyId, Dsn when a Tx fails are also required for both of them to do a retx.

We had a chat last night on this. Let me add my thoughts on this:
Yes. The CslSender and DataPollHandler should act as two parallel mechanisms for tx to sleepy children. They are different/separate modules but they of course need to share/access same info. As you said, they would need to access/share the same Messages in the message queue and same child table and Child entries. For example, they both need to share the ChildInfo fields related to retx of frame Dsn, KeyId, etc (as you mentioned).
A key point here is that Mac operations are serialized and therefore at any time only one Mac operation is active/running, and the updates to these shared variables (almost in all cases) happen from the Mac callbacks when an operation is started/finished. So we don't need to worry about the two modules CslSender and DataPollHandler (and MeshForwarder) being active and controlling Mac at same time and updating/changing same info (e.g., Dsn value update in ChildInfo for retx).

mIndirectTxAttempts should be shared too: If we keep two TxAttempts, how should we handle it when one reaches max while the other doesn't? If we allow the frame to be sent in only one way whose TxAttempts doesn't reach max, it would be very weird. Say, DataPoll's counter reaches max, and CSL's doesn't. And a child sends a Data poll at the moment. Which frame should be sent by the parent? Should it be the failed one or the next one? If there's no CSL at all, the next frame should be sent to the child since the counter already reaches max. But for CSL, since it hasn't reached max, it should still send that old failed frame.

Regarding TxAttemp counter, as I mentioned, we can use the same counter (and it would easier implementation).
However, if we use different counters it can make it more flexible and allow support for different max attempts. Here is how to handle it:

If a frame reaches its max attempts for indirect (data poll based) tx, we drop the message/frame.
If however a frame reaches its limit for CSL tx (which may point that we are out of sync with the child), we stop using
CSL tx for this message, but do not remove/drop the message and let it go through data poll (indirect) tx mechanism.
Note that any point we will still have one (head of queue) message for indirect tx and/or CSL tx (we may not allow CSL tx for the message anymore if we reach its max tx attempts).

I think the logic of PurgeRequest and ReplaceRequest should also apply for CSL. Take ReplaceRequest as an example, it's initiated by upper calls like RemoveMessages. When the a message is removed, we should also let CSL stop sending that message.

As I mentioned in our chat, we can consider using the same purge/replace model for CSL tx as we have for DataPollHandler. but I think we don't need it (specially for the first implementation), and it can realized in a simpler way for CslSender case.
Here is the high-level design of CslSender:
CslSender has a method that calculates the next CSL tx time CalculateNextCslTx():

It would go through all the SED Child entires, check if there is a pending message for the child
(child->GetIndirectMessage() is not nullptr) and determines the earliest CSL tx time (and for which child and message it is).
It should probably keep a pointer to the selected Child for the next CSL tx (variable mCslTxChild) and may be also the actual next tx time (if mCslTxChild is nullptr it indicates we don't have any next CSL tx or it was canceled).
It requests the Mac to schedule a CSL tx for the next tx time.
Then we wait for Mac callback to ask for the frame to prepared. At this point, we track using mBusy flag that we are actually busy sending.
Then wait for Mac callback indicating end of CSL tx and call the appropriate method on next layer to update message (probably IndirectSender()). We need to also call CalculateNextCslTx() to schedule next CSL tx time.

CslSender must also have a method (let's call it Update() for now) that other modules call to notify it that something has changed and it needs to re-calculate the next CSL tx time (and informs Mac if there is any change).

The Update() should be called whenever a message for a child is added/removed, and (more importantly) if a child's CSL parameter changes.
For example, we need to call CslSender::Update() from all places in IndirectSender where we call DataPollHandler methods HandleNewFrame() or RequestFrameChange().
In Update() we do the following:

If we are not busy (mBusy not set), simply call CalculateNextCslTx() - which would calculate next tx and inform Mac of any change in tx time.
If we are busy (Mac is in middle of CSL tx), we wait for the Mac send-done callback to calculate next CSL tx. Basically we let the current CSL tx to finish.

One thing here when mBusy is set is to check if the change (that triggered Update() call) is impacting the current mCslTxChild and the message we are currently sending to it. If so, we just remember that message was replaced/deleted and the next layer (IndirectSender is no longer interested in send-done callback for it). We can simply set mCslTxChild to nullptr and from Mac send-done callback check for it not being nullptr before informing next layer.
I think this check and logic would be basically equivalent to purge/replace frame from DataPollHandler.

I suggest put these things that shared by both ways into IndirectSender. And we always obey the norm that the shared things could only be changed by one of them in the same time.(when a new message is added, or in their callbacks when some message is removed, or a tx fails, or a tx completes). For the rest parts, we separate them into respective modules.

Quick background here: One of the reasons we divided the logic for indirect (data-poll) tx into two modules of  IndirectSender and DataPollhandler is to make it easier to later move some of the logic to RCP. The idea is that IndirectSender would run on host, and DataPollHandler would run on RCP. That's why we have all this async call/callback model between them.
For future consideration I think the CslSender (with functionality as defined above) would be probably running on the host side.
The key difference between CSL tx and indirect (data poll) tx is that for indirect tx, we need to inform RCP of all children for whom we have a pending messages (and maybe pass a frame for each child to RCP). The reason is we don't know when we may receive a data poll and from which child we may get it first. For CSL tx, sender already knows the timing of all future tx, and we only need to let RCP know about the next/upcoming CSL tx.

@abtink
Many thanks. I think we've decided the major issues for the first implementation of CSL. Let me summarize it briefly here:

CSL receiver(SSED): Regarding the coordination issue, we would always prioritize operations from Mac over CSL logic for now. A new method CslListen would be added in SubMac, which could be regarded as a peer of Sleep, Send and Receive. Refer to comment-1 This method would do periodic sleeping and sampling and it would only be called when Mac is in the Idle mode. The reason for such a choice is that it is the most simple way to implement for now. We cannot be sure for if this choice is good enough for now. We may change it and enhance it after we have more performance data.
CSL transmitter:

We would let Mac provide a service RequestCslTransmission which takes an expected tx time as parameter. And the upper layer(the Csl scheduler) may call this method to request csl tx. And Mac would use its timer to be alarmed a little ahead of the expected tx time and request Submac::Send.
Regarding the coordination of DataPollHandler and CslScheduler, we try to make them as independent as possible. Inevitably, they would share the indirect messages, frameCounter, KeyId and DSN(The three are for Re-tx). In terms of TxAttempts, we would also handle them separately.(Refer to comment-2) After another consideration, I think it won't be very complex if we keep a Re-tx counter for each of them.

Do you have any other suggestions or questions? @bukepo @brianljt @jwhui
Of course I may get some other issues when I code and would do corresponding changes then.

@Irving-cl thanks. sounds good to me.

Have updated the implementation as we've discussed. Please have another review :)
I made all the changes in this commit.
@abtink @brianljt @bukepo @librasungirl

Thanks @Irving-cl for updating the PR. The CSL tx related logic is Mac look great. Overall it is a great improvement. :)
Related to CSL rx/sample mechanism,  I have some suggestions. Let me first summarize the list of methods and interactions from the code:
In SubMac we have:

SubMac::SetCslPeriod() - sets a member variable mCslPeriod.
SubMac::IsCslEnabled() - tied to mCslPeriod from SetCslPeriod() being non-zero (seems not to be used).
SubMac::IsCslStarted() - tied to whether StartCsl() is called before (used by Mac).
SubMac::StartCsl() practically starts the CSL timer in sub-mac - returns error (e.g. if mCslPeriod is zero) but the error is always ignored from all places this seems to be called.
SubMac::StopCsl() practically stops the CSL timer in sub-mac.
SubMac::CslSample() which is main method used by Mac to put sub-mac in CSL sampling state/mode.

In Mac we have:

Mac::StartCsl() - it is a public method, I see it used outside of Mac only from Mle::SetStateChild().
Mac::StopCsl() - it is public method, but does not seem to be used outside of Mac itself.
Mac::SetCslPeriod() - it would auto-start the CSL if the given CSL period is non-zero and auto-stop it if it is zero.
Mac::IsCslStarted() - which just calls the SubMac method. This is public is used from Mle class (e.g., to decide if to append certian TLVs),

Within Mac the Start/StopCsl() are used as follows:

From Mac::SetRxOnWhenIdle() based on change to mRxOnWhenIdle flag (device becoming sleepy or always on).
From Mac::SetPanChannel() though I am not sure if its required
From Mac::SetCslPeriod() based on period value (non-zero to start and zero to stop).

I think there are many methods and interactions between Mac and SubMac and there may be some room for simplification.

Here is what I suggest:
In SubMac we just keep two methods (from the list above):

SubMac::SetCslPeriod(aPeroid): Calling this would also start/stop the CSL timer in sub-mac (based on period being zero or not).
SubMac::CslSample(): We keep it as it is currently (i.e. it puts the sub-mac in the CSL sample state).

In Mac we can keep two methods and do the following:

We remove StartCsl() and StopCsl() from Mac.
We keep SetCslPeriod() which informs SubMac and also does Get<Mle::Mle>().ScheduleChildUpdateRequest().
We let Mac itself track whether CSL is started (or may be better calling if CSL is enabled) or not.
We can update/add Mac::IsCslEnabled() (replacing IsCslStarted()) which would do all the same checks done currently from  Mac::StartCsl(), i.e. CSL period is non-zero, device is sleepy and a child with a 1.2  parent, ect:

  bool Mac::IsCslEnabled(void) const
  {
      return (GetCslPeriod() > 0) && !mRxOnWhenIdle && Get<Mle::MleRouter>().IsChild() &&
              Get<Mle::Mle>().GetParent().IsEnhancedKeepAliveSupported();
  }

We can also cache and keep track of this in bool mCslEnabled but we would then need to monitor whenever role changes, or parent changes, or parent version changes, to update the mCslEnabled (so calculating it when we need it is definitely simpler).
In Mac::UpdateIdleMode() we can directly use this:

   #if OPENTHREAD_CONFIG_MAC_CSL_RECEIVER_ENABLE
          if (IsCslEnabled())
          {
              IgnoreError(mSubMac.CslSample());
              ExitNow();
          }
  #endif

I think this would work and should be simpler.

On possible future enhancement is to change the SubMac CSL timer to only run after the SubMac::CslSample() is called.
Note that we would still need to track the mCslSampleTime but can update it and then schedule the CSL timer when SubMac enters kStateCslSample state. One advantage with this is that we may be able to re-use the same mTimer for CSL sample operation (no need to have a seperate mCslTimer).
I think letting timer run in the first implementation is perfectly fine.

Thanks @Irving-cl for updating the PR. The CSL tx related logic is Mac look great. Overall it is a great improvement. :)
Related to CSL rx/sample mechanism, I have some suggestions. Let me first summarize the list of methods and interactions from the code:
In SubMac we have:

SubMac::SetCslPeriod() - sets a member variable mCslPeriod.
SubMac::IsCslEnabled() - tied to mCslPeriod from SetCslPeriod() being non-zero (seems not to be used).
SubMac::IsCslStarted() - tied to whether StartCsl() is called before (used by Mac).
SubMac::StartCsl() practically starts the CSL timer in sub-mac - returns error (e.g. if mCslPeriod is zero) but the error is always ignored from all places this seems to be called.
SubMac::StopCsl() practically stops the CSL timer in sub-mac.
SubMac::CslSample() which is main method used by Mac to put sub-mac in CSL sampling state/mode.

In Mac we have:

Mac::StartCsl() - it is a public method, I see it used outside of Mac only from Mle::SetStateChild().
Mac::StopCsl() - it is public method, but does not seem to be used outside of Mac itself.
Mac::SetCslPeriod() - it would auto-start the CSL if the given CSL period is non-zero and auto-stop it if it is zero.
Mac::IsCslStarted() - which just calls the SubMac method. This is public is used from Mle class (e.g., to decide if to append certian TLVs),

Within Mac the Start/StopCsl() are used as follows:

From Mac::SetRxOnWhenIdle() based on change to mRxOnWhenIdle flag (device becoming sleepy or always on).
From Mac::SetPanChannel() though I am not sure if its required
From Mac::SetCslPeriod() based on period value (non-zero to start and zero to stop).

I think there are many methods and interactions between Mac and SubMac and there may be some room for simplification.
Here is what I suggest:
In SubMac we just keep two methods (from the list above):

SubMac::SetCslPeriod(aPeroid): Calling this would also start/stop the CSL timer in sub-mac (based on period being zero or not).
SubMac::CslSample(): We keep it as it is currently (i.e. it puts the sub-mac in the CSL sample state).

In Mac we can keep two methods and do the following:

We remove StartCsl() and StopCsl() from Mac.
We keep SetCslPeriod() which informs SubMac and also does Get<Mle::Mle>().ScheduleChildUpdateRequest().
We let Mac itself track whether CSL is started (or may be better calling if CSL is enabled) or not.
We can update/add Mac::IsCslEnabled() (replacing IsCslStarted()) which would do all the same checks done currently from  Mac::StartCsl(), i.e. CSL period is non-zero, device is sleepy and a child with a 1.2  parent, ect:

  bool Mac::IsCslEnabled(void) const
  {
      return (GetCslPeriod() > 0) && !mRxOnWhenIdle && Get<Mle::MleRouter>().IsChild() &&
              Get<Mle::Mle>().GetParent().IsEnhancedKeepAliveSupported();
  }

We can also cache and keep track of this in bool mCslEnabled but we would then need to monitor whenever role changes, or parent changes, or parent version changes, to update the mCslEnabled (so calculating it when we need it is definitely simpler).
In Mac::UpdateIdleMode() we can directly use this:

   #if OPENTHREAD_CONFIG_MAC_CSL_RECEIVER_ENABLE
          if (IsCslEnabled())
          {
              IgnoreError(mSubMac.CslSample());
              ExitNow();
          }
  #endif

I think this would work and should be simpler.
On possible future enhancement is to change the SubMac CSL timer to only run after the SubMac::CslSample() is called.
Note that we would still need to track the mCslSampleTime but can update it and then schedule the CSL timer when SubMac enters kStateCslSample state. One advantage with this is that we may be able to re-use the same mTimer for CSL sample operation (no need to have a seperate mCslTimer).
I think letting timer run in the first implementation is perfectly fine.

@abtink Thanks for the very kind review.
I have removed many methods. I implemented it slightly different as you described. I think Mac::SetCslPeriod might not be the only entrance to start or stop CSL. For example, when an SSED changes RxOffWhenIdle mode to RxOnWhenIdle, the csl period is not changed, however I think it should stop CSL sampling at this moment. And a more common case is setting csl period before an SSED attaches to a parent.(I think the spec doesn't mention csl period could only be set after a RxOffWhenIdle child attaches to a parent. A child should start CSL after it attaches to a parent if csl period is set before. And the current test harness also expects this behavior.) If we put everything into SetCslPeriod, then it would be difficult to handle these two case.
So I made a tiny change. I added a new variable mCslPeriodInactive into Mac which always records the latest csl period set by upper layer. And when Mac::SetCslPeriod is called, it might not notify SubMac to do any changes(i.e. the child hasn't attached to a parent), however it would set mCslPeriodInactive first.
And I added a method Mac::ChangeCslState to decide if it would actually notify SubMac to do the changes(start or stop CSL). Mac::ChangeCslState is used in Mac::SetCslPeriod and other places like, when the child is just attached or the Rx mode changes. When is called, it would refer to mCslPeriodInactive and sync the value to SubMac to make an effect(If IsCslEnabled).
Thoughts?

I have removed many methods. I implemented it slightly different as you described. I think Mac::SetCslPeriod might not be the only entrance to start or stop CSL.

Thanks @Irving-cl.
I think with the new approach, we don't need to think of the notions of starting and stopping CSL anymore.
Here is high-level behavior:

SubMac is running a CSL timer which keeps track of the different window intervals.

It starts the timer as soon as a non-zero CSL period is set on it and stops it when period is set to zero.
The timer running does not mean it is actually doing CSL sample (just tracking the different window intervals).
SubMac provides SubMac::CslSample() to allow Mac to control when it should do CSL sampling.

Mac is the module that decides when SubMac is should do CSL sampling.

It makes the decision from UpdateIdleMode() (when it is entering idle mode/operation or is already idle), i.e. it decide whether we need to call mSubMac.Sleep() or issue mSubMac.CslSample().
At the point when the decision is needed, Mac checks all the related conditions (parameters and states) to decide this (i.e., is CSL period non-zero, is device a child, is it sleepy, doest the parent support 1.2 etc.?)

The key advantage of making the decision at the point of time when we want to issue command to mSubMac and based on all parameters/states, is that we don't need to track and/or worry if/when any of these parameters/conditions changes. This helps remove the need for us to think/consider the notions of starting/stopping CSL. It will happen automatically as parameters/states change.
One thing to add here: The CSL timer in SubMac may continue to run even if we don't want to use CSL sampling. But I think it's fine for the first implementation. As I mentioned, we can improve this in a future enhancement where we
start the timer only when SubMac is told to enter kStateCslSample state (i.e., SubMac::CslSample() is called).
For this model to work we just need to make sure UpdateIdleMode() is properly called in Mac from any method that can possibly change any of the conditions/parameters/states that impact the decision in Mac::IsCslEnabled(). This is already done in most cases but we need to add it in few new places:

We need to call UpdateIdleMode() at the end of Mac::SetCslPeriod() (suggested this in the review).
Another place to call UpdateIdleMode() is at the end of Mac::HandleReceivedFrame() after calling Get<MeshForwarder>().HandleReceivedFrame(*aFrame);:

This covers all cases where device role changes, or parent changes, or parent version changes, etc.
Note that these conditions can change after the device receives and processes some frame, or after the device sends a frame.
So calling UpdateIdleMode() after processing a received frame would cover any changes due to a received frame.
The tx of frame is already covered by the existing code.

Tx of frame is itself a Mac operation.
Mac already calls UpdateIdleMode() the end of send (or in general any) operation, and importantly this is done after invoking next layer tx-done callback.
For example, in Mac::HandleTransmitDone() when the active operation is kOperationTransmitDataDirect, we call Get<MeshForwarder>().HandleSentFrame(aFrame, aError); and then after this Mac::PerformNextOperation(); is called which in turn would (if no pending operations) call UpdateIdleMode().

For example, when an SSED changes RxOffWhenIdle mode to RxOnWhenIdle, the csl period is not changed, however I think it should stop CSL sampling at this moment. And a more common case is setting csl period before an SSED attaches to a parent.(I think the spec doesn't mention csl period could only be set after a RxOffWhenIdle child attaches to a parent. A child should start CSL after it attaches to a parent if csl period is set before. And the current test harness also expects this behavior.) If we put everything into SetCslPeriod, then it would be difficult to handle these two case.

I think these cases would actually work fine. Let me try to go through some scenarios in detail. I think it would be a good exercise to sanity-check and verify the approach.

For example, when an SSED changes RxOffWhenIdle mode to RxOnWhenIdle, the csl period is not changed, however I think it should stop CSL sampling at this moment.

Scenario 1
Device is already doing CSL sampling and then RxOnWhenIdle changes.

Initial state:

SubMac is in kStateCslSample
CSL period is non-zero (CSL timer in SubMac is running).
Mac is idle (kOperationIdle) and mRxOnWhenIdle is false (device is sleepy).

Now Mac::SetRxOnWhenIdle(true) is called

At the end of SetRxOnWhenIdle() we call Mac::UpdateIdleMode()
Mac::UpdateIdleMode() would check mRxOnWhenIdle being true and issue SubMac::Receive()
SubMac now puts the radio in receive mode and won't do CSL sampling anymore.
Note that the CSL timer is SubMac continues to run in background, but that's fine for first implementation (can be optimized in future).

Scenario 2
Device is switched from rx-always-on to be sleepy (already attached to a 1.2 parent).

Initial state:

SubMac is in kStateReceive
CSL period is non-zero (CSL timer in SubMac is running).
Mac is idle (kOperationIdle) and mRxOnWhenIdle is true (device is rx-always-on).

Now Mac::SetRxOnWhenIdle(false) is called

At the end of SetRxOnWhenIdle() we call Mac::UpdateIdleMode()
Mac::UpdateIdleMode() would check IsCslEnabled() returning true and would issue SubMac::CslSample()
SubMac now would do CSL sampling.

Scenario 3
Sleepy child device already attached to a 1.2 parent. CSL period is zero. User then sets it to a non-zero value.

Initial state:

SubMac is in kStateSleep (device is sleepy)
CSL period is zero (CSL timer in SubMac is not running).
Mac is idle (kOperationIdle) and mRxOnWhenIdle is false (device is sleepy).

Now Mac::SetCslPeriod(aPeriod) is called with non-zero value.

Mac::SetCslPeriod() would call SubMac::SetCslPeriod() which would in turn start the CSL timer.
After this in  Mac::SetCslPeriod() we call Mac::UpdateIdleMode().
Mac::UpdateIdleMode() would then check that Mac::IsCslEnabled() returning true and issue SubMac::CslSample()
SubMac now would do CSL sampling.

And a more common case is setting csl period before an SSED attaches to a parent.(I think the spec doesn't mention csl period could only be set after a RxOffWhenIdle child attaches to a parent. A child should start CSL after it attaches to a parent if csl period is set before.

Scenario 4
Device is not attached and user has set the CSL period (non-zero value). Device is configured as sleepy. Device then attaches to a 1.2 parent.

Initial state:

CSL period is non-zero so CSL timer in SubMac is running.

Device attached to parent.

The attach process requires frame exchanges with parent, i.e., MLE parent request/response, Child ID request/response.
Device would change its role to kRoleChild on receiving and processing a "Child ID Response" frame from its parent.
At the end of any received frame, in Mac we call UpdateIdleMode() which would then detect that device is now a child and all conditions are met for IsCslEnabled() and would call mSubMac.CslSample().

Scenario 5
SSED device already attached to a 1.2 parent with a non-zero CSL period and does CSL sampling.
We have "periodic parent search" feature enabled on SSED device. The device later finds a new/better parent which is not a 1.2 parent and switches to the new parent.

Device switching to new parent (from "periodic parent search").

The attach to new parent again requires frame exchanges with it.
The parent only changes after "MLE Child ID Response" frame is received from the new parent.
Again since at the end of any received frame, we call Mac::UpdateIdleMode(), this would detect the parent is changed and that the new parent is no longer a 1.2 device, so IsCslEnabled() would return false and Mac would call mSubMac.Sleep() (putting it to sleep instead of CSL sampling).

So I made a tiny change. I added a new variable mCslPeriodInactive into Mac which always records the latest csl period set by upper layer. And when Mac::SetCslPeriod is called, it might not notify SubMac to do any changes(i.e. the child hasn't attached to a parent), however it would set mCslPeriodInactive first.

And I added a method Mac::ChangeCslState to decide if it would actually notify SubMac to do the changes(start or stop CSL). Mac::ChangeCslState is used in Mac::SetCslPeriod and other places like, when the child is just attached or the Rx mode changes. When is called, it would refer to mCslPeriodInactive and sync the value to SubMac to make an effect(If IsCslEnabled).

I think we won't need this with above approach.

@abtink
Many thanks. The new approach makes sense to me. I have updated the implementation.

Thanks @Irving-cl. I read the rest of code and layer above Mac/SubMac, CslTxScheduler, etc.
Overall looks good.  I have some suggestions/comments which I mentioned in the code review above.
There is one thing that I'd like to bring up to discuss in more details here.
First a quick overview/background on the indirect tx
As you may know and/or have heard me mention before, in OT core indirect tx logic is divided across two related modules/classes: IndirectSender and DataPollHandler.

One of the reasons for this division is for future use in RCP/host architecture.
The idea is that IndirectSender contains the code/logic that would run on the host side
And the DataPollHandler would run on the RCP.

That's why we have rather specific interface between the two classes with async requests and callbacks along with an abstracted frame context exchange.
Each module defines its own ChildInfo which includes the set of states/variables that it needs to track per Child/Neighbor for its portion of indirect tx operation.

There are few changes in this PR that complicates the separation of the the modules and their move to RCP and host. I will list them first and then suggest some ways to address/change the code.

In FrameContext in this PR we now have a void *mMessage variable.

This would be only useful for CSL tx and won't be used/needed for data poll indirect tx. Note that  FrameContext is the info that RCP would need to save for each frame per Child, so we want the context to be as small as possible. Also RCP should have any notion of message and deal with frames. The host and IndirectSender would track the Message for each child.

The other change in this PR is that three variables mIndirectFrameCounter, mIndirectKeyId, and mIndirectDsn are now moved from DataPollHandler::ChildInfo to IndirectSender::ChildInfo.

These parameters are required for retx of indirect frames and this logic belongs to the RCP and DataPollHandelr

Here are my suggestion on how to change the code:
On FrameContext change.

We remove the void *mMessage definition in FrameContext.
Instead we can add a Message * mCslTxMessage in class CslTxScheduler

It is set to nullptr
In CslTxScheduler::HandleFrameRequest() (where the MAC is asking for frame to prepared), after frame is prepared we get the message (calling mCslTxChild->GetIndirectMessage()) and save it in mCslTxMessage
In CslTxScheduler::HandleSentFrame() at the same time that we clear mCslTxChild we also set mCslTxMessage to nullptr.

With this change we can actually optimize further and remove mBusy variable in CslTxScheduler

Since the mCslTxMessage pointer itself indicates the same thing, i.e. when it is not null we are busy (i.e. MAC is performing CSL tx) and when it is nullptr we are not busy.

On the three variables mIndirectFrameCounter, mIndirectKeyId, and mIndirectDsn:

We can keep them in DataPollHandle::ChildInfo
The CslTxScheduler is already a friend of the DataPollHandle::ChildInfo so should be able to access these.
This should work fine.

For  future consideration/enhancement, we can think of how to divide the CSL tx logic between RCP and host.
I can see two ways:

We can have most of the code that is today in CslTxShceduer run on host

Host calculates the next CSL tx and which child should be served next
Tells RCP the time and the child.
RCP should already know/have the frame for the child, since it would get it for indirect tx (from data-poll handler logic that is running on the RCP).

We move all the logic code for CSL tx scheduling to RCP itself (I slightly prefer this).

RCP already need to maintain a child table, and can therefore track which child supports CSL and the child CSL timings (phase, period, etc).
RCP would then calculate the next CSL tx on its own (it knows/tracks pending frames for each child again from data-poll handler code).

This is all for future, I don't think we want to decide it in this PR and first implementation of CSL logic. :)

Thanks @abtink for the very useful comments. I've updated the implementation again and I think the code has become neater and nicer. Please check if there are any other problems :)

Thanks @abtink and @brianljt
Have updated. Please have another review.


simulator get stuck only monitor server restart helps for not long

looking at the simulator, sometimes after some time, it gets very slow and not showing current data, refreshing the page wont help either.
only restarting the monitor server helps for a few hours and returnes to being very slow once again

Fixed


Fix types for formdata constructor property

should resolve
https://github.com/mailgun/mailgun-js/issues/110
and https://github.com/mailgun/mailgun-js/issues/107

@from-nibly I have the following code in my node project:
import Mailgun from 'mailgun.js'
import formData from 'form-data'

const mailgun = new Mailgun(formData: new () => FormData)

But I am still having issues. Any word on this?

It looks like you are mixing type definitions into the new of the constructor.
your code should look like this.
const mailgun = new Mailgun(formData);

Unfortunately until this is merged you will have to do
const mailgun = new Mailgun(formData as any);

to fix the type problems

Ok, great. That actually works for now. Thank you!

@andrii-yelis Is this project still maintained?

@from-nibly Yeh, I'm maintaining the project, but don't have much time to do it so sorry for the delays.

@from-nibly thanks for the fix, I've merged it will test and publish it to the npm!

@andreimoustache, were you able to fix the issue? I am experiencing the same problem.


Integrate Kaniko container image build task into pipeline

One of the first steps in the pipeline is to build a container image using Kaniko. We need to define this as a Task in our Tekton Pipeline spec.

Kaniko integrated into Pipeline via build-docker-image-from-git-source.yaml Task definition.


PostOrder traversal of Binary Tree

Please assign this to me.

can you assign this issue to me ?

!assign

@SiddyDevelops @anupeshverma assigned to you both!


CircuitBreaking - Outlier detection to detect custom http codes also(such as 429/400)

(This is used to request new product features, please visit https://discuss.istio.io for questions on using Istio)
Describe the feature request
I am using micorservices pattern deployed in cloud with istio service mesh. i was exploring the circuit breaker supports given by istio. Outlier detection is way to solve my problem but i see that it trip/take actions only with gateway error code[502-504]. How can i configure it to eject the pod if my service starts throwing some other code such as 429(too many requests) or any other error codes.

https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPRetry

@zirain  i want my serviceA to stop calling the serviceB (if serviceB gives 429) for some sleep window.

@ankitdiscovery does this work for you in HTTPRetry?

retryOn: Specifies the conditions under which retry takes place. One or more policies can be specified using a ‘,’ delimited list. If retry_on specifies a valid HTTP status, it will be added to retriable_status_codes retry policy. See the retry policies and gRPC retry policies for more details.

You may not be able to explicitly say not to retry on 429, but you can select from a list of retry other policies here:
https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on
I don't think the existing CircuitBreaker logic available in envoy has the ability to specify a particular return code for immediately ejecting an endpoint:
https://www.envoyproxy.io/docs/envoy/v1.23.1/api-v3/config/cluster/v3/outlier_detection.proto#envoy-v3-api-msg-config-cluster-v3-outlierdetection

@GregHanson You are right, HTTPRetry not gonna work for that usecase.
Also since we have a limited to trip circuit based on 5xx codes only , so i am thinking of returning some 5xx code instead of 429 (too many requests) from application, Although that is not something standard/advisable way. But it will buy some time for application pod to cool down and cleanup their existing work before getting added back to pool again.

marking as resolved


PromiseKit no longer builds for projects targeting 10.10

PromiseKit version: 4.4.2
Xcode version: Xcode Version 9.1 beta (9B46)
CocoaPods version: 1.4.0.beta.1
Carthage version: N/A

It appears that 191950c006909039895a30fb13d34e1d087a4822, which changed the minimum deployment target to OS X 10.11, makes it impossible to use PromiseKit in projects that have a lower deployment target (i.e. 10.10).

Sorry for my delay here, I am pressed for time currently.

As you can see by the tardiness of my replies I'm not really in a rush here.

I may have to say: use Carthage. CocoaPods barely works, here is an example of it. I cannot change the macOS deployment target to a lower value or the podspec won't pass even though there is no reason for this to be the case.

K this is now live on CocoaPods too for 6.0.1.

Thanks!


Incorrect interpolation using AMR

Hello all,
I am trying to implement non-conforming adaptive meshing for DG discretization of Euler equations.
I have implemented an custom error estimator based on the gradient of the density, (Lp norm of the gradient of density normalized by the volume of the element).  The error estimate looks correct but when I update the function spaces and gridfunctions the interpolation on the refined mesh is incorrect.
Here is what I am trying to do
/// Setup the estimator
GradientErrorEstimator estimator(density);
estimator.SetLocalErrorNormP(1);

double max_elem_error = 1.0e-2;
int nc_limit = 3;         // maximum level of hanging nodes
ThresholdRefiner refiner(estimator);
refiner.SetTotalErrorFraction(0.0); // use purely local threshold
refiner.SetLocalErrorGoal(max_elem_error);
refiner.SetNCLimit(nc_limit);

for (int ref_it = 1; ref_it < 2; ref_it++)
{
	refiner.Reset();
	refiner.Apply(pmesh);

	// Update the space: recalculate the number of DOFs and construct a matrix
	// that will adjust any GridFunctions to the new mesh state.
	vfes.Update();
	fes.Update();
	dfes.Update();

	// Interpolate the solution on the new mesh by applying the transformation
	// matrix computed in the finite element space. Multiple GridFunctions could
	// be updated here.
	sol.Update();
	density.Update();

	if (pmesh.Nonconforming())
	{
		// Update the space again, this time a GridFunction redistribution matrix
		// is created. Apply it to the solution.
		//std::cout << "in here: " << std::endl;

		vfes.Update();
		sol.Update();
		fes.Update();

		// Load balance the mesh.
		pmesh.Rebalance();

		// Update the space again, this time a GridFunction redistribution matrix
		// is created. Apply it to the solution.
		vfes.Update();
		sol.Update();

		fes.Update();
		density.Update();

                dfes.Update();

	}

	// Free any transformation matrices to save memory.
	vfes.UpdatesFinished();
	fes.UpdatesFinished();
	dfes.UpdatesFinished();
}

Here I am only doing one level of refinement and updating the spaces after I refine.
Here are some images of what I see.
Density field before refinement

Calculated error estimate for threshold refiner

Density field after the refinement and updates of spaces and gridfunctions (refinement is correct but interpolation is messed up)

My guess would be that you need to rebalance first (in your innermost if block), then update spaces, and finally update the grid-functions.

(Otherwise the space updates are no-ops, since the mesh didn't change from last time).

Is dfes your density FE space?  If so, it looks like the space update and gridfunction update are in the wrong order.

@jakubcerveny ,
I tried flipping the rebalance call in the if block
	for (int ref_it = 1; ref_it < 2; ref_it++)
	{
		refiner.Reset();
		//std::cout << "numElements inside ref Loop:" << pmesh.GetNE() << std::endl;
		refiner.Apply(pmesh);

		// Update the space: recalculate the number of DOFs and construct a matrix
		// that will adjust any GridFunctions to the new mesh state.
		vfes.Update();
		fes.Update();
		dfes.Update();

		// Interpolate the solution on the new mesh by applying the transformation
		// matrix computed in the finite element space. Multiple GridFunctions could
		// be updated here.
		sol.Update();
		density.Update();

		if (pmesh.Nonconforming())
		{

			// Load balance the mesh.
			pmesh.Rebalance();

			// Update the space again, this time a GridFunction redistribution matrix
			// is created. Apply it to the solution.
			//std::cout << "in here: " << std::endl;

			vfes.Update();
			sol.Update();
			fes.Update();
			density.Update();
			dfes.Update();

			// Update the space again, this time a GridFunction redistribution matrix
			// is created. Apply it to the solution.
			vfes.Update();
			sol.Update();

			fes.Update();
			density.Update();

			dfes.Update();
		}

		// Free any transformation matrices to save memory.
		vfes.UpdatesFinished();
		fes.UpdatesFinished();
		dfes.UpdatesFinished();
	}

It does not change the results. The interpolation still looks incorrect.  Also I do not understand why the balance call should be flippled.  I am following example 15 where the rebalance method is called after space update, maybe I am missing something.
@rw-anderson, dfes is a momentum space, fes is the density space so I think the order for that update is correct, maybe I am missing something.  Thanks you both of you for your help.
-Makrand

Do I have to somehow update the BlockVector  ? I am using blockvector to create references to individual variables in the sol GridFunction which contains all convervative variables --> density, momentum_x, momentum_y, Energy

Well a space depends on a mesh, and a grid-function depends on space. If a mesh changes (de/refined or rebalanced), you need to tell the space to update itself and to construct an update matrix, which is then used on the grid-functions. The update sequence is always mesh--space--GF.
If your space changes you may in turn need to update your BlockVectors.

@jakubcerveny Thank you for that explanation I understand that now.  What I was confused about was why the order is different in example 15 inside the if block?
How do I update the corresponding BlockVectors?

Here is my blockvector AMR update:
https://github.com/mfem/mfem/blob/tds-mhd-dev/miniapps/mhd/imAMRMHDp.cpp
One is for VSize and one is for TureVSize.

@makrandak, in example 15p there are two updates taking place. The first is done after refinement (which happens right before UpdateAndRebalance is called, this may be confusing). Then the mesh is rebalanced (if it is an NC mesh) and the space and GF need updating again. In both cases the sequence is always mesh--space--GF.
Refinement, derefinement and rebalancing are distinct operations, each of which requires a different update matrix/operator. We are unfortunately unable to combine these and do a single update after multiple mesh modifications, so the updates need to be done iteratively.

Okay I think the problem was the way I was referencing to the different variables in the solution GridFunction and I was not updating the blockVectors.  I think rest of the sequence for the updates was correct.
I don't know why but this worked
	ParGridFunction u_0, u_1, u_2, u_3;
	u_0.MakeRef(&fes, sol, offsets[0]);
	u_1.MakeRef(&fes, sol, offsets[1]);
	u_2.MakeRef(&fes, sol, offsets[2]);
	u_3.MakeRef(&fes, sol, offsets[3]);

instead of
ParGridFunction u_0(&fes, u_block.GetBlock(0));
ParGridFunction u_1(&fes, u_block.GetBlock(1));
ParGridFunction u_2(&fes, u_block.GetBlock(2));
ParGridFunction u_3(&fes, u_block.GetBlock(3));

Thanks for your help.

Hello everyone,
The problem of synchronization of AMR, BlockVectors and GridFunctions became actual for me too.
I have the similar idea for implementation of AMR with DG: the solution stores in BlockVector and have a reference onto ParGridFunction x; and ParGridFunctions rhok, mom, energy are used for the VTK visualisation.
I read the discussion and created the following update function:
void UpdateAndRebalance(
    ParMesh &pmesh,
    ParFiniteElementSpace &fes,
    ParFiniteElementSpace &dfes,
    ParFiniteElementSpace &vfes,
    ParGridFunction &x,
    MixedBilinearForm &a, /* Aflux */
    ParNonlinearForm &b,  /* A */
    ParGridFunction &rhok,
    ParGridFunction &mom,
    ParGridFunction &energy,
    BlockVector &u_block,
    Array<int> &offsets
)
{
    x.SetFromTrueDofs(u_block);
    rhok.SetFromTrueDofs(u_block.GetBlock(0));
    mom.SetFromTrueDofs(u_block.GetBlock(1));
    energy.SetFromTrueDofs(u_block.GetBlock(num_equation-1));

    cout << "IN Update 1 x = " << endl;
    x.Print(cout);

    fes.Update();
    dfes.Update();
    vfes.Update();

    x.Update();

    cout << "IN Update 2 x = " << endl;
    x.Print(cout);

    rhok.Update();
    mom.Update();
    energy.Update();

    if (pmesh.Nonconforming())
    {
        // Load balance the mesh.
        pmesh.Rebalance();

        // Update the space again, this time a GridFunction redistribution matrix
        // is created. Apply it to the solution.
        fes.Update();
        dfes.Update();
        vfes.Update();

        x.Update();

        rhok.Update();
        mom.Update();
        energy.Update();
    }

    // Compute new offsets
    for (int k = 0; k <= num_equation; k++)
        offsets[k] = k * vfes.GetNDofs();

    u_block.Update(offsets);

    cout << "IN Update 3 u_block = " << endl;
    u_block.Print(cout);

    rhok.GetTrueDofs(u_block.GetBlock(0));
    mom.GetTrueDofs(u_block.GetBlock(1));
    energy.GetTrueDofs(u_block.GetBlock(num_equation-1));

    x.GetTrueDofs(u_block);

    cout << "IN Update 4 x = " << endl;
    x.Print(cout);

   // Free any transformation matrices to save memory.
   fes.UpdatesFinished();
   dfes.UpdatesFinished();
   vfes.UpdatesFinished();
   fes_const.UpdatesFinished();

   // Inform the nonlinear and bilinear forms that the space has changed.
   a.Update();
   b.Update();
   a.Assemble();
}

When I run my computations I see x contains only zeros after x.GetTrueDofs(u_block); line. I guess something went wrong with some reference.
BlockVectors and ParGridFunctions were created at the main function as follows:
    Array<int> offsets(num_equation + 1);
    for (int k = 0; k <= num_equation; k++)
        offsets[k] = k * vfes.GetNDofs();
    BlockVector u_block(offsets);
    ParGridFunction sol(&vfes, u_block.GetData());
    ParGridFunction rhok, mom, energy;
    rhok.MakeTRef(&fes, sol, offsets[0]);
    mom.MakeTRef(&dfes, sol, offsets[1]);
    energy.MakeTRef(&fes, sol, offsets[dim+1]);

I tried to use MakeTRef or MakeRef but it had't have any effect.
Also, if I comment all SetFromTrueDofs and GetTrueDofs lines, zero values appears after u_block.Update operation only in case of mesh refinement.
I'd appreciate any help!
--
Best regards, Victoria

There is one important detail about using ParGridFunction::Update -- it will resize its data array automatically which in most cases means that the new data array will not be allocated as a block/sub-vector of a bigger block vector allocation.
One possible solution, which works well when all blocks use the same scalar FE space, is to only call ParGridFunction::Update for the ParGridFunction that represents all solution components, sol:
   vfes.Update(); // FE space of 'sol'

   sol.Update();

Then the BlockVector can be updated to use the data of sol (after updating offsets):
   // update 'offsets' ...
   u_block.Update(sol, offsets);

It is also good to do this at initialization -- allocate sol and define u_block to point to sol's data (instead of the other way around):
   ParGridFunction sol(&vfes);
   // initialize 'offsets' ...
   BlockVector u_block(sol, offsets);

After updating sol and u_block, all other componentParGridFunctions can be updated to point inside sol at the respective offsets with something like:
   const bool want_transform = false;
   dfes.Update(want_transform);

   mom.MakeRef(&dfes, sol, offsets[1]);
   // or
   // mom.MakeRef(&dfes, u_block.GetBlock(1));

When doing two updates (one for refinement, one for rebalancing), u_block, rhok, mom, energy can be updated only once -- after the second update. However, all FE spaces, fes, dfes, and vfes, and the ParGridFunction sol need to be updated after each mesh modification: refinement and rebalancing.


Error: error creating worker script: could not read response body: %!w()

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version
> terraform --version
Terraform v1.2.3
on linux_amd64

// Cloudflare CDN
cloudflare = {
  source  = "cloudflare/cloudflare"
  version = "3.23.0"
}

Affected resource(s)
cloudflare_worker_script
Terraform configuration files
# Create the worker script
resource "cloudflare_worker_script" "my_worker" {
  name    = "my_worker"
  content = file("my_worker.js")

  r2_bucket_binding {
    name        = "my_bucket"
    bucket_name = "R2MYBUCKET"
  }

}

Link to debug output
/dev/null
Panic output
No response
Expected output
terraform apply should pass and return no errors
Actual output
│ Warning: Argument is deprecated
│
│   with provider["registry.terraform.io/cloudflare/cloudflare"],
│   on main.tf line 32, in provider "cloudflare":
│   32:   account_id = var.CLOUDFLARE_ACCOUNT_ID
│
│ Use resource specific `account_id` attributes instead.
╵

╷
│ Error: error creating worker script: could not read response body: %!w(<nil>)
│
│   with cloudflare_worker_script.my_worker,
│   on main.tf line 367, in resource "cloudflare_worker_script" "my_worker":
│  367: resource "cloudflare_worker_script" "my_worker" {
│
╵

Steps to reproduce
I can produce a full working example but this will take time. Hoping this is sufficient.
Additional factoids
No response
References
No response

please provide the entire TF_DEBUG output and a repro for this to be triaged further.

closing to lack of response. if this is still an issue, please open a new issue filling out all the required fields for maintainers to triage.


decompile and recompile apk without any changes stopped working

Information

Apktool Version (apktool -version) -2.4.0
Operating System (Mac, Linux, Windows) -Windows
APK From? (Playstore, ROM, Other) - Playstore-phone-arm64-v8a-android 6.0

Stacktrace/Logcat
unfortunately stopped error
Include stacktrace here

Steps to Reproduce

apktool
apktool d BHIM.apk
apktool b BHIM
java -jar signapk.jar certificate.pem key.pk8 BHIM.apk BHIM-signed.apk

Frameworks
If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files
(.apks that live in /system/framework or /system/priv-app)
APK
If this APK can be freely shared, please upload/attach a link to it.

Questions to ask before submission

Have you tried apktool d, apktool b without changing anything?yes
If you are trying to install a modified apk, did you resign it?yes
Are you using the latest apktool version?yes

its very interesting without any changes. i am getting error.
i am try many apk file. i am getting this type.
the error comes in apk whos depend size on mobile in google play store.

without any  changes apk running fine. but when i unpack and pack again.
and run it. then i getting error.
if apk have fixed size on google play store then working fine.
i think there are any  framework or other issue. please explain and help me
and test this apps on real mobile. because its need sim card.

It also happened on my device with latest Apktool X 8.2.2
decompiler successfully but when I recompile it it says finished but I can't get the recompiled app of modded apk

and run it. then i getting error.

It would help to know that error via adb logcat or something. Not enough information here to triage. Sorry for delay and closing.


Bugfix and feature add : snapshotpolicy

这个 PR 实现什么功能/修复什么问题:

snapshotpolicy 绑定磁盘数量的显示 bug 修复
添加snapshotpolicy 操作的日志
disk list 接口添加 snapshotpolicy_id(snapshotpolicy, 不仅支持id还支持name)属性，可以通过此过滤disk
公有云的磁盘如果绑定的snapshotpolicy，就会在快照策略应该执行的时间点10分钟后去拉取相应磁盘的snapshot
添加 snapshotpolicy 主动同步cache的接口
私有云下，disk解绑主机的时候，会解绑所有自动快照策略

是否需要 backport 到之前的 release 分支:

release/2.11
release/2.12

Codecov Report

Merging #3127 into master will decrease coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #3127      +/-   ##
=========================================
- Coverage    7.17%   7.16%   -0.02%
=========================================
  Files         478     478
  Lines       76895   76979      +84
=========================================
- Hits         5514    5512       -2
- Misses      70919   71004      +85
- Partials      462     463       +1

Flag
Coverage Δ

#aFlag
7.16% <0%> (-0.02%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/compute/models/disks.go
0.65% <0%> (-0.04%)
:arrow_down:

pkg/compute/models/snapshotpolicydisks.go
0.38% <0%> (-0.01%)
:arrow_down:

pkg/compute/models/snapshotpolicy.go
2.33% <0%> (-0.08%)
:arrow_down:

pkg/util/seclib2/seclib.go
91.48% <0%> (-4.26%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db7880c...33f80a9. Read the comment docs.

/area region

/lgtm

/lgtm

/approve


Fix flakiness of Logging integration tests

Logging integration tests started failing with the following errors:

INVALID_ARGUMENT: Sinks with different output formats cannot share destination. There already is a project sink with V2 output format.

This PR fixes listing and update tests to make them more robust.

@lesv can you have a look please?

LGTM

Coverage remained the same at 85.865% when pulling 8e29feaf1c772838b3eec00abb7437fa24253eb1 on mziccard:fix-logging-its into 23eb620b8c752841a0148d9932964116c7e13463 on GoogleCloudPlatform:master.


feat: Add native lima Docker provider support

The Issue
Colima has gotten quite unstable for lots of people, but it's just a wrapper on Lima, and so we should be able to use Lima directly as another open-source alternative.
How This PR Solves The Issue
It's worth considering whether we should do this at all, as adding more options doesn't necessarily help people or our support environment.
TODO

[x] Docs
[x] Turn on tests (preferably without putting red marks on other PRs)
[ ] Make it pass tests

Manual Testing Instructions
Use the docs to start a lima instance, then do a simple quickstart test.
Automated Testing Overview

Related Issue Link(s)
Release/Deployment Notes

Fails TestDdevAllDatabases with non-writable filesystem:
https://buildkite.com/ddev/macos-lima/builds/12#018e1f6d-091a-4022-8753-f14e26bb1c74


Add tests for ser/de reflected unit structs

Objective

Reflected unit structs fails to deserialize with no field at index 0 on struct <name>

Solution

Test this case.
~Fix it~.

Changelog
Fixed

Deserialization for reflected unit structs.

Fixes https://github.com/bevyengine/bevy/issues/6713.

Duplicate of #6722, I somehow missed it :smile:


Release new crates with cargo-lock version bump

@tarcieri please check that everything works as intended before I cargo publish

Bumped these locally and everything builds and the tests pass 🎉


Support bootstrap icons

Hope st-pages can add bootstrap icons too here

Any efforts on this one? Would also love the implementation of bootstrap icons ;)


Implemented Certify

Issue #: /bcgov/entity#4851
Description of changes:

Import and tie Certify Component to Store

Fixed Unit tests(WIP)

Fixed time conversion error in Date Mixin

Notes:

Will need to Update Date Mixin in Filings/Create UI to correct bug.

A small layout refactor is required for the Shared Component to bring in line with similar components, work is captured in this PR: https://github.com/bcgov/bcrs-shared-components/pull/11

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the bcrs-entities-create-ui license (Apache 2.0).

Looks good to me.
Unit tests?

I'm on it!

LGTM. Merge when you want.

LGTM. Merge when you want.

Copy that. work to cover to the update date display formats across the UI's is captured in ticket:
https://app.zenhub.com/workspaces/entity-5bf2f2164b5806bc2bf60531/issues/bcgov/entity/5425


Quick-start doesn't take effect when have installed other logging components like logging-stack.

As the title described, quick-start doesn't take effect when have installed other logging components like logging-stack.
This is not to say that there is a problem with the quick-start file, but that the name of fluentbit/fluentbitconfig and selectors is the same as logging-stack.
Since the problem occurs when applying or deleting the quick-start to the environment that has installed logging-stack or enabled KubeSphere logging components, it is not recommended to installing both components without pre-processing to the quick-start.
If you want to install both, pls do a pre-processing like follows:
apiVersion: logging.kubesphere.io/v1alpha2
kind: FluentBit
metadata:
  name: fluent-bit-quickstart
  namespace: kubesphere-logging-system
  labels:
    app.kubernetes.io/name: fluent-bit
spec:
  image: kubesphere/fluent-bit:v1.8.3
  fluentBitConfigName: fluent-bit-config-quickstart

---
apiVersion: logging.kubesphere.io/v1alpha2
kind: FluentBitConfig
metadata:
  name: fluent-bit-config-quickstart
  namespace: kubesphere-logging-system
  labels:
    app.kubernetes.io/name: fluent-bit
spec:
  inputSelector:
    matchLabels:
      quickstart.logging.kubesphere.io/enabled: "true"
  outputSelector:
    matchLabels:
      quickstart.logging.kubesphere.io/enabled: "true"
---
apiVersion: logging.kubesphere.io/v1alpha2
kind: Input
metadata:
  name: dummy
  namespace: kubesphere-logging-system
  labels:
    quickstart.logging.kubesphere.io/enabled: "true"
spec:
  dummy:
    tag: my_dummy

---
apiVersion: logging.kubesphere.io/v1alpha2
kind: Output
metadata:
  name: stdout
  namespace: kubesphere-logging-system
  labels:
    quickstart.logging.kubesphere.io/enabled: "true"
spec:
  match: "*"
  stdout: {}

@FeynmanZhou

Thanks, it works.


FinRL_PortfolioAllocation_NeurIPS_2020.ipynb

Hello!
I'm testing this notebook and realized that: each round show much different results in df_actions, even that I use the same data and params. That it, each round I have a portfolio totally different.
This behavior occur with both A2C and PPO.
No problem with DDPG, SAC, and TD3.
Could anyone explain why, please?
Thanks
Notebook: FinRL_PortfolioAllocation_NeurIPS_2020.ipynb

Please refer to this issue

Thanks @Athe-kunal


Unable to do the CD in Pipeline

My pipeline running and deploying the code successfully into the Kubernetes for the first time. When I make some changes and try to deploy, the code is pushed to the ACR but cannot see the changes in the IP Address. I need to un-deploy and deploy every time when I want to make some changes in project. This is the pipeline I am using.
trigger:

master

pool:
name: ubuntu-latest
steps:

task: DotNetCoreCLI@2
displayName: 'dotnet build'

task: DotNetCoreCLI@2
inputs:
command: 'custom'
custom: 'tool'
arguments: 'update -g Microsoft.Tye --version "0.11.0-alpha.22111.1"'

task: KubectlInstaller@0
inputs:
kubectlVersion: 'latest'

task: CmdLine@2
inputs:
script: 'az login -u $(username) -p $(password)'

task: CmdLine@2
inputs:
script: 'az acr login -n Myacr'

task: CmdLine@2
inputs:
script: |
az aks get-credentials --resource-group My-rg --name Myaks

task: CmdLine@2
inputs:
script: 'kubectl apply --validate -f manifest.yaml'

task: CmdLine@2
inputs:
script: 'tye deploy'

Can you please suggest how to do the CD in azure pipeline.

Solution:
Kubectl rollout restart deployment/{pod_name}


fix typo

fix some typo
fix little code err

@ibuildthecloud @galal-hussein please have a review,thanks.

ping @ibuildthecloud @galal-hussein please have a review,thanks.


Covers most cases of required validator

RequiredValidator for https://clever.atlassian.net/browse/DATA-839
Basically implements "field is required" kind of logic

this is looking good!

LGTM, it's still early so I think you should feel very empowered to refactor as you run into friction using this in your next project. I would even consider making a warning in the readme or something to make sure that others know it'll be iterated on rapidly.

I'm not sure what the usage for this looks like, but I wonder if it would be simpler if the "required" and the "allow empty" were two different validators?

I echo @schimmy's comment about putting a warning in the README that the API isn't stable and feeling free to play around with it

Stability warnings: 33fd975

@azylman would you please review the last 3 commits ?

Curious: why do you have separate AllowEmpty fields based on type? Seems like a single AllowEmpty would be good enough, and is simpler. For instance, I can't imagine a use case where you have string data and set AllowEmptyMap to true but not AllowEmptyString.

Seemed flexible to me. Is it worth changing to one?
On Feb 14, 2015 12:23 PM, "Alex Zylman"<EMAIL_ADDRESS>wrote:

Curious: why do you have separate AllowEmpty fields based on type? Seems
like a single AllowEmpty would be good enough, and is simpler. For
instance, I can't imagine a use case where you have string data and set
AllowEmptyMap to true but not AllowEmptyString.
—
Reply to this email directly or view it on GitHub
https://github.com/Clever/go-validation/pull/1#issuecomment-74388362.

It seems to me like we don't have a use-case that needs that flexibility, so I'd favor simplicity instead.

Ok, done: 5aba14d

lgtm

The idea originally was that it could be composable and therefore composite validators would need to return a slice of errors.  eg: EmailValidator might be length + required + regex and would need to aggregate the errors from each.  I've changed the interface to a single error until we need more.
16a86bd


pwsh 7.2.1 $PSStyle Color Rendering Fails on Windows Terminal and ConsoleHost

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
coming  from and issue about  oh-my-posh at Seems oh-my-posh does not work with latest PowerShell core (pwsh)
using  powershell 7.21 in windows terminal
there is an issue with powershell colors, since it works well in vscode terminal but does not work in windows terminal or cmd host
Expected behavior
PS> $PSStyle
`prints scape sequences with colors`

Actual behavior
PS> $PSStyle
`prints scape sequences without colors`

Error details
No response
Environment data
PS> $PSStyle

Visuals
windows terminal output with pwsh 7.2

vscode  terminal output with very same pwsh 7.2 and same profile:

#https://github.com/JanDeDobbeleer/oh-my-posh/issues/1501


Update SPA JS and add orgs example

Description

Update SPA JS
Make sure SPA JS organization docs appear in auth0-react docs
Add an organizations example

References

Thanks for this! Would love to leverage the new redirect method param introduced in https://github.com/auth0/auth0-spa-js/pull/717. When does this get published to npm?

Looks like it went out with https://github.com/auth0/auth0-react/pull/223