EPoSe Movement #1 - Proposing Changes in Block Reward Algorithm

This is the first proposal to change our reward system for Qwertycoin Network.
Purpose: Qwertycoin aims to incentivize/disincentivize the network through a systematic approach by adjusting equations related to block reward calculations.
Current Status:

Basic Block Reward Emission Equation (Abbreviated):
((Total Supply - Previously Emitted Supply) / Emission Factor) - Penalty for Block Size

Proposed Change:

Basic Block Reward Emission Equation (Abbreviated):
(((Total Supply - Previously Emitted Supply) / Emission Factor) - Penalty for Block Size) * ((Timestamp of New Block - Timestamp of Previous Block) / Block Target)

Premises:

The block reward is designed to decrease per each block based on the previous emission.
Since Basic Block Reward Emission is controlled and reduced to a fraction of normal block reward from such attacks, the overall block reward reduction will not be affected by this change.

Implications of This Change:

By reinforcing the consistency of block reward emission over time (instead of block rewards by its sequence in the blockchain), the network hash rate is expected to be supported continuously and find an equilibrium automatically.
Even if an attacker creates a high difficulty for the next block and exit, existing miners are incentivized to receive a huge block reward when they find the next block.
Prevention of attacks with intermittent hash rate jumps must be dealt in a separate issue.

There shall be lower and upper bounds for this timestamp factor. Considering 2 for upper bound and 0 for lower bound.

The difficulty algorithm will be modified that it will help to keep the block time as consistent as possible.

You may find something usable here concerning punishing miners for deviation from T https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a#commitcomment-23983192

I am leaning towards fixing difficulty algo for the efficiency of blockchain instead of controlling the number of blocks:

If there are many pending txs, for an example, under a massive tx attack, diff algo should be lowered than difficulty target to allow those to be included in the blocks and help reduce mempool load.

cannot think no more to night will edit when i can think. sorry.

Joining and synchronizing node can't verify that at the time some block was mined mempool was overcrowded. You can only estimate by taking into account transactions from previous blocks. If transaction count is above median you can lower the difficulty, this way it will be verifiable by the algorithm.

Just like how our wallet cannot make transactions until fully synced, joining and synchronizing nodes cannot perform verification under such stage, or we could make it so.

Joining and synchronizing node can't verify that at the time some block was mined mempool was overcrowded. You can only estimate by taking into account transactions from previous blocks. If transaction count is above median you can lower the difficulty, this way it will be verifiable by the algorithm.

So for this, an index of mempool status at the time of the most recent block can be included in the block header blob like an index figure. Is this what you mean?

Yes, but this is an overhead impossible to agree upon and verify, whereas you already have required data that allow you to estimate the demand for transactions  - their count in previous blocks. Create a median of transaction count and use it to adjust difficulty to speedup blocks so you can include transactions faster. But this duplicates variable block size - if there is demand for transactions block size will grow to accommodate. There's no need to speed up block time.

This difficulty adjustment will only activate when total prnding tx size is beyond max block size. Now when it is below 1mb(current limit)

Is Basic Block Reward Emission Equation (Abbreviated): (((Total Supply - Previously Emitted Supply) / Emission Factor) - Penalty for Block Size) * ((Timestamp of New Block - Timestamp of Previous Block) / Block Target)
the desired new formula for calculating the next block reward?

we just need to multiply ((timestamp of new block - timestamp of previous block)/blocktarget) to existing reward formula.

I propose to create a variable, called Consistency = ((Timestamp of New Block - Timestamp of Previous Block) / Block Target) and set the minimum and maximum range for this variable between 0 and 2.

@speqtr will work on the implementation.

Updated algo implementation is available in PR #99.

In addition to lower and upper limits (0 and 2) for consistency, I propose to add "smart rounding" for all the values within 0..2 range. This way it will make consistency value more predictable. For example:

everything lower than 0.25 should be rounded to 0;
everything between 0.25 and 0.75 should be rounded to 0.5;
everything between 0.75 and 1.0 should be rounded to 1.0;
-etc.

Thanks for the input @speqtr.
I think that consistency is just a reflection of what the entire network is doing, So rounding may not be necessary at this point so let's keep that point open to see how network behaves and decide whether to implement rounding or not.
But the number of decimals for consistency may need to be declared. 3 decimal points would be appropriate at the moment, meaning 4th decimal will be rounded to 3rd decimal points.
We need to see how this equation behaves to our current hashrate pattern before deploying. Can you please check? @nnian

PR #100 with new block reward formula was recently merged into qwertycoin:master.

@nnian I can see that PR #99 was merged first and then instantly reverted (removed from master branch). Do I need to reopen it? Or will you manually re-add changes from PR #99 later?

Maybe you'll open it again and we'll discuss my concerns? With an implementation of a update height. What do you think about?

@nnian later today I'll open new PR with the same changes as in PR #99 so we could add "activation height" for new block reward formula.

Okay, that sounds good. I think we shouldn't change the block reward without the assurance that every user has the same daemon version.

New PR containing updated block reward formula is available. PR #101.

What this change entails...

The above graph shows how block reward emission has been for all PoWs.

EPoSe PoW will change this  like above graph with a maximum cap up to 2X normal rewards for longer solve time but almost zero block rewards in case of any timestamp attacks by posting many blocks in a short period of time.
This is a major step in order to make EPoSe(Egalitarian Proof of Service) PoW fair for all miners and users.
Thanks for the work and I think we can close this issue until we have a problem from this change.

Thanks for the work and I think we can close this issue until we have a problem from this change.

@Xecute0101 Hopefully, we won't have any problems with this change 🙂

This makes miners to forge timestamps as close to target as possible to get max reward. The way CN pools operate they are not ready for this, they normally prepare block template with timestamp once and issue it to miners to hash, therefore every CN block has timestamp like one block behind (close to prev. block real solution time). This is easy to fix in pool software (in modified daemon in fact), instead of current timestamp during block template creation they will use prev block timestamp + target. Some fast and large miner may forge timestamps ahead from real time hitting FTL limit, I don't know what happens then and for other honest miners. What happens if miner forges timestamp ahead and then honest miner's solve time becomes negative?

Let's try to make a small change to the equation.
Previously, the most recent change in the formula is like this.
If consistency is less than equal to 1,
New Basic Block Reward Emission Equation (Abbreviated)
Else if consistency is greater than 1 and if ((difficulty windows * block target) - (total timestamp difference for the duration of difficulty windows)) is greater than 0,
The new addition I would like to suggest is like this.
New Basic Block Reward Emission Equation(the whole equation above) * (Difficulty block window x Block Target(120 seconds)) / (

Zawy, the difficulty algo dev mentioned this in PR #99 that

But make sure out of sequence timestamps are handled correctly:
// Safely handle out-of-sequence timestamps
if ( timestamps[i]  >= previous_timestamp ) {   this_timestamp = timestamps[i];  }
else {  this_timestamp = previous_timestamp+1;   }

Do we have this measure in place? If not, we should. @speqtr @Sw0rdf1sh1 @nnian

I think we have no time to rewrite this PR again and again if we want to have the new reward formula up to network height 400,000.
in 24k blocks would be the change. We still need time to create the new wallets, to distribute them and, if necessary, to provide assistance while every user update to this new wallet in time.
We should think about it.

Version 5.2.0 has been merged with the PR 101 in the master for 18 days now. The new Chinese Exchanges may already use this version. Also some pools have already changed. We have to decide soon. But we don't have time to change this formula just a few blocks before that soft fork date.
We should provide new wallets at least 14 days before we reach height 400k. I don't know what happens when there are different reward formulas available on the network and especially in various pools. We need a network with the same software version.

If there is a potential problem or even some minor changes with this merged calculation formula, I have to postpone and increase the Softfork height, revert version 5.2.0 and we need to provide 5.2.1 version and inform exhanges, pools, node owner and wallet user to update again to 5.2.1. After this we can talk about this formula and change whatever we want.

@nnian , Thanks for your opinion on this improvement.
I suggest we postpone this until block height 500,000. I hope that we can settle issues with all desktop and mobile wallets by block height 500,000 and move on with EPoSe implementation.

Had run a node that activated this at the height of 500,000 and this is the message we get.
2020-Feb-11 19:10:20.936531 INFO
[checkpoints] CHECKPOINT PASSED FOR HEIGHT 500000 432b0c6cfbb5967950efc2f2358f42dc85ddbc5dd270b619cd1f466970a41236
2020-Feb-11 19:10:20.936531 ERROR
[Blockchain] Coinbase transaction spend too much money:<PHONE_NUMBER>3386, block reward is 0.00000000
The gap between height 499,999 and 500,000 was about 67 seconds. So, if we did correct implementation of the formula, the block reward error should show that the reward should be
new reward =<PHONE_NUMBER>3386 * consistency(which is 67sec/120sec) = 75,846.82786557
So it should be showing this error message instead,
2020-Feb-11 19:10:20.936531 ERROR
[Blockchain] Coinbase transaction spend too much money:<PHONE_NUMBER>3386, block reward is 75,846.82786557
The reason being might be that we did not multiply consistency to baseReward, instead we multiplied.
bool Currency::getBlockReward(
uint8_t blockMajorVersion,
size_t medianSize,
size_t currentBlockSize,
uint64_t alreadyGeneratedCoins,
uint64_t fee,
uint64_t &reward,
int64_t &emissionChange,
uint32_t height,
uint64_t blockTarget) const
{
assert(alreadyGeneratedCoins <= m_moneySupply);
assert(m_emissionSpeedFactor > 0 && m_emissionSpeedFactor <= 8 * sizeof(uint64_t));
// Tail emission

uint64_t baseReward = (m_moneySupply - alreadyGeneratedCoins) >> m_emissionSpeedFactor;
if (alreadyGeneratedCoins + CryptoNote::parameters::TAIL_EMISSION_REWARD >= m_moneySupply
    || baseReward < CryptoNote::parameters::TAIL_EMISSION_REWARD) {
    baseReward = CryptoNote::parameters::TAIL_EMISSION_REWARD;
}

size_t blockGrantedFullRewardZone = blockGrantedFullRewardZoneByBlockVersion(blockMajorVersion);
medianSize = std::max(medianSize, blockGrantedFullRewardZone);
if (currentBlockSize > medianSize * UINT64_C(2)) {
    logger(TRACE)
        << "Block cumulative size is too big: "
        << currentBlockSize
        << ", expected less than "
        << medianSize * UINT64_C(2);
    return false;
}

uint64_t penalizedBaseReward = getPenalizedAmount(baseReward, medianSize, currentBlockSize);
uint64_t penalizedFee = fee;
if (blockMajorVersion >= BLOCK_MAJOR_VERSION_2 || cryptonoteCoinVersion() == 1) {
    penalizedFee = getPenalizedAmount(fee, medianSize, currentBlockSize);
}

double consistency = 1.0;
if (height >= CryptoNote::parameters::UPGRADE_HEIGHT_REWARD_SCHEME && difficultyTarget() != 0) {
    // blockTarget is (Timestamp of New Block - Timestamp of Previous Block)
    consistency = blockTarget / difficultyTarget();

    // consistency range is 0..2
    consistency = std::max<double>(consistency, 0.0);
    consistency = std::min<double>(consistency, 2.0);
}

double penalizedReward = static_cast<double>(penalizedBaseReward + penalizedFee);

emissionChange = penalizedBaseReward - (fee - penalizedFee);
reward = static_cast<uint64_t>(penalizedReward * consistency);

return true;

}

Besides this error you probably should fix emissionChange so it takes into account new penalizedReward not penalizedBaseReward.

Perhaps need to cast in consistency = blockTarget / difficultyTarget(); like consistency = (double)blockTarget / (double)difficultyTarget();

@speqtr @exploshot @nnian thought?

Also, the controversial part is going to interfere when we get to tail emission.
assert(alreadyGeneratedCoins <= m_moneySupply);
@speqtr erased and it kinda raise a flag but it was a right decision.

Yes, it should be deleted

where are this penalizedreward and reward variables used? only declared here maybe?

They are used in this function to calculate final reward, but emissionChange is also used elsewhere so it is important that it returns the correct value.

@Xecute0101 @aivve I can implement discussed changes and open a new PR, so we could continue dicussing and testing proposed changes at the same time 🙂

I just realized that you probably are making totally different scheme than I thought, the one that was also suggested by @zawy12 - if the solve time is smaller than target time then reward is proportionally smaller AND vice versa - if solve time is longer, then reward is bigger. @Xecute0101 this then changes the incentives for miners to artificially create as long gaps as they could to maximize the reward. Of course, they will hit the FTL limit how far timestamp can go into the future and you then should really tighten that limit. (I thought that you are going to punish miners for deviation from target solve time in ANY direction so they will be incetivized to keep solve times as close to target as possible).

Yes, we talked about this. The reward gets bigger if they miners mine longer in a sense. Have to look into FTL and Difficulty Algo now.

Just tighten the limits so they can't fake timestamps.

We need to get this calculation right, first I think. If anyone of you is going to test this, change the height in the config and try to see the error msg displays correct rewards.

Also originally doubles were avoided in CryptoNote... Perhaps find a way to calculate this using integer math.

Increasing reward based on solvetime has the same effect as decreasing difficulty.  Miners are only interested in getting the highest reward/difficulty ratio.
Another option  I've been thinking about (that deadalnix and jl777 like) actually depends on on-off mining.  The idea is that your dedicated miners can be identified as being the ones who are still mining when the difficulty is high. Ostensibly, they will not attack the coin and you want them to win in a chain race. They paid a higher price in the past so give them something extra to win chain races.   As an example plan, if difficulty is more than 1 std deviation (16% of blocks) above the 1 day average, then credit the miner's output address with 1/2 of the above-average difficulty. This can be thought of as a 2nd coin that can't be sent to any other address.   If there is a battle between 2 or more tips with at least 3 blocks each, then miners with this coin saved up from the past can use it to artificially lower difficulty to win a chain race.  They can "get back" 1/2 of the excess difficulty they spent on blocks that were in the the upper 16% of difficulties.  It specifically is NOT  meant to motivate the mining of higher difficulty.  We want higher difficulty so that we can identify our friends, and let them win chain races.  A limit might be placed on the amount of "credit" they can get, and the credit might deteriorate over a year.   So if you have 10% of your miners really dedicated, and 16% of your difficulty swings everyday are 20% too high, then that 10% of miners are getting 16% / 2 * 20% = 1.6% of the day's chain work back as credit.  In 100 days they will 160% of a days chain work to fight attacks.

Yes, it should be deleted

Or just in case, commenting out the line would be okay too.

2020-Feb-12 15:50:23.631190 ERROR   Coinbase transaction spend too much money:<PHONE_NUMBER>3386, block reward is 67922.53241693
This changed when
bool Currency::getBlockReward(
    uint8_t blockMajorVersion,
    size_t medianSize,
    size_t currentBlockSize,
    uint64_t alreadyGeneratedCoins,
    uint64_t fee,
    uint64_t &reward,
    int64_t &emissionChange,
    uint32_t height,
    uint64_t blockTarget) const
{
    assert(alreadyGeneratedCoins <= m_moneySupply);
    assert(m_emissionSpeedFactor > 0 && m_emissionSpeedFactor <= 8 * sizeof(uint64_t));

    // Consistency
    double consistency = 1.0;
    if (height >= CryptoNote::parameters::UPGRADE_HEIGHT_REWARD_SCHEME && difficultyTarget() != 0) {
        // blockTarget is (Timestamp of New Block - Timestamp of Previous Block)
        consistency = blockTarget / difficultyTarget();

        // consistency range is 0..2
        consistency = std::max<double>(consistency, 0.0);
        consistency = std::min<double>(consistency, 2.0);
    }

    // Tail emission

    uint64_t baseReward = ((m_moneySupply - alreadyGeneratedCoins) >> m_emissionSpeedFactor)*0.5;
    if (alreadyGeneratedCoins + CryptoNote::parameters::TAIL_EMISSION_REWARD >= m_moneySupply
        || baseReward < CryptoNote::parameters::TAIL_EMISSION_REWARD) {
        baseReward = CryptoNote::parameters::TAIL_EMISSION_REWARD;
    }

    size_t blockGrantedFullRewardZone = blockGrantedFullRewardZoneByBlockVersion(blockMajorVersion);
    medianSize = std::max(medianSize, blockGrantedFullRewardZone);
    if (currentBlockSize > medianSize * UINT64_C(2)) {
        logger(TRACE)
            << "Block cumulative size is too big: "
            << currentBlockSize
            << ", expected less than "
            << medianSize * UINT64_C(2);
        return false;
    }

    uint64_t penalizedBaseReward = getPenalizedAmount(baseReward, medianSize, currentBlockSize);
    uint64_t penalizedFee = fee;
    if (blockMajorVersion >= BLOCK_MAJOR_VERSION_2 || cryptonoteCoinVersion() == 1) {
        penalizedFee = getPenalizedAmount(fee, medianSize, currentBlockSize);
    }

    uint64_t penalizedReward = static_cast<uint64_t>(penalizedBaseReward + penalizedFee);

    emissionChange = penalizedReward - fee;
    reward = static_cast<uint64_t>(penalizedReward);

    return true;

I also tried to multiply consistency instead of 0.5, then the block reward was 0.0000000000. This means that numerator of consistency is either 0 or very close to zero.

2020-Feb-13 07:15:41.977399 ERROR   Coinbase transaction spend too much money:<PHONE_NUMBER>3386, block reward is 75846.82786557
Above happens when following changed.
consistency = (double)blockTarget / (double)difficultyTarget();
uint64_t baseReward = ((m_moneySupply - alreadyGeneratedCoins) >> m_emissionSpeedFactor) * consistency;

The working? code looks like
bool Currency::getBlockReward(
    uint8_t blockMajorVersion,
    size_t medianSize,
    size_t currentBlockSize,
    uint64_t alreadyGeneratedCoins,
    uint64_t fee,
    uint64_t &reward,
    int64_t &emissionChange,
    uint32_t height,
    uint64_t blockTarget) const
{
    // assert(alreadyGeneratedCoins <= m_moneySupply);
    assert(m_emissionSpeedFactor > 0 && m_emissionSpeedFactor <= 8 * sizeof(uint64_t));

    // Consistency

    double consistency = 1.0;
    if (height >= CryptoNote::parameters::UPGRADE_HEIGHT_REWARD_SCHEME && difficultyTarget() != 0) {
        // blockTarget is (Timestamp of New Block - Timestamp of Previous Block)
        consistency = (double) blockTarget / (double) difficultyTarget();

        // consistency range is 0..2
        consistency = std::max<double>(consistency, 0.0);
        consistency = std::min<double>(consistency, 2.0);
    }

    // Tail emission

    uint64_t baseReward = ((m_moneySupply - alreadyGeneratedCoins) >> m_emissionSpeedFactor)*consistency;
    if (alreadyGeneratedCoins + CryptoNote::parameters::TAIL_EMISSION_REWARD >= m_moneySupply
        || baseReward < CryptoNote::parameters::TAIL_EMISSION_REWARD) {
        baseReward = CryptoNote::parameters::TAIL_EMISSION_REWARD;
    }

    size_t blockGrantedFullRewardZone = blockGrantedFullRewardZoneByBlockVersion(blockMajorVersion);
    medianSize = std::max(medianSize, blockGrantedFullRewardZone);
    if (currentBlockSize > medianSize * UINT64_C(2)) {
        logger(TRACE)
            << "Block cumulative size is too big: "
            << currentBlockSize
            << ", expected less than "
            << medianSize * UINT64_C(2);
        return false;
    }

    uint64_t penalizedBaseReward = getPenalizedAmount(baseReward, medianSize, currentBlockSize);
    uint64_t penalizedFee = fee;
    if (blockMajorVersion >= BLOCK_MAJOR_VERSION_2 || cryptonoteCoinVersion() == 1) {
        penalizedFee = getPenalizedAmount(fee, medianSize, currentBlockSize);
    }

    uint64_t penalizedReward = static_cast<uint64_t>(penalizedBaseReward + penalizedFee);

    emissionChange = penalizedReward - fee;
    reward = static_cast<uint64_t>(penalizedReward);

    return true;
}

@nnian @exploshot @speqtr @aivve

I just realized that you probably are making totally different scheme than I thought, the one that was also suggested by @zawy12 - if the solve time is smaller than target time then reward is proportionally smaller AND vice versa - if solve time is longer, then reward is bigger. @Xecute0101 this then changes the incentives for miners to artificially create as long gaps as they could to maximize the reward. Of course, they will hit the FTL limit how far timestamp can go into the future and you then should really tighten that limit. (I thought that you are going to punish miners for deviation from target solve time in ANY direction so they will be incetivized to keep solve times as close to target as possible).

Either set FTL to really low like 1 or 1.05 or just set max consistency to 1.0 will do the trick.

Added #include <cmath> and changed when consistency formula when it is above 1.
bool Currency::getBlockReward(
    uint8_t blockMajorVersion,
    size_t medianSize,
    size_t currentBlockSize,
    uint64_t alreadyGeneratedCoins,
    uint64_t fee,
    uint64_t &reward,
    int64_t &emissionChange,
    uint32_t height,
    uint64_t blockTarget) const
{
    // assert(alreadyGeneratedCoins <= m_moneySupply);
    assert(m_emissionSpeedFactor > 0 && m_emissionSpeedFactor <= 8 * sizeof(uint64_t));

    // Consistency

    double consistency = 1.0;
    double exponent = 0.25;
    if (height >= CryptoNote::parameters::UPGRADE_HEIGHT_REWARD_SCHEME && difficultyTarget() != 0) {
        // blockTarget is (Timestamp of New Block - Timestamp of Previous Block)
        consistency = (double) blockTarget / (double) difficultyTarget();

        // consistency range is 0..2
        if (consistency < 1.0) {
            consistency = std::max<double>(consistency, 0.0);
        }
        else if (consistency > 1.0) {
            consistency = pow(consistency, exponent);
            consistency = std::min<double>(consistency, 2.0);
        }
        else {
            consistency = 1.0;
        }
    }

    // Tail emission

    uint64_t baseReward = ((m_moneySupply - alreadyGeneratedCoins) >> m_emissionSpeedFactor) * consistency;
    if (alreadyGeneratedCoins + CryptoNote::parameters::TAIL_EMISSION_REWARD >= m_moneySupply
        || baseReward < CryptoNote::parameters::TAIL_EMISSION_REWARD) {
        baseReward = CryptoNote::parameters::TAIL_EMISSION_REWARD;
    }

    size_t blockGrantedFullRewardZone = blockGrantedFullRewardZoneByBlockVersion(blockMajorVersion);
    medianSize = std::max(medianSize, blockGrantedFullRewardZone);
    if (currentBlockSize > medianSize * UINT64_C(2)) {
        logger(TRACE)
            << "Block cumulative size is too big: "
            << currentBlockSize
            << ", expected less than "
            << medianSize * UINT64_C(2);
        return false;
    }

    uint64_t penalizedBaseReward = getPenalizedAmount(baseReward, medianSize, currentBlockSize);
    uint64_t penalizedFee = fee;
    if (blockMajorVersion >= BLOCK_MAJOR_VERSION_2 || cryptonoteCoinVersion() == 1) {
        penalizedFee = getPenalizedAmount(fee, medianSize, currentBlockSize);
    }

    emissionChange = penalizedBaseReward - (fee - penalizedFee);
    reward = penalizedBaseReward + penalizedFee;

    return true;
}

double exponent = 0.25;  is same as raising a number to (1/4) power. The impact of this is shown in the attached excel file.
Any extra time over difficulty target(120 sec) will get additional reward but not enough to forge a timestamp beyond real time because it yields less profit than mining a next block less than 120 seconds.

You can find PR from here.

It appears you're make substantial changes to reward if there is a fast solvetime.  The purpose of the solvetime is to prevent orphans due to finding blocks at about the same time.  So you don't want to motivate a narrow range of mining with > 20% changes in difficulty per block, or it indicates you should have been already been using blocks 1/2 the size that come 2x faster.  In other words, if this is right, 120 seconds was always wrong.
More importantly, you don't want to give an advantage to on-off mining over dedicated mining like this, unless you change the required POW every block, which would be a great improvement to security that I discuss here.
The max change per block I would do would be about 20% less reward (or 20% higher difficulty which is the same thing) for having a solvetime of 1 second.  It should be done like the following which is about a 1% change per 6 seconds for T=120, So your FTL needs to be less than 6 seconds. An selfish mine attacker will find that there is no way to manipulate this to increase his gains above simply using honest timestamps.  This formula will not throw off you coin emission rate if hashrate is constant and in most cases even if hashrate is varying. This is a more precise version of what I said July 2019.
// Disallow negative solvetimes by method you posted July 19 above (that I just now saw)
maxFractionChange = 0.2; //  20%
N= 1/maxFractionChange;  // N=5
T=120;
Reward = RewardPerBlock * (1 + currentSolveTime/T/N - 1/N);

I do not know for sure the above will keep coin emission rate and total coin emission correct.  After thinking about how to make sure coin emission is correct, I wrote an article Perfecting Coin Emission Rate.  Plug the RewardPerBlock from that article into the above reward formula.
To prevent negative solvetimes, I did not show the complete code in July 2019 and I did not see your question above a few days later.  Here's the complete code:
previous = timestamps[0];
for ( uint64_t i = 1; i <= N; i++) {  // N is most recent block
      if (timestamps[i] > previous_timestamp  ) {
            this_timestamp = timestamps[i];
      } else {  this_timestamp = previous_timestamp+1 ;   }
      solvetimes[i] = this_timestamp - previous_timestamp;
      previous_timestamp = this_timestamp;
}
// do algorithm with the individual solvetimes stored in solvetimes vector.

EPoW reward algorithm is entirely different from the conventional reward algorithm as you can see in the graph. This is not promoting miners to mine quicker at all. The purpose of solvetime is not to create orphans. I get that but this is not about the solve time. This is something difficulty algorithm should react to not mining rewards.
Miners simply don't have any incentives to influence thereby fluctuate > 20% difficulty at all under 120 seconds. More so after 120 seconds. They just need to mine with their honest hash rate to get the maximum benefit.
Regarding on-off mining, that is like holding a lot of hash rate and dumping it at a specific instance when miners get the best reward/difficulty ratio or rewards/mining. The current system is also susceptible as well as after EPoW algorithm. That is another important subject that needs to be dealt with besides this update.
Negative solvetime will yield zero so that is a miners' dumb move because it give consistency a value of zero. nada for blockReward.

previous = timestamps[0];
for ( uint64_t i = 1; i <= N; i++) {  // N is most recent block
      if (timestamps[i] > previous_timestamp  ) {
            this_timestamp = timestamps[i];
      } else {  this_timestamp = previous_timestamp+1 ;   }
      solvetimes[i] = this_timestamp - previous_timestamp;
      previous_timestamp = this_timestamp;
}
// do algorithm with the individual solvetimes stored in solvetimes vector.

Shouldn't this be included in the block validation portion of the code?

Yes, it would be better to simply force sequential timestamps instead of using that code.
My post shows I understand what you're doing.   A >51% attacker will do a selfish mine and set every timestamp equal to target solvetime. This will make his blocks < 1/2 the cost of your dedicated miners who are more likely to get fast solvetimes due to the exponential distribution.

The only way I see the proposed formula is that the  relaxation of blockReward using
Reward = RewardPerBlock * (1 + currentSolveTime/T/N - 1/N);
give more incentives (within the initial window of 120 seconds) for larger miners get larger block rewards by doing on-off mining or doing normal mining with higher probability of winning block rewards for less work. Isn't this true?
I was going to write about the 51% attack but it was another subject so I erased it and I really resnt doing so. We know that the only way to reduce the risk of 51% attack is to reduce the difficultyTarget to a lower number when blockReward is fixed per block. At 1 second time, 9 to 1 hash distribution still gives the minority to win at 11%.

To explain my equation, from 1 second to 2x T seconds, reward goes from 80% to 120% of the baseline reward. You get the baseline reward for giving a block a timestamp (solvetime) of T=120. I don't really understand your questions or points.  My point is that it can be done, but it strongly motivates large miners to do a selfish mine.  If they can't change coins in about 30 seconds, it sort of requires them to do a selfish mine.  Even my 20% is a large motivation to do a selfish mine.  There's not a good solution except to suffer as usual or become a KMD coin.

@zawy12, I fully understand the impact of the equation and but was not agreeing on how it is better and its implications on changes in miners' behaviorrs. I do appreciate sharing your thoughts and opinions. I am glad that we have different opinions in a way.

I can't follow what you're saying except that you think it's not going to be a problem. I'm saying it's a risk.  I'm not sure I've said anything that is an opinion.
The exponential distribution means 63% of blocks are found less than T.  50% are less than 0.69xT.
An integral I just did indicates those 63% of blocks will get an average reward of 0.26 of the baseline.  The rest of the solves will take substantially so that the average solvetime will be T.   A 51% miner doing a selfish mine will also find a lot of fast blocks and a few really long-solvetime blocks.  He only needs to set timestamps for every solvetime equal to T so he sacrifices the few large rewards to not suffer all the large discounts in reward that he would have gotten like the other miners, so he gets at least 3x what your dedicated miners will be getting.  So instead of getting 2x profit from normal 50% selfish mining attack that gets 100% of the blocks, he'll get about 3x to 4x more profit than honest miners.

I created a model based on your description. Take a look at it first. EPoW Worksheet R1.xlsx
In the sheet RS 0-240 63% Modeling, you will see that reward penalty from your equation is not stringent enough over time.
Let's not talk about selfish mining because their rewards will be substantially less for doing so.
If they are after taking the maximum block reward at 120 seconds, honest miners have more additional time to solve the block before them unlike how it is done now.
Power of 1/2 seems to yield the closest penalty for the reduction % in mining time for larger miners.

If they are after taking the maximum block reward at 120 seconds, honest miners have more additional time to solve the block before them unlike how it is done now.

On average, your normal miners will solve blocks at 120 s if there is no on-off mining.  If there is on-off mining to get the higher reward, there will not be many long solvetimes to bring the average down to offset the fast solvetimes,  so the dedicated miners will have to suffer a larger difficulty for the < 120 solves in order to keep the avg 120 s. A > 50% will look at this and say "there's no way I'm going to accept 1/2 or 1/3 reward by giving fast solvetimes" and he will set all his timestamps to 119 seconds to beat out both the dedicated miners and the on-off miners in a secret aka private aka selfish mine for as many blocks as he wants.   If there is no on-off mining then the private mine will only give the normal 100% benefit no matter what his timestamps, as long as they are not too far into the future and you have a tight FTL.
Typically, like your coin in the past, the on-off miners will come in at >3x and sometimes 40x your baseline hashrate if your difficulty is 5% to 20% below average, and stop when it is 5% to 20% above average. See your chart below from January 2019.  It sounds like you have a much different assumption about miner motivation. This is why I suggest a "small" +/- 20% change.  The 20% is a large value for me because it is pushing the limits of what I consider safe, trying to prevent encouraging miners from doing the on-off mining which will in turn encourage constant private mining attacks.

Then now, I will assess two more models along with decision trees.
One with optimized on-off mining and one without it. I will post an update soon.
Assumption 1)
A > 50% will look at this and say "there's no way I'm going to accept 1/2 or 1/3 reward by giving fast solvetimes" and he will set all his timestamps to 119 seconds to beat out both the dedicated miners and the on-off miners in a secret aka private aka selfish mine for as many blocks as he wants.
Assumption 2)
If there is no on-off mining then the private mine will only give the normal 100% benefit no matter what his timestamps, as long as they are not too far into the future and you have a tight FTL.

My assumptions are the followings:
Large Miner Behaviour #1 a-1) 100% Honest Mining (Not Realistic)
r1a-1) Receives proportional rewards based on actual mining time / design mining time.
r2a-1) Difficulty algorithm operates within its equilibrium range.
Large Miner Behaviour #2 a-2) Secret, Selfish, Private Mining (Realistic)
r1a-2) Since rewards is proportional to reported solvetime (block timestamp) up to 120 seconds from previous block timestamp, mining many short solvetime blocks will only increase difficulty without profits as was in CryptoNote block reward algorithm.
r2a-2) A miner wants to maximize rewards all the time, therefore, post the optimum timestamp = previous timestamp + 120 seconds though the block was solved before the posted timestamp.
r3a-2) Posting timestamp = previous timestamp + 120 seconds allows large portion of honest miners to take advantage of additional solvetime until previous timestamp + 120 seconds.
r4a-2) Win/Loss ratio within 120 seconds timestamp period will be determined by the distribution of hash rate between participating parties.
r5a-2) So the miner considers doing on-off mining with 120 seconds timestamp.
r6a-2) The best on-off mining timing and duration will be determined by the private or secrete hash rate.
r7a-2) The on-off mining practice shall not increase difficulty too much.

"Assumption 1)" describes a security hole. It may not happen if the coin can't be easily sold on an exchange. Assuming a security hole will be exploited is the only correct assumption.
"Assumption 2)" is a mathematical fact (except it's too optimistic, see below). Your "Zawy" column seems to conclude the same thing, except an attacker will mine with>50%, getting the 119 s stamps quickly (say, with 60% or 80% HR), but he will hold them until the future time arrives.  If some other chain has more work by that time (they accidentally solved their blocks faster than his timestamps at 119 s), then he has to come back and mine a few more blocks with low reward to pull back ahead.
 If the difficulty and hashrate are stable, all of them have the same solvetime which is the time column.
I forgot that your method is not "symmetrical" so your rewards are showing around 20% below CN rewards. Due to this my Assumption 2 is 20% too optimistic. A selfish mine will get 120% of coin instead of the 50% if he was an honest miner. My algorithm is symmetrical which is why the rewards are the same as CN. There's no additional 20% gain from selfish mining because it has the correct kind of symmetry. You're having to do a lot of modelling because of the lack of symmetry and you're looking for an optimal tradeoff based on intuition. My equation is the optimal one.

@zawy12, it was a nice discussion and thanks for the valuable input. I hope other coins agree with your idea and maybe implement the upgrades later on.
By the way, there is nothing incorrect about the excel sheet.

Update for EPoW is complete for block version 6 update. Thanks for all your efforts guys.


Support Application Default Credentials

support for Application default credentials
https://cloud.google.com/docs/authentication/application-default-credentials
It may be useful to support ADC, especially if the client code will run inside GCP environments.

Add this as a 3rd option for authentication in client.rb
Add details to the README.md
Add a devcontainer.json file to make development in vscode devcontainers easier

@niciliketo, thank you for your contribution! It's definitely helpful.
I'm reading the documentation you shared and taking a look at the PR.

That looks great.
I took the liberty of changing a few things before merging:

Rearranged the README to include extra details about this authentication method at the top of your contributions.
Removed scopes from get_application_default (I tested, and apparently we don't need them).
Added .devcontainer/devcontainer.json to .gitignore.
Applied RuboCop corrections.

Hope that's okay; feel free to open another PR if you prefer to change something.
Thank you very much for your contribution. I will issue a new Gem version throughout the day.

New version released and available: https://github.com/gbaptista/gemini-ai/releases/tag/v2.1.0


Monkey patch for reset_raid_config,clear_foreign_config and is_idrac_ready best practice

Created monkey patch of reset_raid_config, clear_foreign_config (RAID deletion enhancement) and is_ready best practice (Checking LCStatus instead of MessageID) for dracclient  client.py and raid.py .

Looks good!


fix: do not wait to confirm unsupervised S2 delivery while bootstrapping

The changes to confirm S2 delivery for unsupervised SET-type commands in #5475 accidentally caused us to wait during S2 bootstrapping as well, resulting in us missing requests from the node.
This fixes it.

@zwave-js-bot automerge


boilerplate-update

Keeping up to date with boilerplate as a result of the changes in OSD-13790

Depends on https://github.com/openshift/release/pull/34127

/retest

/retest

/retest

Per @mjlshen changes have been validated and tested in commercial, changes should not impact the functionality of the operator
/lgtm


Feature Request: Ability to edit the Title and Description Text when adding new entries

Currently you can either use the info that is prepopulated, or write your own from scratch, there is no ability to edit the prepopulated text.
The majority of the time I just want to make a slight change to the info that is grabbed from the page.

Added on v1.5.0.


Add setundef -autoports

Some flows require all ports to be set to defined values, even those missing from instantiations. setundef on its own isn't enough to deal with this as the ports won't exist at all if not included in the instantiation, this adds a new mode to setundef to deal with this.

This seems to be another iteration of the same problem as #2390, where we decided to solve it by using default values on the ports in cell library instead; what do you think about that solution?  I'd rather avoid doing it in inconcistent ways between targets.

Yes, that approach seems reasonable. I'll close this PR as in the short term we ended up working around this in nextpnr anyway, in order to deal with cases of cells created during PnR too.


Single line comments aren't greyed out outside of a function

I'm not quite sure why but single line comments outside of a function don't get greyed out, as in they are not recognized as comments
 Card [] testingSet; // THESE COMMENTS ARE NOT GRAYED OUT
 Card [] trainingSet; //  THESE COMMENTS ARE NOT GRAYED OUT

 void loadData() { // In this function we initialise all out data in two seperate arrays, training[] and test[]
 }

Hi @arashout , this seems to be a problem with your particular color theme, as it works fine for me. For example, switch to any of the built-in sublime color themes, and you'll see that comments are styled correctly.


Clients should be able to retrieve our ip-address through an api endpoint

closed with 7c3d0ed15c7581a2a73b883cec69ed5d1d4f6bab (intellij code with me)


Support for new features in devfile 2.1

/kind feature
Goal
Feature #4589 will be used to adopt the devfile 2.1 library, react to any breaking changes, and do any urgent new features in order to support devfiles created with 2.1 that are almost equivalent to current 2.0 devfiles. This feature will be used to add support for all other new features and complete 2.1.
The Devfile 2.1 spec (https://github.com/devfile/api/milestone/4) has the following changes:
New support

Top level attribute def (issue 239) - will be handled via feature #4589
Extend Volume components to support more features (issue 310)
Add a link to an external website (issue 38)
Better support of resources settings (issue 131)
Add attributes free-form map field for tool-specific fields in a number of places (issue 169)

Cleanup

Remove sparseCheckoutDir (issue 128)
What are valid name values? (issue 201)

Dev workspace related

Commands scoped to a project (issue 165)
Root folders for projects (issue 385)
Remove devfile.projects[].github (issue 340)

Not applicable/internal fixes, etc

Extending DevWorkspace Status fields (issue 370)
Remove plugins from devfile schema (issue 407)
Provide and publish types for different languages (issue 353)
Make vscodeLaunch / vscodeTask more generic (issue 344)
Parse plugin component and add component, command and events to the main devfile obj (issue 240)
Attributes overriding fails when overriding for non primitive attributes (arrays or objects) (issue 263)
The metadata filed should allow additionalproperty to be backward compatible (issue 236)
Define a way to contribute environment variables to all the workspace POD containers (issue 149)
Define the devfile 2.0 equivalent of the Devfile 1.0 chePlugin preferences (issue 170)

Based on the categorization above, 'Cleanup' and 'Not applicable' should not affect odo, so there are 5 issues for new support, and 3 dev workspace related-issues.
Proposal
Use this feature to explicitly reject 'workspace' devfiles that contain features odo cannot easily handle. If commands are scoped to a project or contain project root folders, odo will output a message like "This devfile contains support for multiple projects (a workspace) and is not supported by odo".
We can always discuss how odo might support some of these features later, but we can handle as a separate feature and this avoids odd user errors in the meantime. Doing this makes it clear that a dev workspace is not currently supported in odo.
Acceptance Criteria

[x] Review Devfile 2.1 plan to confirm no additional items and categorization above (https://github.com/devfile/api/milestone/4)

New support

[ ] Extend Volume components to support more features (issue 310) - Devfiles containing persistent and external types must work  (created #5131 to track this)
[ ] Better support of resources settings (issue 131) - Resource limits should be applied such that applications that exceed these limits fail. (tracked in  #5132)

Dev workspace

[x] Using a devfile with project-scoped commands should give an error. (this was eventually not added to devfile https://github.com/devfile/api/issues/165#issuecomment-823302158)
[x] Using a devfile with root project folders should give an error. (roots were not included in devfile https://github.com/devfile/api/issues/385)

Optional / stretch

[ ] Add attributes free-form map field for tool-specific fields in a number of places (issue 169) - Confirm if there are any reasons why odo would need to use tool-specific info.
[ ] Add a link to an external website (issue 38) - Provide a command that opens a browser on the embedded website url.

/remove-lifecycle stale
/lifecycle frozen

/assign @kadel

[ ] Add attributes free-form map field for tool-specific fields in a number of places (issue 169) - Confirm if there are any reasons why odo would need to use tool-specific info.

odo levareges tools-specif attributes for selective push this feature is documented at https://github.com/openshift/odo/blob/main/docs/public/using-devfile-odo.dev.push.path-attribute.adoc

All items in acceptance criteria checked. Closing this issue.
/close


Clean-up some noisy logging

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

Thanks @chidozieononiwu for finishing up the sync process here.


feat: add script to make full nextclade parallel run

Description of proposed changes
This PR adds script ./bin/run-nextclade-full and a set of helper scripts, to trigger full run of Nextclade: all available sequences will be processed again. The goal here is to automate, at least partially, the tedious task of performing these full runs.
The changes in this PR should not affect the normal daily flow and are just helpers for the full reruns that are performed occasionally.
Previously
In the routine daily ingests only the new sequences get processed by Nextclade. Clade results (and other metrics) from previous runs are cached in nextclade.tsv file. Upon rerun, the sequences in sequences.fasta and nextclade.tsv files are compared and only sequences that are missing in nextclade.tsv undergo analysis with Nextclade. The results are then appended to nextclade.tsv. Then nextclade.tsv gets merged with metadata.tsv and that's how clades and some of the other Nextclade metrics appear in the medatata file.
However, the cache needs to be invalidated when

clades names or definitions change
we want to upgrade Nextclade to a new version that has breaking changes
we want to change what columns from nextclade.tsv we add to metadata.tsv (e.g. we added aminoacid mutations and frame shifts recently)

It takes a while to run it, and back in the day, when ingest ran purely on GitHub Actions, @rneher was running these full runs on an external cluster. It's a tedious task, so my goal here is to automate the flow, at least partially. Especially now that ingest is on AWS.
In this PR
The following scripts are introduced:

./bin/run-nextclade-full - runs locally (not feasible computationally for most people). For development and debugging. The script accepts --database=gisaid|genbank flag to set the input and output sources on S3. It downloads currently available sequence.fasta (from daily workflow). It splits this file into batches and runs several instances of Nextclade in parallel on a single batch to achieve better resource utilization. The batch size and number of threads per Nextclade instance is configurable. Nextclade instances are allocated a certain number of threads and they wait until previous instances finish to not cause too much oversubscription. A small overlap is given to oversubscribe slightly for better resource use.

./bin/run-nextclade-full-docker - runs ./bin/run-nextclade-full in a Docker container (still not feasible computationally for most people). For more convenient development and debugging.

./bin/run-nextclade-full-aws - schedules ./bin/run-nextclade-full on AWS Batch. There are additional flags to set number of requested CPUs and memory from AWS Batch.

./bin/split-fasta - brought back from the olden days: 9bce646534ac436a414d31ba7126135a446bf0c4. It was removed in 94b0372d0e9afbe2c7668c5af92f034f541d98a5 when Nextclade became multithreaded. This scrip is what splits sequences into batches.

Intended usage
When it's decided that a full run is needed, a dedicated person should run:
cd ncov-ingest
./bin/run-nextclade-full-aws --database=gisaid
./bin/run-nextclade-full-aws --database=genbank

(making sure AWS credentials are available for the script)
The resulting nextclade.tsv.gz should be then available in the subdirectory nextclade-full-run in the usual S3 location for the particular database:
s3://nextstrain-ncov-private/nextclade-full-run/nextclade.tsv.gz
s3://nextstrain-data/files/ncov/open/nextclade-full-run/nextclade.tsv.gz

The nextclade.tsv.gz should be manually inspected for scientific correctness, comparing to the old one, if necessary. If all went well, after agreeing with ingest team, the entire directory: nextclade.tsv.gz and also aligned.fasta.xz should be copied to the location where the daily ingest can find it, overwriting the old one. These locations are:
s3://nextstrain-ncov-private/nextclade.tsv.gz
s3://nextstrain-data/files/ncov/open/nextclade.tsv.gz

(just omit the subdirectory nextclade-full-run)
The next ingest will then include the freshly computed clades. We might try to automate this step as well. However, given that manual inspection is needed, the automation options are limited.
Related issue(s)
None
Testing
The examples of AWS Batch jobs:
GISAID:
nextstrain build --aws-batch --no-download --attach 0591961b-fb72-4f18-b55f-38e8f8c9f20d .

GenBank:
nextstrain build --aws-batch --no-download --attach 20698f2c-69f1-4302-8df0-3b300ea69411 .

We need to make sure the workflow for full reruns is convenient and the results are correct (that is that the full run actually happens and clades get recomputed).

Quick note for what to do after full run is triggered:
The nextclade.tsv.gz should be manually inspected for scientific correctness, comparing to the old one, if necessary.
If all went well, after agreeing with ingest team, the entire directory (nextclade.tsv.gz and also aligned.fasta.xz) should be copied to the location where the daily ingest can find it, overwriting the old one. These locations are:
s3://nextstrain-ncov-private/nextclade.tsv.gz
s3://nextstrain-data/files/ncov/open/nextclade.tsv.gz

(just omit the subdirectory nextclade-full-run)
It's important to copy aligned.fasta.xz as well because otherwise there will be duplicated sequences (see #281)
Make sure you copy when ncov-ingest is not running (you can check in Slack). Good time to copy is 1-5pm Basel time.


Proper Documentation to dimscord library

Hello,Firstly a huge thanks for writing such a wonderful discord module for nim.
Secondly as the title suggests .a proper beginners documentation would be helpful to anyone who wants to use the library.there is a documentation at https://krisppurg.github.io/dimscord/dimscord.html but it isnt very beginners friendly.

Hm, what parts of the documentation are not beginner friendly? There is also a Dimscord Wiki, but I think I might as well update that.

Thanks for the quick reply.what i mean is ,i also started programming in nim not too long ago ,from what i see this module is also a asynchronous module like pythons discord.py.i think you should start with a installation part ,followed by a quickstart part and then you can explain all the procedures afterwards .or maybe it might be me being used to reading python style rst format docs.also it would be a huge help if you could compare it to pythons discord.py module,i mean a minimal example for both programming language to help python programmers start with this module.
regards,

Well, there is an installation guide in the README. The procedures one is self explanatory. So:
For restapi:

api Of course is the RestApi object. In which can be found on discord.api
Some parameters have none ... in them, so: none ChannelType, is basically just Option[ChannelType], except it defaults to be none so it wouldn't force you to provide.
reason is audit log reason 0..512 characters.
As you may notice some procedures have comments, these are usually for notes and additional information.

For gateway:

Again contains comments for additional information.

For objects:

The only thing you might find is probably the Events objects, which I might actually provide more information on

Each field in Events, is just basically

proc events_field_name(params...) {.event(discord).} =
  discard # by default

i see,greatly appreciated !

So is there going to be new docs?

So is there going to be new docs?

hmm, I wouldn't rewrite documentation but I would prefer to make some adjustments and clarifications on some certain methods and procedures (e.g. restapi methods). I could make a document or some special page to help guide and provide some other extra info along with some FAQs, that way it's a lot more easier. I'll give it some thought for the time being.


At least one CSS reset is not available in production

I noticed that sub and sup CSS resets are not available in production, but works perfectly in development. But I can confirm some reset can be seen in production.
I can gather more details if necessary. I would copy and paste the style in head but it's empty. But working parts work somehow.

fixed in #199

Thanks a lot. I'll try this as soon as there's a release with it.

Is there a release coming, @threepointone?

hey, haven't touched this in a while. gimme a day, I'll cut a release.

Can I do anything to help?

just pubbed 2.20.23 with whatever was in master, lemme know if it works?

Seems to be working. Thanks!

Cheers, sorry for the delay, my bad.
On Tue, 14 Feb 2017 at 12:43 PM, Fatih Altinok<EMAIL_ADDRESS>wrote:

Seems to be working. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/threepointone/glamor/issues/192#issuecomment-279697688,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJeNG1aBT8cxjGSUsoYjRRjdZJJ2yCks5rcaGHgaJpZM4Lq_nl
.

--
π


copy go.mod and go.sum for the whitesource scanner

@heidemn-faro Please review and merge, when ready


how does SingularityS3Downloader "avoid use page cache" works

Hi,
I seen your doc "how it works", and learn that the SingularityS3Downloader can avoid use page cache, and I look at the code, you are using cmd call wget download file. Then I used it too, but I still got page cache oom error, so can you tell more
Thanks!

The downloader 'avoiding the page cache' is more simply put that it is using memory outside of the container trying to be launched. In other words, your task won't oom from a mishandled download, since the memory being used is in a shared pool on the host, not the memory of the executor process which is inside the container (and therefor limited by cgroups/etc). It also allows you to inject S3 access/secret keys more easily, allowing the downloader to hold master keys as opposed to generating expiring urls for downloads


Hi~Dear Dr, a little request @~@

Hello, I feel that this thing you are doing now has a great fundamental contribution to multi-UAV path planning. I have also built the multi_agent_pkgs environment based on my own interests, and the result is that you can see the static UAV and environment. However, the dynamic implementation steps of your UAV operation or path planning after obstacle avoidance have not been sent yet. Could you please send them to me? I want to run it and get a feel for the whole process of virtual simulation.

Can you please explain further what you mean by the dynamic implementation?


Please include flow for signout of Okta in the sample

As per this  SO question, how can we signout of Okta upon signing out of the client application?

This is handled client side in SignOut.cshtml.
However, as noted in this answer, RP-initiated logout is coming!

For the benefit of other users - since the client side signout is done via cross-origin ajax, it is also necessary to add the client app to the trusted origins under Admin > Security > API > Trusted Origins


Fix some low-hanging fruit

This PR fixes bunch of cosmetic and easy-to-fix issues that prevents the usage of gfakluge as a library.

Thanks @asl ! I don't see any trouble with this after review, so I'm going to go ahead and merge it.

Also, if you have any other issues preventing the use of gfakluge, please let me know. I'd be eager to fix them.


balancer config name

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Other information

Codecov Report
Attention: 26 lines in your changes are missing coverage. Please review.

Comparison is base (4e5ae05) 66.69% compared to head (d485a07) 66.73%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #890      +/-   ##
==========================================
+ Coverage   66.69%   66.73%   +0.03%
==========================================
  Files         257      258       +1
  Lines       24276    24319      +43
==========================================
+ Hits        16191    16229      +38
- Misses       7242     7251       +9
+ Partials      843      839       -4

Flag
Coverage Δ

53.36% <33.82%> (+0.05%)
:arrow_up:

go-1.17.x
65.27% <61.76%> (+0.02%)
:arrow_up:

go-1.20.x
66.47% <61.76%> (-0.14%)
:arrow_down:

go-1.21.x
38.94% <50.00%> (-27.64%)
:arrow_down:

integration
53.36% <33.82%> (+0.05%)
:arrow_up:

macOS
38.91% <50.00%> (-0.04%)
:arrow_down:

ubuntu
38.91% <50.00%> (-0.07%)
:arrow_down:

unit
39.00% <50.00%> (-0.07%)
:arrow_down:

windows
38.58% <50.00%> (-0.46%)
:arrow_down:

ydb-22.5
52.63% <33.82%> (-0.32%)
:arrow_down:

ydb-23.1
52.74% <33.82%> (-0.35%)
:arrow_down:

ydb-23.2
53.05% <33.82%> (-0.07%)
:arrow_down:

ydb-23.3
52.99% <33.82%> (-0.08%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

balancers/config.go
95.94% <100.00%> (ø)

internal/balancer/balancer.go
82.70% <100.00%> (ø)

internal/balancer/connections_state.go
96.93% <100.00%> (ø)

internal/balancer/config/routerconfig.go
81.81% <81.81%> (ø)

balancers/balancers.go
55.88% <40.54%> (-20.72%)
:arrow_down:

... and 9 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Cannot get items from grid when upgrading from jQuery 2.2.4 to 3.1.0

Hi,
Thanks for an excellent grid plugin! :-)
I recently upgraded jQuery from version 2.2.4 to 3.1.0. After the upgrade I cannot get items from the grid. The Grid('option', data) method simply returns an empty array!??
var items = $('#grid').jsGrid('option', 'data');

Are you sure it worked before the update?

Sorry , my mistake ... I've solved it! ;-) I assume that the items shouldn't be 'available' until loadData has completed ... But when using jQuery 2.2.4 the following code seem to work in my case!?
      controller: {
        loadData: function(filter) {
          var d = $.Deferred();
          $.ajax({
            type: 'GET',
            url: '/Home/UsersPage',
            data: {
              groupName: group
              pageSize: filter.pageSize,
              pageIndex: filter.pageIndex,
            },
            dataType: 'json',
          }).done(function(result) {
            d.resolve(result);

            ... ... ...
            var items = $("#grid").jsGrid("option", "data");
            ... ... ...

          }).fail(function() {
            d.reject();
          });
          return d.promise();
        },
      },

When I upgrade to jQuery 3.1.0 the code above stopped working. I moved Grid('option', data) method to the onDataLoaded event which solved the issue.
      onDataLoaded: function(args) {
         ... ... ...
          var items = $('#grid).jsGrid('option', 'data');
          ... ... ...
      }

There should not be difference between 2.4 and 3.1 in this case. Not sure, maybe it was synchronous, and now you made it asynchronous.
Anyway, glad that you figured it out.
BTW you can get data from args of the callback http://js-grid.com/docs/#ondataloaded


runserver_plus does not work with werkzeug 2.1.0

Werkzeug just released version 2.1.0 on March 28, 2022 (4-5 hours ago as of writing this).  When using runserver_plus with that version of werkzeug, the development server cannot be started.
As part of werkzeug 2.1.0, some previously deprecated code was removed, including werkzeug.server.shutdown
From the werkzeug release notes:  "Remove the non-standard shutdown function from the WSGI environ when running the development server. See the docs for alternatives."
In django-extensions runserver_plus.py - line 326 we have this:
del environ['werkzeug.server.shutdown']
which results in a KeyError since shutdown was removed
Rolling back to werkzeug <2.1 (2.0.3) resolves the issue.
This is the first time I've looked into code for runserver_plus -- but would be happy to provide a PR if anyone with more familiarity can provide some rough guidance.
Also, while making this change, perhaps the preceding line is from runserver_plus (if not options['keep_meta_shutdown_func']:) is also no longer needed as well...?

I just commented out the two lines in runserver_plus and so far the server is working fine in debug mode.

I just commented out the two lines in runserver_plus and so far the server is working fine in debug mode.

Same here, lines 325 and 326. No problems after a few hours of heavy Django testing.

This is affecting me too, namely because of:
  File "/path/to/lib/python3.9/site-packages/django_extensions/management/commands/runserver_plus.py", line 326, in make_environ
    del environ['werkzeug.server.shutdown']
KeyError: 'werkzeug.server.shutdown'

This is affecting me too, namely because of:
  File "/path/to/lib/python3.9/site-packages/django_extensions/management/commands/runserver_plus.py", line 326, in make_environ
    del environ['werkzeug.server.shutdown']
KeyError: 'werkzeug.server.shutdown'

If you open runserver_plus.py and comment out lines 325 and 326, the problem should go away.

So a
if 'werkzeug.server.shutdown' in environ:
    del environ['werkzeug.server.shutdown']

should fix this for all versions ?

There is actually a workaround for this that doesn't involve locally editing runserver_plus.py (or creating a fork): the --keep-meta-shutdown option seems to prevent exactly this behaviour (wanting to delete that key from environ). So you can just add that to the runserver_plus command line, until it gets fixed.


adds the GlobalDescriptorTable::pop() function.

It is not currently possible to load separate TSS's for APs using the x86_64 GDT structure (segment entries cannot be edited).
This PR enables that.

Thank you for your contribution! May I ask why you closed this?

Thank you for your contribution! May I ask why you closed this?

It occurred to me sometime later that this approach seems a bit better:
        let tss_ptr = alloc_tss_somehow();

        use x86_64::VirtAddr;
        tss_ptr.as_mut().unwrap().interrupt_stack_table[0] =
            VirtAddr::from_ptr(alloc_stack_somehow());
        tss_ptr.as_mut().unwrap().interrupt_stack_table
            [crate::tables::idt::DOUBLE_FAULT_IST_INDEX as usize] =
            VirtAddr::from_ptr(alloc_stack_somehow());

        let tss_descriptor = {
            use bit_field::BitField;

            let tss_ptr_u64 = tss_ptr as u64;

            let mut low = x86_64::structures::gdt::DescriptorFlags::PRESENT.bits();
            low.set_bits(16..40, tss_ptr_u64.get_bits(0..24));
            low.set_bits(56..64, tss_ptr_u64.get_bits(24..32));
            low.set_bits(0..16, (core::mem::size_of::<TaskStateSegment>() - 1) as u64);
            low.set_bits(40..44, 0b1001);

            let mut high = 0;
            high.set_bits(0..32, tss_ptr_u64.get_bits(32..64));

            Descriptor::SystemSegment(low, high)
        };

        let cur_gdt = tables::sgdt();
        let mut temp_gdt = GlobalDescriptorTable::new();
        temp_gdt.add_entry(Descriptor::kernel_code_segment());
        temp_gdt.add_entry(Descriptor::kernel_data_segment());
        let tss_selector = temp_gdt.add_entry(tss_descriptor);
        temp_gdt.load_unsafe();

        // Load TSS from temporary GDT.
        tables::load_tss(tss_selector);
        // Restore cached GDT.
        tables::lgdt(&cur_gdt);

This approach manages to keep everything core-local, and only requires locking on the global allocator (which is ideally optimized for concurrent access). While pushing and popping from the GDT is certainly an option, it does seem like keeping the GDT as an immutable state object is a better approach.

This approach manages to keep everything core-local, and only requires locking on the global allocator (which is ideally optimized for concurrent access).

Good idea, that looks like a very good solution!

While pushing and popping from the GDT is certainly an option, it does seem like keeping the GDT as an immutable state object is a better approach.
If the function is appreciated regardless of that, I can (?) reopen the PR and fix the integration issues.

I think we should keep the GDT append-only for now. The GDT type almost follows the builder pattern and removing doesn't really fit into that.
Removing entries from the GDT only makes sense when that GDT has already been loaded at some point in the past because otherwise one could have just not added those entries. While this could work fine, this seems prone to UB, so I don't think we should have such an API unless there is a strong use case for this.


Home/HardwareProjects/RaspberryPi

RaspberryPi Shake (Earthquake Monitoring)

Model
RaspberryPi Shake Seismometers
Price

Watch the earth move with Raspberry Shake - Raspberry Pi

Earthquake and Infrasound Monitors-Seismograph - Earthquake and Infrasound Sensors

Raspberry Shake seismometers - Raspberry Pi

RS3D
Vertical & Lateral Motion Seismograph
$799.99 – $1,414.99 USD

RS1D
Vertical Motion Seismograph
$174.99 – $894.99

RS4D
Strong Motion Seismograph
$434.99 – $1,084.99 USD

RS&BOOM
Seismo Acoustic Monitor
$569.99 – $1,224.99 USD

#3619 closed #3558


Extract Ranking

Hi,
I'm trying to understand if it is possible to get metrics from the code that let me know if it converged, and also how to obtain the ranked list of the items.
If not, do you have any suggestions on a good way to extract the ranking?
Thanks!

Hi,
Thanks for raising the issue!
I will add code to extract ranking by the end of the day. This will consist of the score and the standard deviation of the score.
I will let you know once done.
As for convergence - you could monitor both the change in the scores and the standard deviation - setting a threshold on the required confidence.
Thanks again,
Aliaksei

Hi @SasCezar,
It's there!
Thanks again for raising the issue


Looking for contributors?

We're considering adopting this table into our platform. I am wondering if you are looking for contributors? Also, it looks like you have duplicated the VueTableLite for typescript. Wondering if we should look at another way of handling that so we don't have to duplicate the code.
Anyways DM me. Would love to chat about this project and where you see it going.

Hi @jnarowski

We're considering adopting this table into our platform. I am wondering if you are looking for contributors?

Thanks for you are like this component. And I'm glad you brought up this issue, it will be more perfect with your help.

Also, it looks like you have duplicated the VueTableLite for typescript. Wondering if we should look at another way of handling that so we don't have to duplicate the code.

Yes, It's has 2 files, one for Javascript, one for TypeScript, and  more than 80% same...
In fact, I don't know how to Integrate to one file, maybe use TypeScript on dev, and deploy javascript file after compiler?


3.6.1 Helm chart not found in repository

What is the version?
3.3.9-3.6.1
What happened?
The freshly released 3.6.1 version of the chart is not found in https://nvidia.github.io/dcgm-exporter/helm-charts.
$ helm repo list
NAME           	URL
gpu-helm-charts	https://nvidia.github.io/dcgm-exporter/helm-charts

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "gpu-helm-charts" chart repository

$ helm search repo  gpu-helm-charts/dcgm-exporter --versions
NAME                         	CHART VERSION	APP VERSION	DESCRIPTION
gpu-helm-charts/dcgm-exporter	3.6.0        	3.6.0      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.5.0        	3.5.0      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.4.2        	3.4.2      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.4.1        	3.4.1      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.4.0        	3.4.0      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.3.1        	3.3.1      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.3.0        	3.3.0      	A Helm chart for DCGM exporter
gpu-helm-charts/dcgm-exporter	3.2.0        	3.2.0      	A Helm chart for DCGM exporter
...

Thanks for the report. The helm charts have been updated for 3.6.1.

@glowkey wow, thank you so much for the fast response! Can confirm that I was able to pull the newest version!


pybindgen cannot find stddef.h under nix-shell

I am trying to generate Python bindings for NS-3 in the nix-shell environment. It uses pybindgen and castxml for that.
I have constructed a minimal example:
#!/usr/bin/env nix-shell
#!nix-shell -i python -p castxml pythonPackages.pygccxml
from pygccxml import utils, parser

generator_path, generator_name = utils.find_xml_generator()

xml_generator_config = parser.xml_generator_configuration_t(
    xml_generator_path=generator_path,
    xml_generator=generator_name)

parser.parse_string("#include <wchar.h>", xml_generator_config)

I expect this code to exit without errors as it does on my Debian system:
$ pip install --user pybindgen
# apt-get install castxml
$ python example
$ # no errors

This is how it fails under nix-shell:
$ chmod +x example
$ ./example
warning: argument unused during compilation: '-nostdlibinc' [-Wunused-command-line-argument]
In file included from /run/user/1000/tmpdUub09.h:1:
/nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/wchar.h:35:10: fatal error: 'stddef.h' file not found
#include <stddef.h>
         ^~~~~~~~~~
1 error generated.
Traceback (most recent call last):
  File "./example", line 11, in <module>
    parser.parse_string("#include <wchar.h>", xml_generator_config)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/__init__.py", line 60, in parse_string
    declarations = parser.read_string(content)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line 369, in read_string
    decls = reader.read_string(content)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 430, in read_string
    decls = self.read_file(header_file)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 356, in read_file
    return self.read_cpp_source_file(source_file)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 375, in read_cpp_source_file
    xml_file = self.create_xml_file(ffname)
  File "/nix/store/yc25lx6hpnfphll9adscwznxv1yvxbf9-python2.7-pygccxml-1.9.1/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 324, in create_xml_file
    ": %s status:%s" % (gccxml_msg, exit_status))
RuntimeError: Error occurred while running CASTXML:  status:1

This problem was discussed before in #nixos channel: https://logs.nix.samueldr.com/nixos/2018-06-01#1259324
Related NS-3 PR by @teto stuck with the same problem: https://github.com/NixOS/nixpkgs/pull/41264
Merge request on GitLab where I try to build a nix-shell environment for Python bindings generation: https://gitlab.com/nsnam/ns-3-dev/merge_requests/88

hi, so current nixpkgs's ns3 can't build the bindings but I've seen on ns-3 mailing list other people having a hard time building them so I was waiting ns-3.30 to try again.
A release candidate was published yesterday hence my update in nixpkgs to pybdingen 0.20. I will not be able to look at it before next friday most likely so feel free to investigate.

@teto

hi, so current nixpkgs's ns3 can't build the bindings but I've seen on ns-3 mailing list other people having a hard time building them so I was waiting ns-3.30 to try again.

This bug has nothing to do with ns-3 or pybindgen. I have already reduced it to a minimal example that only installs pygccxml, see above. pygccxml is already the latest release in nixpkgs: 1.9.1, same as on https://github.com/gccxml/pygccxml/releases

ns-3.30 patches pygccxml, not sure if that's to fix this bug or another one.

@teto

ns-3.30 patches pygccxml, not sure if that's to fix this bug or another one.
There is no bug outside nix-shell, so ns-3 doesn't even need to fix it. But how can ns-3 patch pygccxml if pygccxml is not even a part of ns-3? Can you link to something about ns-3 patching pygccxml?

By the way, ns-3 bindings build successfully in this Vagrant setup, which uses pybindgen from pypi: https://gitlab.com/nsnam/ns-3-dev/merge_requests/14

The bug is not even about pygccxml, I have reduced it to castxml only:
#!/usr/bin/env nix-shell
#!nix-shell -i sh -p castxml
cat >example.cc <<EOF
#include <wchar.h>
EOF
castxml
castxml --castxml-cc-gnu gcc example.cc

Output:
$ ./example
warning: argument unused during compilation: '-nostdlibinc' [-Wunused-command-line-argument]
In file included from example.cc:1:
/nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/wchar.h:35:10: fatal error: 'stddef.h' file not found
#include <stddef.h>
         ^~~~~~~~~~
1 error generated.

Save this to 1.c:
#include <stddef.h>

Run nix-shell -p clang castxml and compile the file with clang:
$ clang -c 1.c -v
...
clang -cc1 version 7.1.0 based upon LLVM 7.1.0 default target x86_64-unknown-linux-gnu
ignoring duplicate directory "/nix/store/bsdig6vdnmlxmvmqwg0hfvp3hvhb9myi-compiler-rt-7.1.0-dev/include"
ignoring duplicate directory "/nix/store/bsdig6vdnmlxmvmqwg0hfvp3hvhb9myi-compiler-rt-7.1.0-dev/include"
#include "..." search starts here:
#include <...> search starts here:
 /nix/store/bsdig6vdnmlxmvmqwg0hfvp3hvhb9myi-compiler-rt-7.1.0-dev/include
 /nix/store/q82rffgvpsr3j1v009hf75yj2lhy91fl-clang-wrapper-7.1.0/resource-root/include
 /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include
End of search list.

The file /nix/store/q82rffgvpsr3j1v009hf75yj2lhy91fl-clang-wrapper-7.1.0/resource-root/include/stddef.h is in the search list.
Run castxml:
$ castxml -v 1.c
...
clang -cc1 version 7.1.0 based upon LLVM 7.1.0 default target x86_64-unknown-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
 /nix/store/k0dda3b4z4hz4vvfbid7lqpy7c91qjc1-CastXML-0.2.0/share/castxml/clang/include
End of search list.
1.c:1:10: fatal error: 'stddef.h' file not found
#include <stddef.h>
         ^~~~~~~~~~
1 error generated.

The command
find /nix/store/k0dda3b4z4hz4vvfbid7lqpy7c91qjc1-CastXML-0.2.0/share/castxml/clang/include/ | grep -c stddef

outputs "0".
In Debian, CastXML package contains /usr/share/castxml/clang/include/stddef.h: https://packages.debian.org/sid/amd64/castxml/filelist
Debian has CastXML version 0.1+git20180702-3 according to https://packages.debian.org/sid/castxml

Definitely a bug in CastXML package. According to CMakeLists.txt should install stddef.h, see these lines:
https://github.com/CastXML/CastXML/blob/67e01959e94c8579e339c0d3f7acbed658950383/CMakeLists.txt#L165-L173

Looks like this line is wrong:
https://github.com/NixOS/nixpkgs/blob/208aad46916d97b5c2a094f630edb7e470254be8/pkgs/development/tools/castxml/default.nix#L23
Looks like I need to change this one and submit a PR.


Add hint for early monadic let

It might be nice to suggest moving monadic let bindings closer to their first use; I can take a crack at this if you think it would be worthwhile.

I'm not sure how useful this would be - often people care about the order of definitions a lot. I can think of many instances where the let binding is deliberately not near the use, and relatively few where it would be improved by being so but isn't already. Did you have a particular fragment of code that made you think this would help?

I don't have a particular code snippet, I was just thinking of instances where code develops over time and the structure becomes muddled. Maybe this is best handled manually.

The general pattern is that given a set of statements you can sometimes reorder them or separate them to group related concepts. I certainly do that with code I refactor, but usually more on bigger scales - eg. moving whole functions nearer to each other, or to a separate module. I can imagine a tool that analysed use patterns and suggested tighter clusterings would be valuable - but I suspect it's too far away from what HLint does to make sense integrated into HLint.


tests: new sanitycheck tag "emu_time"; increase some timeouts

Quoting bug "Configure QEMU to run independent of the host clock #14173"

We have been struggling for years with issues related to how QEMU
attempts to synchronize guest timer interrupts with the host clock,
for example #12553. The symptom is that heavily loaded sanitycheck
runs have tests spuriously failing due to timing related issues.
This creates noise in our CI runs which masks true bugs in our system
which manifest only intermittently, causing real issues that will
happen all the time at scale to be 'swept under the rug'; right now
any time a test fails sanitycheck retries it a few times and only
consecutive failures produce an error.

There's also a lot of relevant information and more links in:
"List of tests that keep failing sporadically" #12553
This new "emu_time" tag helps by letting users either select or exclude
the tests that really need accurate time to pass and have a high chance
to actually be impacted by this emulation issue. As an example, it's
only with 'sanitycheck --exclude emu_time' that I could spot and file
intermittent but non-emu_time issue #16915. As Andrew predicted above,
it was drown in emu_time noise before that.
Conversely, "--tag emu_time" can be used by developers focusing on
fixing qemu's -icount feature, for instance #14173 or others.
Even before qemu's -icount is fixed, Continuous Integration could be
split in two separate runs: A. --tag emu_time and B. --exclude
emu_time. Only A tests would be allowed retries which would stop hiding
other, unrelated intermittent issues affecting B tests.
This initial commit does not pretend to exhaustively tag all affected
tests. However it's an already functional and useful start of 14 tests
collected from and tested over weeks of local sanitycheck runs and
partially reviewed by qemu clock expert Andy Ross.
This commit also increases the individual timeout of 7 tests that have
been observed to consistently take much longer than the
median (2-3s). This flags how unusually long these are, lets users
temporarily reduce the very long 60s default timeout in their local
workspace and finally should reduce the chance of them timing out on a
heavily loaded system. Set their timeout to 3-4 times the duration
observed in CI and locally.
Signed-off-by: Marc Herbert<EMAIL_ADDRESS>

This new tag:

doesn't hide anything,
exposes non-qemu issues that have been hidden for years and are still routinely being hidden now. See commit message.

Only A tests would be allowed retries which would stop hiding
other, unrelated intermittent issues affecting B tests.

Great!! Is there related logic implemented in CI to make this happen?

I don't know. I've used sanitycheck a million times but I'm not familiar with the CI-only code. I mention this in the commit message because 1. there's already some (indiscriminate) retry logic, so CI is hopefully "half-way there", 2. Isolating flaky tests is the well known practice: https://martinfowler.com/articles/nonDeterminism.html#Quarantine

Initially people will look at the failure report and notice that the failures are in non-deterministic tests, but soon they'll lose the discipline to do that. Once that discipline is lost, then a failure in the healthy deterministic tests will get ignored too. At that point you've lost the whole game and might as well get rid of all the tests.

There's plenty of literature about flaky tests, here's another one: https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html

still think this is a bad idea and not sure who is going to be the user of this. Tags actually are not great in this case, because you are going to exclude everyone, when we have some architecture that have no issues with timing.
The qemu timing issue happens mostly in CI, when running locally, if something fails, you always have the option to rerun the tests and get them all pass on second run, so again, not sure who is going to be the user of this. I'd rather exclude tests selectively than use such a tag.

Definitely approve of the tag.  Ideally we would augment the "--only-failed" feature to be able to use it (maybe that works already; not sure what the precedence is of filters vs. --only-failed).  Long term, yes, we want deterministic tests.  But realistically we're going to be struggling with issues like this repeatedly, and having a framework for handling it can't be a bad thing.

you are going to exclude everyone, when we have some architecture that have no issues with timing.

Fair point. Which architectures? The failures I observed seemed spread across all qemus but I didn't look closely.
(This tag should obviously not be used when testing hardware, it's pretty clear from the name.)

when running locally, if something fails, you always have the option to rerun the tests and get them all pass on second run,

Some of my runs last a day (I don't have a powerful system for every OS) so waiting a second day and manually comparing the sets of failures isn't really an option. Unless you're referring to --only-failed too? I haven't looked into it yet, I will.
In any case an indiscriminate retry is dropping on the floor other intermittent failures for other, unrelated causes. Known as the "broken window" in CI. Even with --exclude emu_time I'm still seeing a couple dozens of other tests failing intermittently. I filed a couple bugs already but I'm not going to keep bothering unless a little bit is done to help with the categorization. I'll just keep a big fat "flaky" tag for myself and done. Once PR #14593 reaches some level of completion I will hopefully not need wide and frequent test runs any more so it'll be even easier for me to ignore intermittent failures.

I'd rather exclude tests selectively than use such a tag.

I miss the conceptual difference between a list of tests and a tag (and no one has to use either)

not sure who is going to be the user of this

Anyone or anything interested in accurate test reports.

Long term, yes, we want deterministic tests.

I found this interesting: https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html

We have invested a lot of effort in removing flakiness from tests, but overall the insertion rate is about the same as the fix rate, meaning we are stuck with a certain rate of tests that provide value, but occasionally produce a flaky result.

Maybe it's because engineers have more time to write more (stress) tests when they run out of bugs to fix. The number of engineers is finite, the number of bugs not so sure. Especially not when concurency's involved.

The addition of the tag alone doesn't hurt anyone and might let CI make intelligent decisions about when to retry and when not to.  However, adjusting the timeout values in this PR might need to be debated.  That's an active change to the tests regardless of whether the tag is used or not.

Does not seem to be going anywhere, closing. Rebase and reopen with new data if you think this still needs to go in.


[CI/CD] automatically trigger yarn run prettier by using git hooks

Description
Maybe husky can help.
If this issue is resolved, we should also modify PR template to remove "I have run yarn run prettier " from checklist.

Can refer to https://github.com/hypertrons/hypertrons/blob/master/.huskyrc Hypertrons pre commit config

Thanks
获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Frank Zhao @.>
发送时间: Thursday, June 23, 2022 6:00:22 PM
收件人: hypertrons/hypertrons-crx @.>
抄送: Lam Tang @.>; Author @.>
主题: Re: [hypertrons/hypertrons-crx] [CI/CD] automatically trigger yarn run prettier by using git hooks (Issue #364)
Can refer to https://github.com/hypertrons/hypertrons/blob/master/.huskyrc Hypertrons pre commit config
―
Reply to this email directly, view it on GitHubhttps://github.com/hypertrons/hypertrons-crx/issues/364#issuecomment-1164215558, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHXOSWASMCKFRZ55JKWYXRDVQQYTNANCNFSM5XW6MJVQ.
You are receiving this because you authored the thread.Message ID: @.***>

closed via #386.


⚠ ClamAV has degraded performance

In 2c566fd, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 62 ms

Resolved: ClamAV performance has improved in 13eaf6d after 11 minutes.


Refactor and cleanup utils

Bug fix
Fixed bug
Utility libraries were not being properly linked and one of them (the one in rmf_building_sim_common) only happened to work because the code was duplicated between header and .cpp file.
Fix applied
This PR removes all the code duplication in the utils.hpp file and links correctly the libraries through target_link_libraries.
The utils.cpp file was actually in the rmf_building_sim_gazebo_plugins namespace so it wasn't used anywhere.
TODO
Figure out if we should move the utils code to a common library, a lot of functions are now duplicated between rmf_robot_sim_common and rmf_building_sim_common

Maybe we should consider merging rmf_robot_sim_common and rmf_building_sim_common into a single rmf_sim_common library. How much not-duplicated code is there?

Maybe we should consider merging rmf_robot_sim_common and rmf_building_sim_common into a single rmf_sim_common library. How much not-duplicated code is there?

The rmf_building_sim_common utils are completely contained in the rmf_robot_sim_common utils, it's effectively a subset of the utilities


Do not initialize the scratch buffers in encode_tx_block

We allocate the largest possible size of those buffers and
work only on a subset of it.
Fix #242.
~20% faster encoding on -s 0

LGTM, Thanks for improving this!


Problem generating 404MHz. The example with 80Mhz works. Reference is 25MHz.

I have a problem generating 404MHz out with the code below. If I modify the code below to set frequency to 80MHz with vfo.setf(80000000UL) it works fine as seen on the oscilloscope. There is a sine out at 80Mhz.
If I try to set it to 404MHz with  vfo.setf(404000000UL) I don't get 404MHz out.
The reference is 25MHz.
Is there something with some dividers I have set wrong since this is a higher frequency or is there a bug in the library? I have used your example code as a basis but simplified it a bit (see below). Can you say what is wrong?

#include <Arduino.h>
#include "adf4351.h"

#define SWVERSION "1.0"

#define PIN_SS 9  ///< SPI slave select pin, default value

ADF4351  vfo(PIN_SS, SPI_MODE0, 1000000UL , MSBFIRST) ;

void setup()
{
  delay(5000);
  Serial.begin(9600) ;
  Serial.print("Hello adf4351 demo v") ;
  Serial.println(SWVERSION) ;
  Wire.begin() ;
  /*!
     setup the chip (for a 10 mhz ref freq) EDIT I did use 25 MHz as reference input, not 10 MHz
     most of these are defaults
  */
  vfo.pwrlevel = 0 ; ///< sets to -4 dBm output
  vfo.RD2refdouble = 0 ; ///< ref doubler off
  vfo.RD1Rdiv2 = 0 ;   ///< ref divider off
  vfo.ClkDiv = 150 ;
  vfo.BandSelClock = 80 ;
  vfo.RCounter = 1 ;  ///< R counter to 1 (no division)
  vfo.ChanStep = steps[2] ;  ///< set to 10 kHz steps

  if ( vfo.setrf(25000000UL) ==  0 )
    Serial.println("ref freq set to 25 Mhz -- modded") ;
    else
         Serial.println("ref freq set error") ;
     vfo.init() ;
     vfo.enable() ;
}

void loop()
{
  delay(1000);

  // ERROR! Doesn't generate 404MHz out.
  // However if I edit this to 80MHz it works and the freq out is correct.
  Serial.println("setf() to 404 MHz instead of 80MHz") ;

  if ( vfo.setf(404000000UL) == 0 ) {
    Serial.print("setf() success freq:") ;
    Serial.println(vfo.cfreq) ;
  } else {
    Serial.println("setf() error") ;
  }
}

A couple of issues.  Your reference frequency settings for 25 Mhz should be set to:
 vfo.RD2refdouble = 1 ; ///< ref doubler on
 vfo.RCounter = 10 ;  ///< R counter to 10

so your Fpfd = 25 mhz * [ ( 1 + 1) / (10* (1+0)) ] = 5 Mhz.
You may have to change your step size as well ; try a step size of steps[4] = 500 kHz.
Not every frequency is available at every step size ; See the datasheet for a discussion of the PLL formula for calculating frequencies.


Add support for storing partner information

When delivering data to customers, some partners may have access to
various data before general availability. It is necessary to store this
kind of information. This patch adds storage for partner metadata and
allows linking partners to specific Variant.Arches on releases.
Partner data can be completely edited through the API as well as the
mapping. Partner types can only be modified in Django admin.
JIRA: PDC-1048

I tried to simplify the serializer. We don't have to care update for the mapping, the only meaningful actions are create and delete.
By the way, it would probably make sense to move VariantArch field to release app and use it for RepoSerializer as well, but that would be a breaking change. I will file an issue about it.

Looks good to me.

:+1:


bosh-cli: 6.4.16 -> 6.4.17

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/cloudfoundry/bosh-cli/releases.
meta.description for bosh-cli is: "A command line interface to CloudFoundry BOSH"
meta.homepage for bosh-cli is: "https://bosh.io"
meta.changelog for bosh-cli is: "https://github.com/cloudfoundry/bosh-cli/releases/tag/v6.4.17"
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17/bin/.bosh-wrapped had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17/bin/bosh had a zero exit code or showed the expected version
0 of 2 passed binary check by having a zero exit code.
0 of 2 passed binary check by having the new version present in output.
found 6.4.17 with grep in /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17
found 6.4.17 in filename of file in /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
bosh-cli

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bosh-cli https://github.com/r-ryantm/nixpkgs/archive/ccb4820b682055c1eb5011dfef80b04517909a55.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17
ls -la /nix/store/55c7xasgnbgap9wvsz2p21qkzi7idqb8-bosh-cli-6.4.17/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

bosh-cli

Maintainer pings
cc @risicle @kalbasit for testing.

@ofborg eval


Add doc-comment to check README file code examples

Considering that there is actually only one example, don't hesitate to close it if you find it not worth it.

CI fails because rustfmt is sad.

rustfmt, my nemesis! (updated btw)

@bors r+

:pushpin: Commit a82da0a36b83386f4be6ea58b1db0cf7e79d1079 has been approved by Amanieu

Thanks!

:hourglass: Testing commit a82da0a36b83386f4be6ea58b1db0cf7e79d1079 with merge 49cfef732b74b60179746e140be7a710423681c7...

:sunny: Test successful - checks-travis
Approved by: Amanieu
Pushing 49cfef732b74b60179746e140be7a710423681c7 to master...


CharacterSetupGui fully updates the entire ui on any change

prob a bit excessive

@metalgearsloth did this get fixed


Simple

Expanding the simple library with more basics and examples.
Just anything I can think of, while waiting for real feedback :)
This is for finding good defaults for the common user. The config and routing system can do far more advanced work than this.
TODO

[x] per zone distributors (some better shorthand would be nice? but at least need to set the seed)
[ ] Continue using "s." syntax or just push global functions and make the syntax cleaner? (I think so yes)
[ ] More features.

think I'll remove the "s." stuff since it's silly...

hmmm.. wondering where/how to document the simple library. wiki in this repo I guess?

think I need to swap out the require in examples with an "always reload the lib" command, else if someone's modifying simple but expecting reload to work... it won't. heh.


PR: Enhance statusbar API

Description of Changes

[ ] Wrote at least one-line docstrings (for any new functions)
[ ] Added unit test(s) covering the changes (if testable)

[ ] Included a screenshot or animation (if affecting the UI, see Licecap)

This PR enhances the statusbar API, so external plugins can define if they add their widget to the right or to the left of the statusbar.

Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: Steff456

@mrclary, building the Mac app started to fail today (as you can see here). Could you help us to take a look at it? Thanks!

@ccordoba12 @steff456, Looks like there was an issue installing spatialindex from homebrew, so the libraries were not found.
>Run brew install spatialindex
  brew install spatialindex
  brew link<EMAIL_ADDRESS>  shell: /bin/bash -l {0}
  env:
    LITE_FLAG: --lite
    DMGNAME: Spyder-Lite.dmg
    DISTDIR: /Users/runner/work/spyder/spyder/dist
==> Downloading https://homebrew.bintray.com/bottles/spatialindex-1.9.3.catalina.bottle.tar.gz
curl: (22) The requested URL returned error: 403 Forbidden
Error: Failed to download resource "spatialindex"
Download failed: https://homebrew.bintray.com/bottles/spatialindex-1.9.3.catalina.bottle.tar.gz

ValueError: dylib /usr/local/lib/libspatialindex.dylib could not be found

I could not reproduce on my local machine. Can you retry the pipeline?

@ccordoba12 @steff456, Looks like there was an issue installing spatialindex from homebrew, so the libraries were not found.
>Run brew install spatialindex
  brew install spatialindex
  brew link<EMAIL_ADDRESS>  shell: /bin/bash -l {0}
  env:
    LITE_FLAG: --lite
    DMGNAME: Spyder-Lite.dmg
    DISTDIR: /Users/runner/work/spyder/spyder/dist
==> Downloading https://homebrew.bintray.com/bottles/spatialindex-1.9.3.catalina.bottle.tar.gz
curl: (22) The requested URL returned error: 403 Forbidden
Error: Failed to download resource "spatialindex"
Download failed: https://homebrew.bintray.com/bottles/spatialindex-1.9.3.catalina.bottle.tar.gz

ValueError: dylib /usr/local/lib/libspatialindex.dylib could not be found

I could not reproduce on my local machine. Can you retry the pipeline?

Sure! I already restarted two times the pipeline, but I can give it another go

@ccordoba12, I see that the build completed successfully, so maybe it was just a temporary hiccup in the connection from GitHub to Homebrew...

@mrclary it was a weird temporal issue, but we are glad that was something temporary. Thanks for your help!