Fix: Refactoring some code? #56 #brokenearlyaccess

[x] https://www.shellcheck.net/
[x] alignments (line length, comments, ...)
[x] remove some comments
[x] inputTapSleep
[x] testColor
[x] fix
[x] Update testColor to be able to send as many possible colors as we want... loop and exit at the first match.
[x] Add a testColorNANDto be able to check !=.
[x] On launch, just switch to usefull tabs (if at least one option is at true)
[x] fix again
[x] fix again, and again *
[ ] Update the way we check colors, because many times we call takeScreenshot whether it's needed or not... something like checking if we had tapped since last capture, or having a force capture parameter if we are in waiting for color loop.
[ ] refacto every PR 😿

[x] PR #48 > merge + fix
[x] PR #51 > Closed
[ ] PR #52
[x] PR #54 > merge + fix
[x] PR #55 > merge + fix
[ ] PR #57
[ ] PR #58
[ ] PR #60
[ ] PR #61

[ ] need profound review 😢
[ ] heavy testing 😭

I want to help. I ran it and got this.

I did not run the script, I'm waiting for other PR to be accepted to update everything, because I will get so much conflict...
And I was thinking to update the way we check colors, because many times we do multiple captures whether it's needed or not... something like checking if we had tapped since last capture, or having a force capture parameter.

script hangs here. waited 10 min to be sure lol

Well, I think I'll start to run the script too :finnadie: (this weekend, I will need more than a few hours to fix my mess)
And activate the DEBUG=2 at the beginning of afk-daily.sh it will show what it's looking for and where it's tapping.
There will be much verbose, but it's better than nothing I think 😄

I think it's all fixed for the most basic task:

config.sh
#!/bin/bash
# --- CONFIG: Modify accordingly to your game! --- #
# --- Use this link for help: https://github.com/zebscripts/AFK-Daily#configvariables --- #
# Player
canOpenSoren=false
arenaHeroesOpponent=5

# General
waitForUpdate=true
endAt="championship"

# Repetitions
maxCampaignFights=10
totalAmountArenaTries=2+0
totalAmountGuildBossTries=2+0

# Store
buyStoreDust=false
buyStorePoeCoins=false
buyStoreEmblems=false

# --- Actions --- #
# Campaign
doLootAfkChest=true
doChallengeBoss=false
doFastRewards=false
doCollectFriendsAndMercenaries=false

# Dark Forest
doSoloBounties=false
doTeamBounties=false
doArenaOfHeroes=false
doLegendsTournament=false
doKingsTower=false

# Ranhorn
doGuildHunts=false
doTwistedRealmBoss=false
doBuyFromStore=false
doStrengthenCrystal=false
doCompanionPointsSummon=false
doCollectOakPresents=false                      # Only works if "Hide Inn Heroes" is enabled under "Settings -> Memory"

# End
doCollectQuestChests=false
doCollectMail=false
doCollectMerchantFreebies=false

output with debug=2
Task: Checking for adb...
Success: Found locally!
Task: Checking for updates...
Success: Checked/Updated!
Task: Searching for config.sh file...
Success: Found!
Task: Validating config.sh...
Success: Passed!
Task: Checking Line endings of file config.sh...
Success: Passed!
Task: Checking Line endings of file afk-daily.sh...
Success: Passed!
Task: Checking last time script was run...
Task: Restarting ADB...
Success: Restarted!
Task: Searching for device through ADB...
Success: Found Bluestacks!

Info: Platform: Bluestacks
Info: Script Directory: storage/emulated/0/scripts/afk-arena

[INFO] Starting script... (Wed Apr 28 10:13:25 CEST 2021)

[DEBUG] testColorOR 450 1775 cc9261
[DEBUG] getColor 450 1775 > RGB: 111111
[DEBUG] inputTapSleep 970 380 0
[DEBUG] inputTapSleep 550 1850
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] inputTapSleep 550 1850
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] testColorOR 740 205 ffc15b
[DEBUG] getColor 740 205 > RGB: 393d2f
[DEBUG] inputTapSleep 550 1850
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] inputTapSleep 550 1500 1
[DEBUG] inputTapSleep 750 1350 3
[DEBUG] inputTapSleep 550 1850 1
[OK] AFK Chest looted.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] inputTapSleep 550 1500 1
[DEBUG] inputTapSleep 750 1350 3
[DEBUG] inputTapSleep 550 1850 1
[OK] AFK Chest looted.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] inputTapSleep 300 1850
[OK] Switched to the Dark Forest Tab.
[DEBUG] getColor 240 1775 > RGB: d49a61
[DEBUG] inputTapSleep 740 1050
[DEBUG] inputTapSleep 550 1370 0

[INFO] End of script! (Wed Apr 28 10:14:11 CEST 2021)

I'll add more levels for DEBUG in futur updates.

Love this debug option!

Also, I've changed my mind: I will first attempt to merge all open PRs that introduce new features to the script without thinking too much if code is written in the "correct shell way", as I'd need to "lose time" on learning that first. So I'd rather create the core script features first and then attempt to correct everything at once. Sorry about that.

Mmh, some experiments later I have multiple possibilities:
# Execute all myfunc then echo OK
$ myfunc(){ echo "DEBUG" >&2; echo 1; echo "DEBUG" >&2; }; if [ "$(myfunc)" = '1' ]; then echo "OK"; fi
DEBUG
DEBUG
OK

# Execute myfunc until return, then echo OK (2 line: myfunc; if [ $? = "1" ])
$ myfunc(){ echo "DEBUG"; return 1; echo "DEBUG"; }; myfunc; if [ $? = "1" ]; then echo "OK"; fi
DEBUG
OK

# Execute myfunc until return, then echo OK (1 line if if myfunc || [ $? = "1" ])
$ myfunc(){ echo "DEBUG" >&2; return 1; echo "DEBUG" >&2; }; if myfunc || [ $? = "1" ]; then echo "OK"; fi
DEBUG
OK

Small test runs and something is broken in my loops (while, until,...) :finnadie:

log (DEBUG=4)
Task: Checking for adb...
Success: Found locally!
Task: Searching for config.sh file...
Success: Found!
Task: Validating config.sh...
Success: Passed!
Task: Checking Line endings of file config.sh...
Success: Passed!
Task: Checking Line endings of file afk-daily.sh...
Success: Passed!
Task: Checking last time script was run...
Task: Restarting ADB...
Success: Restarted!
Task: Searching for device through ADB...
Success: Found Bluestacks!

Info: Platform: Bluestacks
Info: Script Directory: storage/emulated/0/scripts/afk-arena

[INFO] Starting script... (Thu Apr 29 13:30:47 CEST 2021)

[DEBUG] loopUntilNotRGB 1 450 1775 cc9261
[DEBUG] testColorOR 450 1775 cc9261
[DEBUG] getColor 450 1775 > RGB: 0f0f0f
[DEBUG] switchTab Campaign
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] switchTab Dark Forest
[DEBUG] switchTab Ranhorn
[DEBUG] switchTab Campaign true
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] testColorOR 740 205 ffc15b
[DEBUG] getColor 740 205 > RGB: 393d2f
[DEBUG] switchTab Campaign
[OK] Switched to the Campaign Tab.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] lootAfkChest
[OK] AFK Chest looted.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] challengeBoss
[DEBUG] testColorORTapSleep 550 740 f2d79f
[DEBUG] testColorOR 550 740 f2d79f
[DEBUG] getColor 550 740 > RGB: 1f2a1d
[DEBUG] testColorORTapSleep 450 1775 cc9261 0
[DEBUG] testColorOR 450 1775 cc9261
[DEBUG] getColor 450 1775 > RGB: cc9261
[OK] Challenged boss in campaign.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] fastRewards
[OK] Fast rewards collected.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] collectFriendsAndMercenaries
[OK] Sent and recieved companion points, as well as auto lending mercenaries.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] lootAfkChest
[OK] AFK Chest looted.
[DEBUG] getColor 450 1775 > RGB: cc9261
[DEBUG] switchTab Dark Forest
[DEBUG] switchTab Ranhorn
[DEBUG] collectQuestChests
[DEBUG] quickCollectQuestChests
[DEBUG] testColorOR 700 670 82fdf5
[DEBUG] getColor 700 670 > RGB: 82fdf5
[DEBUG] quickCollectQuestChests
[DEBUG] testColorOR 700 670 82fdf5
[DEBUG] getColor 700 670 > RGB: 4d3417
[ERROR] VerifyRGB: Failure! Expected d49a61, but got 54331a instead.

[ERROR] Failed to collect daily and weekly quest chests.
[DEBUG] getColor 20 1775 > RGB: 54331a

It's starting to look like something that could be used 😄
I hope that I will be able to do a default config.sh this weekend.

Progression of my testing

[ ] Player

[ ] canOpenSoren
[x] arenaHeroesOpponent

[ ] General

[ ] waitForUpdate
[x] endAt

[x] Campaign

[x] doLootAfkChest
[x] doChallengeBoss
[x] doFastRewards
[x] doCollectFriendsAndMercenaries

[ ]  Dark Forest

[x] doSoloBounties
[x] doTeamBounties
[x] doArenaOfHeroes
[ ] doLegendsTournament
[ ] doKingsTower

[ ] Ranhorn

[ ] doGuildHunts
[ ] doTwistedRealmBoss
[x] doBuyFromStore

[x] buyStoreDust
[x] buyStorePoeCoins
[x] buyStoreEmblems

[x] doStrengthenCrystal
[x] doCompanionPointsSummon
[ ] doCollectOakPresents

[ ] End

[ ] doCollectQuestChests
[x] doCollectMail
[x] doCollectMerchantFreebies

So cool! Great work! Love to see this progressing.
I also wonder how much impact it'll have on the time it'll take for the script to run. Whenever this is ready, I'll make some small timing tests! And don't worry, I will for sure welcome this PR whenever it's ready!

In my opinion, the impact on the speed of the script will be small, but on the efficiency of development / debugging will be great.
But for example, if I end up doing that:

Update the way we check colors, because many times we call takeScreenshot whether it's needed or not... something like checking if we had tapped since last capture, or having a force capture parameter if we are in waiting for color loop.

There will be only one or two updates to do in the code to update every capture.
Or some of my PR will be reduced to 1 line instead of 4 lines (getColor > test > tap)

I totally agree with that. You're developing tools to assist us in making our tool better! :+1:

Here is my progress:

add of sRGBColorDelta
prepare to use screenshotRequired to optimise the code.

And my test run:
afk-daily.sh > DEBUG=4 & SHOW_DELTA=1

config.sh
#!/bin/bash
# --- CONFIG: Modify accordingly to your game! --- #
# --- Use this link for help: https://github.com/zebscripts/AFK-Daily#configvariables --- #
# Player
canOpenSoren=false                  # Not tested I can't :(
arenaHeroesOpponent=5

# General
waitForUpdate=true
endAt="mail"

# Repetitions
maxCampaignFights=5
maxKingsTowerFights=5
totalAmountArenaTries=2+6
totalAmountGuildBossTries=2+1

# Store
buyStoreDust=true
buyStorePoeCoins=true
buyStoreEmblems=true

# --- Actions --- #
# Campaign
doLootAfkChest=true
doChallengeBoss=false               # Not tested
doFastRewards=true
doCollectFriendsAndMercenaries=true

# Dark Forest
doSoloBounties=true
doTeamBounties=true
doArenaOfHeroes=false                           # not working, try to find a fight then exit, but failed to exit :/
doLegendsTournament=true                        # break if try to launch a fight but no more tickets
doKingsTower=true                               # Open towers but no fight

# Ranhorn
doGuildHunts=false                  # Not tested
doTwistedRealmBoss=false            # Not tested
doBuyFromStore=true
doStrengthenCrystal=true
doCompanionPointsSummon=true
doCollectOakPresents=true

# End
doCollectQuestChests=true
doCollectMail=true
doCollectMerchantFreebies=true                  # not working

And the broken things:

doArenaOfHeroes i think it's a color problem, or maybe i broke something...
doLegendsTournament it's a bug of waitBattleStart 😢 infinite loop without possibility to escape if you don't have remaining tries
doKingsTower Didn't realy look the code

log
[DEBUG] kingsTower
[DEBUG] inputTapSleep 500 870
[DEBUG] battleKingsTower 550 900
[DEBUG] inputTapSleep 550 900 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 540 1350
[DEBUG] testColorOR -f 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1d201c
[DEBUG] testColorOR 1d201c != 1a1212 [Δ 96%]
[DEBUG] inputTapSleep 70 1810 3
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 70 1810 0
[DEBUG] battleKingsTower 250 500
[DEBUG] inputTapSleep 250 500 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 540 1350
[DEBUG] testColorOR -f 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 614317
[DEBUG] testColorOR 614317 != 1a1212 [Δ 83%]
[DEBUG] inputTapSleep 70 1810 3
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 70 1810 0
[DEBUG] battleKingsTower 800 500
[DEBUG] inputTapSleep 800 500 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 540 1350
[DEBUG] testColorOR -f 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 8d5728
[DEBUG] testColorOR 8d5728 != 1a1212 [Δ 73%]
[DEBUG] inputTapSleep 70 1810 3
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 70 1810 0
[DEBUG] battleKingsTower 250 1400
[DEBUG] inputTapSleep 250 1400 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 926d49
[DEBUG] testColorOR 926d49 != 1a1212 [Δ 65%]
[DEBUG] battleKingsTower 800 1400
[DEBUG] inputTapSleep 800 1400 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 550 150 > RGB: 926d49
[DEBUG] testColorOR 926d49 != 1a1212 [Δ 65%]
[DEBUG] inputTapSleep 70 1810
[DEBUG] verifyRGB 240 1775 d49a61
[DEBUG] getColor 240 1775
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 240 1775 > RGB: d49a61
[OK] Battled at the Kings Tower.

doCollectMerchantFreebies looks to be a color problem, or maybe i broke something...

Log
[DEBUG] collectMerchants
[DEBUG] inputTapSleep 120 300 3
[DEBUG] inputTapSleep 510 1820
[DEBUG] testColorNAND 375 940 0b080a
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 375 940 > RGB: bfc5d8
[DEBUG] testColorNAND bfc5d8 != 0b080a [Δ 24%]
[DEBUG] inputTapSleep 200 750 1
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 280 1620 1
[DEBUG] testColorNAND 375 940 050a0f
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 375 940 > RGB: 62401f
[DEBUG] testColorNAND 62401f != 050a0f [Δ 78%]
[DEBUG] inputTapSleep 200 750 1
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 460 1620 1
[DEBUG] testColorNAND 375 940 0b080a
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 375 940 > RGB: 4d321e
[DEBUG] testColorNAND 4d321e != 0b080a [Δ 83%]
[DEBUG] inputTapSleep 200 750
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 70 1810 1
[DEBUG] verifyRGB 20 1775 d49a61
[DEBUG] getColor 20 1775
[DEBUG] takeScreenshot (screenshotRequired=true)
[DEBUG] getColor 20 1775 > RGB: d49a61
[OK] Collected daily/weekly/monthly offer.

PS: I reversed sRGBColorDelta after my testing. Delta is a distance, so 0 should be for being the same and 100 for being opposite

After more than 1 hour of pain :feelsgood:, reading every modification, I did find some mistakes :trollface:, will see later if that fixes all the problems.
But now, it's coffee time :coffee:.

@Zebiano  Should I group function/subfunctions by tabs (Campaign, Dark Forest, Ranhord), to have a more local view of the code?

Is there a way to do loopUntilRGB with 2 different rgb values?

Is there a way to do loopUntilRGB with 2 different rgb values?

At the moment, no, you need to copy/paste the code:
while testColorNAND -f <X> <Y> <COLOR> <COLOR>; do
    sleep 1
done

But it's a good idea, I'll add it in the next commit.

Still broken 😢

doChallengeBoss

failed to detect the fight ending with waitBattleFinish

doArenaOfHeroes

failed to pick an opponent (a test problem I hope)

legendsTournament

failed to go to the screen

doKingsTower

open the right tower, but failed to start the fight then loop in waitBattleStart

doTwistedRealmBoss

looks to be a coordinates problem, but work after entering in the screen
failed to detect the fight ending with waitBattleFinish

doCollectMerchantFreebies

I think test problem

Problems with:

waitBattleFinish & waitBattleFinish may be fix (missing 1 shift)
Failed to go to the screen may be fix (problem with Check if starting from tab or already inside activity)

Working:

Restart on error 😄

Will be working soon:

screenshotRequired values seem to be good, just need to add the condition before taking a screenshot. I'll add it when everything will be working.

@Zebiano  Should I group function/subfunctions by tabs (Campaign, Dark Forest, Ranhord), to have a more local view of the code?

If you think it helps you and others to read the code better, go ahead! I'm not entirely sure I understood what you plan on doing, but I'm pretty sure it's for the better. :)

Is there a way to do loopUntilRGB with 2 different rgb values?

Out of the top of my head I don't think there's a function for that in place no. But I do believe I've used that before somewhere... Might be wrong though.
Why do you need that in the first place though? Can't you work around it by using only one RGB maybe? And it's totally fine if not btw. Just asking in case you're missing something :)

@Zebiano  Should I group function/subfunctions by tabs (Campaign, Dark Forest, Ranhord), to have a more local view of the code?

If you think it helps you and others to read the code better, go ahead! I'm not entirely sure I understood what you plan on doing, but I'm pretty sure it's for the better. :)

I want to have first the core function, then the global games function, then group by a tab (with subfunctions just before where they are needed).
For example, when I was looking for buyStoreItem I did scratch my head for quite a long time 😉

Current order
test()
sRGBColorDelta()
wait()
startApp()
closeApp()
switchApp()
disableOrientation()
takeScreenshot()
readRGB()
getColor()
verifyRGB()
inputSwipe()
inputTapSleep()
testColorOR()
testColorNAND()
testColorORTapSleep()
loopUntilRGB()
loopUntilNotRGB()
doAuto()
doSpeed()
doSkip()
waitBattleStart()
waitBattleFinish()
switchTab()
buyStoreItem()
oakSearchPresent()
oakPresentTab()
oakTryCollectPresent()
checkWhereToEnd()
quickBattleGuildBosses()
lootAfkChest()
challengeBoss()
fastRewards()
collectFriendsAndMercenaries()
soloBounties()
teamBounties()
tapClosestOpponent()
arenaOfHeroes()
legendsTournament()
battleKingsTower()
kingsTower()
guildHunts()
twistedRealmBoss()
buyFromStore()
quickCollectQuestChests()
collectQuestChests()
collectMail()
collectMerchants()
strengthenCrystal()
nobleTavern()
oakInn()
init()
run()

Another day another story or not, still broken 😉

doArenaOfHeroes

Still can't find start a fight, starting to sob 😭

log
[OK] Collected/dispatched team bounties.
[DEBUG] getColor 240 1775 > RGB: d49a61
[DEBUG] arenaOfHeroes
[DEBUG] inputTapSleep 740 1050 3
[DEBUG] inputTapSleep 550 450 3
[DEBUG] inputTapSleep 1000 1800
[DEBUG] inputTapSleep 980 410
[DEBUG] inputTapSleep 540 1800
[DEBUG] testColorNAND 200 1800<PHONE_NUMBER>14
[DEBUG] getColor 200 1800
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 200 1800 > RGB: 150d08
[DEBUG] sRGBColorDelta 150d08 382314
[DEBUG] testColorOR 150d08 != 382314 [Δ 10%]
[WARN] Unable to fight in the Arena of Heroes because a new season is soon launching.
[DEBUG] inputTapSleep 70 1810
[DEBUG] verifyRGB 760 70 1f2d3a Checked the Arena of Heroes out. Failed to check the Arena of Heroes out.
[DEBUG] getColor 760 70
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 760 70 > RGB: 213241
[DEBUG] sRGBColorDelta 213241 1f2d3a
[INFO] Restarting
[ERROR] VerifyRGB: Failure! Expected 1f2d3a, but got 213241 instead. [Δ 2%]
[ERROR] Failed to check the Arena of Heroes out.

legendsTournament

5 first try working, but failed at the 6 try, but will be fixed by #67

doKingsTower

Enter king's tower, but don't start any fight, will look into it

log
[OK] Switched to the Dark Forest Tab.
[DEBUG] getColor 240 1775 > RGB: d49a61
[DEBUG] kingsTower
[DEBUG] inputTapSleep 500 870
[DEBUG] battleKingsTower 550 900
[DEBUG] inputTapSleep 550 900 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 540 1350
[DEBUG] testColorNAND -f 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 1d201c
[DEBUG] sRGBColorDelta 1d201c 1a1212
[DEBUG] testColorOR 1d201c != 1a1212 [Δ 4%]
[DEBUG] inputTapSleep 70 1810 3
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 70 1810 0
[DEBUG] battleKingsTower 250 500
[DEBUG] inputTapSleep 250 500 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 926d49
[DEBUG] sRGBColorDelta 926d49 1a1212
[DEBUG] testColorOR 926d49 != 1a1212 [Δ 35%]
[DEBUG] battleKingsTower 800 500
[DEBUG] inputTapSleep 800 500 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 926d49
[DEBUG] sRGBColorDelta 926d49 1a1212
[DEBUG] testColorOR 926d49 != 1a1212 [Δ 35%]
[DEBUG] battleKingsTower 250 1400
[DEBUG] inputTapSleep 250 1400 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 540 1350
[DEBUG] testColorNAND -f 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: a1e8c8
[DEBUG] sRGBColorDelta a1e8c8 1a1212
[DEBUG] testColorOR a1e8c8 != 1a1212 [Δ 70%]
[DEBUG] inputTapSleep 70 1810 3
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 1a1212
[DEBUG] inputTapSleep 70 1810 0
[DEBUG] battleKingsTower 800 1400
[DEBUG] inputTapSleep 800 1400 2
[DEBUG] testColorOR 550 150 1a1212
[DEBUG] getColor 550 150
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 550 150 > RGB: 926d49
[DEBUG] sRGBColorDelta 926d49 1a1212
[DEBUG] testColorOR 926d49 != 1a1212 [Δ 35%]
[DEBUG] inputTapSleep 70 1810
[DEBUG] verifyRGB 240 1775 d49a61 Battled at the Kings Tower. Failed to battle at the Kings Tower.
[DEBUG] getColor 240 1775
[DEBUG] takeScreenshot [screenshotRequired=true]
[OK] Battled at the Kings Tower.

doTwistedRealmBoss

Fight but don't detect end of the fight

log
[DEBUG] loopUntilRGB 30 420 380 ca9c5d
[DEBUG] testColorOR -f 420 380 ca9c5d
[DEBUG] getColor 420 380
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 420 380 > RGB: ca9c5d
[DEBUG] testColorOR -f 420 380 ca9c5d
[DEBUG] getColor 420 380
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 420 380 > RGB: ca9c5d
[DEBUG] testColorOR -f 420 380 ca9c5d
[DEBUG] getColor 420 380

doCollectMerchantFreebies

Still not clicking on the correct one...

log
[DEBUG] getColor 20 1775 > RGB: d49a61
[DEBUG] collectMerchants
[DEBUG] inputTapSleep 120 300 3
[DEBUG] inputTapSleep 510 1820
[DEBUG] testColorNAND 375 940 0b080a
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 375 940 > RGB: bfc4d8
[DEBUG] sRGBColorDelta bfc4d8 0b080a
[DEBUG] testColorOR bfc4d8 != 0b080a [Δ 76%]
[DEBUG] inputTapSleep 200 750 1
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 280 1620 1
[DEBUG] testColorNAND 375 940 050a0f
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 375 940 > RGB: 62401f
[DEBUG] sRGBColorDelta 62401f 050a0f
[DEBUG] testColorOR 62401f != 050a0f [Δ 22%]
[DEBUG] inputTapSleep 200 750 1
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 460 1620 1
[DEBUG] testColorNAND 375 940 0b080a
[DEBUG] getColor 375 940
[DEBUG] takeScreenshot [screenshotRequired=true]
[DEBUG] getColor 375 940 > RGB: 62401f
[DEBUG] sRGBColorDelta 62401f 0b080a
[DEBUG] testColorOR 62401f != 0b080a [Δ 22%]
[DEBUG] inputTapSleep 200 750
[DEBUG] inputTapSleep 550 300 1
[DEBUG] inputTapSleep 70 1810 1
[DEBUG] verifyRGB 20 1775 d49a61 Collected daily/weekly/monthly offer. Failed to collect daily/weekly/monthly offer.
[DEBUG] getColor 20 1775
[DEBUG] takeScreenshot [screenshotRequired=true]
[OK] Collected daily/weekly/monthly offer.

Good points:

Love [DEBUG] ❤
Restart working better than anticipated (well, it's skipping the problem)
screenshotRequired is looking good, soon will be used 😈

Will check if the fixes are working tomorrow...

Fix delta in testColorOR & testColorNAND (min/max swapped...)

Should fix doCollectMerchantFreebies
Maybe doArenaOfHeroes & doKingsTower

Fix loopUntilRGB (was using testColorOR instead of testColorNAND)

Should fix doTwistedRealmBoss

Also, could I ask you to create a message with all the changes so I can take a look at them instead of having to cycle between each message within this issue? Not yet, as you're not finished. But whenever you think the code refactoring is ready, that would be really nice. :)

Few fixes before the test run (seems I can't read properly 🤓)
Another day, another story 📅

[x] Campaign
[x] Dark Forest
[x] Ranhorn
[x] End

Everything seems to be working 👍
TODO 🔖

Try using screenshotRequired
@Zebiano Waiting for other PR to be done, then, group functions by tabs

Things I have done ✔

[x] Fix shell scripts (https://www.shellcheck.net/)
[x] Code/Comments alignments
[x] Remove unused comments
[x] Normalized functions comments
[x] Add stdout (>&1) or stderr (>&2) on echo
[x] Add DEBUG
[x] Add condition on switch tab (only switch to useful tabs)
[x] Add functions to test color:

testColorOR <X> <Y> <COLOR> [<COLOR>...], instead of:

getColor <X> <Y>
if [ "$RGB" = "<COLOR>" ]; then

getColor <X> <Y>
if [ "$RGB" = "<COLOR>" ] || [ "$RGB" = "<COLOR>" ]; then

testColorOR <X> <Y> <COLOR> [<COLOR>...], instead of:

getColor <X> <Y>
if [ "$RGB" != "<COLOR>" ]; then

getColor <X> <Y>
if [ "$RGB" != "<COLOR>" ] && [ "$RGB" != "<COLOR>" ]; then

[x] Add sRGBColorDelta to see color delta
[x] Add inputTapSleep to tap & sleep in 1 line (and it adds the possibility to add "default" actions when you tap, like screenshotRequired=true) + inputSwipe
[x] Move tests/init/run in functions
[x] Add restart on error
[x] Add wiki (@Zebiano Could be interesting to look at this)

I will try and find some time for the PRs this week still, most likely on the weekend. I just recently started a new job and it's taking a lot of the time I previously had for my personal projects.
Regarding the wiki action for the project, it looked interesting. Not sure yet if we want to use it as it might look a bit overkill. But, if it helps us keeping a nice wiki, as well as everyone being able to work on it without permissions, it looks good. Definitely a low priority though I'd say.

I will try and find some time for the PRs this week still, most likely on the weekend. I just recently started a new job and it's taking a lot of the time I previously had for my personal projects.

No problem 😉
I'm not available this weekend and the second half of next week, so take the time that you need.

The sections will look like this (with functions order by name inside)

Sections
# ##############################################################################
# Section       : Variables
# ##############################################################################

# ##############################################################################
# Section       : Core Functions
# Description   : It's like a library of useful functions
# ##############################################################################
closeApp()
disableOrientation()
getColor()
inputSwipe()
inputTapSleep()
loopUntilRGB()
loopUntilNotRGB()
readRGB()
startApp()
sRGBColorDelta()
switchApp()
takeScreenshot()
testColorOR()
testColorNAND()
testColorORTapSleep()
verifyRGB()
wait()

# ##############################################################################
# Section       : Game SubFunctions
# Description   : It's functions of this specific game
# ##############################################################################
doAuto()
doSpeed()
doSkip()
switchTab()
waitBattleFinish()
waitBattleStart()

# ##############################################################################
# Section       : Campaign
# ##############################################################################
challengeBoss()
collectFriendsAndMercenaries()
fastRewards()
lootAfkChest()

# ##############################################################################
# Section       : Dark Forest
# ##############################################################################
arenaOfHeroes()
battleKingsTower()
kingsTower()
legendsTournament()
soloBounties()
tapClosestOpponent()
teamBounties()

# ##############################################################################
# Section       : Ranhorn
# ##############################################################################
buyFromStore()
buyStoreItem()
guildHunts()
nobleTavern()
oakInn()
oakPresentTab()
oakTryCollectPresent()
oakSearchPresent()
quickBattleGuildBosses()
strengthenCrystal()
twistedRealmBoss()

# ##############################################################################
# Section       : End
# ##############################################################################
checkWhereToEnd()
collectQuestChests()
collectMail()
collectMerchants()
quickCollectQuestChests()

# ##############################################################################
# Section       : Test
# ##############################################################################
test()
tests()

# ##############################################################################
# Section       : Script Start
# ##############################################################################
init()
run()

And if you want, it's the moment to rename functions that have not the best name, and write naming convention.
Maybe using function_subfunction could be a good idea, for example:

Dark Forest

tapClosestOpponent() -> arenaOfHeroes_tapClosestOpponent()
battleKingsTower()-> kingsTower_battle()

Ranhorn

quickBattleGuildBosses() -> guildHunts_quickBattle()
buyStoreItem() -> buyFromStore_buyItem()
oakTryCollectPresent() -> oakInn_tryCollectPresent()
oakSearchPresent() -> oakInn_searchPresent()
oakPresentTab() -> oakInn_presentTab()

End

quickCollectQuestChests() -> collectQuestChests_quick()

Sections
# ##############################################################################
# Section       : Variables
# ##############################################################################

# ##############################################################################
# Section       : Core Functions
# Description   : It's like a library of usefull functions
# ##############################################################################
closeApp()
disableOrientation()
getColor()
inputSwipe()
inputTapSleep()
loopUntilRGB()
loopUntilNotRGB()
readRGB()
startApp()
sRGBColorDelta()
switchApp()
takeScreenshot()
test()
testColorOR()
testColorNAND()
testColorORTapSleep()
verifyRGB()
wait()

# ##############################################################################
# Section       : Game SubFunctions
# Description   : It's the extension of the Core for this specific game
# ##############################################################################
doAuto()
doSpeed()
doSkip()
switchTab()
waitBattleStart()
waitBattleFinish()

# ##############################################################################
# Section       : Campaign
# ##############################################################################
challengeBoss()
collectFriendsAndMercenaries()
fastRewards()
lootAfkChest()

# ##############################################################################
# Section       : Dark Forest
# ##############################################################################
arenaOfHeroes()
arenaOfHeroes_tapClosestOpponent()
kingsTower()
kingsTower_battle()
legendsTournament()
soloBounties()
teamBounties()

# ##############################################################################
# Section       : Ranhorn
# ##############################################################################
buyFromStore()
buyFromStore_buyItem()
guildHunts()
guildHunts_quickBattle()
nobleTavern()
oakInn()
oakInn_presentTab()
oakInn_searchPresent()
oakInn_tryCollectPresent()
strengthenCrystal()
twistedRealmBoss()

# ##############################################################################
# Section       : End
# ##############################################################################
checkWhereToEnd()
collectQuestChests()
collectQuestChests_quick()
collectMail()
collectMerchants()

# ##############################################################################
# Section       : Test
# ##############################################################################
tests()

# ##############################################################################
# Section       : Script Start
# ##############################################################################
init()
run()

With screenshotRequired it's still working well 😄

Maybe using function_subfunction could be a good idea

I guess this is generally a good idea, I just didn't understand in your examples why a function like arenaOfHeroes_tapClosestOpponent() even exists outside of the arenaOfHeroes() function. Doesn't it make sense to have the action of tapping the closest opponent inside the arenaOfHeroes() function? No need to call it a function in there either.

tapClosestOpponent is the function added few fix ago to try to fight another opponent if we don't find the desired one.
It's already in the current version, but it's not ordered or renamed in a way we can understand it's only used in the arenaOfHeroes function.

Looks to be working, still some values missing (currently 000000) to add as soon as possible, and it would be cool if every inputTapSleep was commented to where it's tapping 😄

Log
Task: Checking for adb...
Success: Found locally!
Task: Searching for config.ini file...
Success: Found!
Task: Validating config.ini...
Success: Passed!
Task: Checking Line endings of file config.ini...
Success: Passed!
Task: Checking Line endings of file afk-daily.sh...
Success: Passed!
Task: Checking last time script was run...
Task: Restarting ADB...
Success: Restarted!
Task: Searching for device through ADB...
Success: Found Bluestacks!

Info: Platform: Bluestacks
Info: Script Directory: storage/emulated/0/scripts/afk-arena

[INFO] Starting script... (Wed May 12 07:39:54 CEST 2021)

[OK] Switched to the Campaign Tab.
[OK] Switched to the Dark Forest Tab.
[OK] Switched to the Ranhorn Tab.
[OK] Switched to the Campaign Tab.
[OK] AFK Chest looted.
[OK] Challenged boss in campaign.
[OK] Fast rewards collected.
[INFO] No mercenaries to lend...
[OK] Sent and recieved companion points, as well as auto lending mercenaries.
[OK] AFK Chest looted.
[OK] Switched to the Dark Forest Tab.
[OK] Collected/dispatched solo bounties.
[OK] Collected/dispatched team bounties.
[OK] Checked the Arena of Heroes out.
[OK] Battled at the Legends Tournament.
[OK] Battled at the Kings Tower.
[OK] Switched to the Ranhorn Tab.
[OK] Battled Wrizz and possibly Soren.
[OK] Checked Twisted Realm Boss out.
[OK] Visited the Store.
[OK] Strenghened resonating Crystal.
[INFO] No heroes to ascend...
[OK] Attempted to ascend heroes.
[OK] Summoned one hero with Companion Points.
[OK] Attempted to collect Oak Inn presents.
[OK] Collected daily and weekly quest chests.
[INFO] No mail to collect...
[OK] Collected Mail.
[INFO] No Weekly to collect...
[INFO] No Monthly to collect...
[OK] Collected daily/weekly/monthly offer.

[INFO] End of script! (Wed May 12 07:56:39 CEST 2021)

ToDo
    # TODO:
    # check red dot to see if free fast reward is avaible
    # buyAmplifyingEmblem check yellow
    # Buy Elite Hero Soulstone
    # test 420 850 # Row 1, slot 2

    ## Next month
    # Check if red mark - Monthly Deals
    # Weekly - Purchase an item or hero from the Labyrinth store once
    # test 900 1500 # row 6, item 4 >  60 Rare Hero Soulstone / 2400 Labyrinth Tokens
    # test 660 1500 # row 6, item 3 >  60 Rare Hero Soulstone / 2400 Labyrinth Tokens
    # test 420 1500 # row 6, item 2 >  60 Rare Hero Soulstone / 2400 Labyrinth Tokens

So, I just started VSCode up thinking I'll be going through all those PRs and test them one by one, and suddenly realise they're all gone! 😄
I guess you decided to join them all into one big thing, which I respect. I also just took a glance over the wiki and I'm completely blown away by all the work you've put into it. Like seriously, thank you ever so much. Incredible. And the same applies to this PR of course.
Well... I guess what is missing now is me actually testing it all and validating it all. Oh boy, here we go...!

I of course didn't want to ping myself but instead ask @kevingrillet to take a look at my requested changes.

The .wiki folder with the individual files, are those actually linked up with the wiki of the project? Or are you making them ready to implement this action?

It's to help me prepare the fusion, you just need to copy/paste into the wiki :)
Or if you want to implement the action, it's already here.

I want to remove nox support, as it only gave me headaches and is then one less emulator we need to test every time.

No problem, it's just a few lines to remove.

Changing RGB variable name to HEX is also something I'd like to do in this PR.

Ok, just need to search & replace :)

Something else I'd like to define is the use of infinitive form or not. For example, should we use "buys stuff" or "buy stuff" when using comments in the code or even when writing in the README.md? I tend to prefer "buy stuff", "check if", "execute when", etc.

Ok, I'll try to look at this.

I will take a look at the wiki when this PR is ready to be merged, aka it'll be the last thing.

:+1:

I still need to go over the README.md, and I also want to add the stuff you mentioned here to this PR.

:+1:

Overall I'd say some more comments here and there would help, especially in those long ifs or those complicated functions. But this is minor, as mostly you can read the description above each function and you already have a good enough idea on what and how it's supposed to work! Reminder to self: check all of the comments!

I did add many comments to know where the script taps. But sometimes I didn't look at that (how the inn work for example)
I'll try to add all your suggestions/comment them as soon as possible (next week)

OK, I'll start testing the features one by one now @kevingrillet. Not sure if you still have some uncommitted changes that you might want to commit before I start? I'll try my best to do it in small commits, instead of big ones, so it's easier to pinpoint certain things.

I wanted to add a few things too, like check red dot before entering the inn / guild.

Let's keep out commits small then so we can easily merge stuff in case we're working on the same things! Feel free to code as well of course :)

Let's keep our commits small then so we can easily merge stuff in case we're working on the same things! Feel free to code as well of course :)

I'm with family today, so I'll work on it tomorrow

One more thing I'd like to quickly discuss: when the script encounters an error, it restarts and then skips the part it found the error in. So for example, the script gets hung up fighting in the campaign, so it then restarts and instead of attempting to fight in it again, skips it and goes on.
Why did you decide to do it this way? Any specific reason? I think it would be better if the script would try it again first, and then if that fails again, skip it. Maybe we could define a max amount of tries or something like that (yet another config variable, ugh)?
Forgive me if we've talked about this before and I'm not remembering. :)

I decided that because when I added the functionality the script would break if we try to redo something already half done like:

FastRewards, if you already collect the free reward
ArenaOfHeroes, if you don't have any remaining tries
LegendsTournament, if you don't have any remaining tries
GuildHunts, if you don't have any remaining tries

So when there is an error I will do it manually or try it with my debug config file

Can't we up the default DEFAULT_DELTA a bit @kevingrillet? 3 seems a bit low honestly. I'd say max 10, so we could do with 5 maybe? Have you played around with this and seen how much the delta changes things?

If you set default delta at 10 it will see dark brown & dark red as the same color :/
So you will need to remove it from 90% of the code and update it manually everywhere...
For example, it will break the new inn function, the shop too I think, ...

Hm, I see. Then I guess for the shop we could use -d 5 instead of the default value. Some more testing will be needed! Working on it now :)

Alright! I'm mostly done with testing basic feature of the script, and I can happily say they're all working as expected. Nice work @kevingrillet!
Next up is finishing the TODOs that are left, and this PR is basically ready to be finally merged. 🎉


The results of vllm deployment of qwen-14B are inconsistent with the results of the original qwen-14B

The results of vllm deployment of qwen-14B are inconsistent with the results of the original qwen-14B, and we hope to optimize the integration effect of qwen-14B as soon as possible

The same to me.

@qingjiaozyn @huihuiustc As I know, vLLM does not ensure bit-wise correctness (See Page 38 of the meetup slides). If the result answer's meaning is the same literally. It is fine.

@guoyuhong There is still a big difference between the two answers. Sometimes, VLLM results are only part of the answer, not comprehensive

The same to me.

+1


Adding new translation for Field Issues label.

Q
A

Bug fix?
[ ]

New feature?
[X]

New sample?
[ ]

Related issues?
fixes #X, partially #Y, mentioned in #Z

What's in this Pull Request?
Add a new translation for the text "Field issues:" that is missing in the PropertyFieldCollectionData control.

Thank you @nviolero for the contribution!


Guide for Blazor Server

I will appreciate a setup guide for server side Blazor.

Hi there, we have some new community posts on our forums which may be able to help you. Please check out
How to run sciChart JS in Server Side Blazor
and also try the open source SciChart.js Blazor wrapper here.
Best regards
Andrew


[android] Scrolling in feeds has noticably less FPS than on web

Describe the bug
Scrolling is noticably less performant in Bluesky's native app than on web. This is most noticable on mid-tier Android phones, especially when under throttling on low charge.
To Reproduce
Steps to reproduce the behavior:

Do this on a mid tier Android phone. I personally notice this on an Xperia 10 III when under throttling due to high heat or low battery. This could also be noticable on a sufficiently throttled emulator.
Open Bluesky, log in, then scroll around in a feed; note the fps feel
Open bsky.app and do the same. Note how it scrolls more smoothly
For that matter, open TOKIMEKI (which uses Capacitor, ie. a web view) and compare the scrolling performance

Expected behavior
The scrolling performance should not be noticably worse than similar apps or Bluesky itself on web.
Details

Platform: Android
Platform version: Android 13
App version: 1.7.8

Is this still really bad? There were quite a few scroll perf improvements that were merged in over the past month or so, so hoping those might have helped out with this!


EAGLE-476: outdated hbase audit log parser

https://issues.apache.org/jira/browse/EAGLE-476

great fix and merged


Cursor & screen movement feels slow in large files when context is enabled

Consider strategies to reduce the delay, such as doing some work asynchronously or throttling the context update.

In 500 lines of typescript i essentially cant move my cursor anymore, without waiting ~1 second for a redraw.

Quite concerning. I'll try to investigate this issue in the next days.

This has been fixed in https://github.com/romgrk/nvim-treesitter-context/commit/5130e4bb78a77a95815bcd5c7a434406e3283595. It was trying to highlight all the children of the contexted nodes, which isn't just the function/class name, it's all its content as well. There shouldn't be any more performance delays now, please re-open if it persists.

works like a charm now :) thanks for the quick processing.


dracula-theme for Thunderbird

New Theme?
Is is possible to apply dracula-theme into [Thunderbird](https://www.thunderbird.net/en-US/)?

I would also very much like a dracula theme for thunderbird. (and, if some makes so.. not only color but perhaps it would be possible to do a little wider line spacing :-))

@avatar-lavventura @pragomer @zenorocha Did you already see my Thunderbird theme?
See #473.

@avatar-lavventura @pragomer @zenorocha Did you already see my Thunderbird theme?
See #473.

That's awesome @maxfrei750, closing this issue in favor of #473 :)

That's awesome @maxfrei750, closing this issue in favor of #473 :)

Seems like it does not work for Thunderbird newest version; Works with Thunderbird 60.0 - *

Seems like it does not work for Thunderbird newest version; Works with Thunderbird 60.0 - *

@avatar-lavventura
I'm currently running Thunderbird version 78.6.0 (32-bit) on Windows 10 and the theme is working

@avatar-lavventura
I'm currently running Thunderbird version 78.6.0 (32-bit) on Windows 10 and the theme is working


Send all pending messages at once

Apropos of issue #3, I've done a little patch to the websockets lib [1] that allows you to send lots of messages with a single call to writev() rather than just the one, and this PR makes use of this to send things in bulk. It seems to get better performance but I'd like some independent verification please...
[1] https://github.com/DaveCTurner/websockets/tree/bulk-sending

Initial testing seem to show that the performance is better.

Perhaps there is something wrong with my build setup, but I count the same number of calls to writev in both cases.

Ok, so I forgot to merge the changes to the actual benchmark :)
I tried running it 5 times, and calls to writev went from exactly 600200 to [45668, 49315, 46786, 45827, 49551], so it's a great improvement on that front.

Performance is still random with my test setup, so I can't really comment on it ...

/cc @lpeterse - can you verify this evinces an improvement? Seems like it should.

Tidying up by closing some of my stale PRs. Sorry for the noise.


ERROR: The argument type 'PageController?' can't be assigned to the parameter type 'ScrollController'.

I was using the Marquee Package that depends on this package and I noticed that the pub.dev
shows the following error during the analysis issues.
ERROR: The argument type 'PageController?' can't be assigned to the parameter type 'ScrollController'.

When I ran my project using the marquee package, it sure enough, it showed an execution error.

Thank you for pointing out the problem. I see it existed in the main Flutter channel. It's good to be prepared for upcoming changes. I've published a new version (4.1.1) with updates that fixed the static analysis issues.
However, I believe the problem still exists in the mentioned package, as it needs to be modified for this change.

Thank you for pointing out the problem. I see it existed in the main Flutter channel. It's good to be prepared for upcoming changes. I've published a new version (4.1.1) with updates that fixed the static analysis issues.
However, I believe the problem still exists in the mentioned package, as it needs to be modified for this change.

I recommend adding an assertion to ensure that the user adds the controller, or maybe create one if child.controller is null.
Option A:
assert(child.controller != null, 'Your PageView widget needs a PageController')
Option B:
controller: child.controller ?? PageController()


add serial to json example

This example is very similar to the existing stream-requests example, but emits JSON instead of log messages.
This example is particularly useful as a monitor when combined with a JSON tool like jq. For example, to display only heartbeat messages you can do:
$ go run ./examples/serial-to-json | jq 'select(.MessageType == "*minimal.MessageHeartbeat")

Thanks for the review. PR updated.

I'm not particularly happy about the fact that the standard JSON library can't handle NaNs, but i think it's a problem that can be handled in a second moment.

It's not the standard library which is at fault here: JSON has no way of representing NaNs, indeed there is no string format for NaNs that I know of.


ENOENT error on pycache when running push or pull after deleting lambda layer from cloud

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v16.14.2
Amplify CLI Version
8.1.0
What operating system are you using?
Windows 10
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
Deleted Lambda Layer via AWS Console
Amplify Categories
function
Amplify Commands
pull, push
Describe the bug
When running amplify push or amplify pull even with --force command, I get the following error:
Could not initialize 'staging': ENOENT: no such file or directory, realpath 'C:\Users\<UserName>\Documents\...\...\...\amplify\.temp\#current-cloud-backend\function\<layer name>\lib\python\lib\python3.10\site-packages\pipenv\patched\notpip\_internal\distributions\__pycache__'

Expected behavior
Expected changes from cloud to be pulled without error
Reproduction steps
Not entirely sure if this will replicate the issue, but here is what I personally am stuck at.

Create a lambda layer via CLI using amplify add function
Delete layer in cloud
Run amplify pull to pull changes from cloud

GraphQL schema(s)
No response
Log output
No response
Additional information
I have tried removing the .temp folder,  and I have verified that the directory that the error is presenting is in fact existing. I can provide any additional information as needed

Ah, good to know. Unfortunately that does not work with the error:
An error occurred fetching the latest layer version metadata for "<lambda layer name>": Cannot read property 'LogicalName' of undefined

Hey @jpollmann102 I believe this error is due to the Layer no longer existing, so the CLI cannot get the related layer metadata.

An error occurred fetching the latest layer version metadata for ""

> amplify remove function
? Choose the resource you would want to remove 10263layer58c94806 (layer)
When you delete a layer version, you can no longer configure functions to use it.
However, any function that already uses the layer version continues to have access to it.
✖ Loading layer data from the cloud...
An error occurred fetching the latest layer version metadata for "10263layer58c94806": Cannot read property 'LogicalName' of undefined

But no worries! We can do a few manual edits to remove this layer from our local project:

remove the layer directory from amplify/backend/function
remove the layer references in amplify/team-provider-info.json
remove the layer references in amplify/backend/backend-config.json
run amplify env checkout <current-env-name>
this should now show your layer in the DELETE state
push with amplify push -y
observe successful removal from project 🙌

Awesome, thanks for the help, I appreciate it


Fixes #13 - Implement Wrapper methods for Cassandra connections, results and statements

@sualeh I think I'll release a new version including these changes tomorrow.


Fullscreen mode no longer works in the latest version

Prerequisites

[] I'm using latest version: v 0.19.0

Description
The latest version, fullscreen mode does not work correctly. Instead, I get a small box in the top left corner of the screen.
Operating system: Windows 10
Steps to Reproduce

Ensure latest version
Tick "breaks are fullscreen" in the settings
Wait for next break

Expected behavior: the break screen takes up the whole screen
Actual behavior: the break screen only takes up a small portion of the screen in the top left corner
**Reproduces how often:**100% of the time
Additional Information
Any additional information, configuration or data that might be necessary to reproduce the issue.

Thanks for reporting. This has been already fixed in master and hopefully over weekend I will release fix version ;]


Ability to pass an existing JSON schema

My understanding is that currently the only way to pass a JsonScheme is to use the JsonScheme builder on a Java class. But what if I already have my json schema file available ? Can I read it and pass it as a parameter ?

Hi @tfrancart, thanks for using simple-openai. Currently you cannot. This could be an improvement.

Thank you for your nswer. I would very much welcome this improvement. My use-case involves working with externally generated JSON schemas that I don't want to encode as java classes.
Could this be as simple as adding an extra constructor to JsonSchema taking a JsonNode as an input ?

@tfrancart It is possible to enable passing a predefined Json schema (JsonNode) directly instead of creating a class that defines the structure of your function, but, you should be aware that you will not be able to use the FunctionExecutor utility class to automatically invoke your function with the parameter values returned by OpenAI and you will have to do it manually, because FunctionExecutor requires the class that you would no longer create.
Are you okay with losing that ability?

Thank you for your proposal. My need is on structured output, not function executor. Here are some details on what I would need:

I would like to pass a custom JSON schema for structured output (as exemplified at https://github.com/sashirestela/simple-openai?tab=readme-ov-file#chat-completion-with-structured-outputs).
I also would like to use the FunctionExecutor, this time with one function having a schema encoded as a java class.

Would that be possible ? or would the usage of a custom schema for structured output render impossible the use of a class for FunctionExecutor ?

Could you share a real Json schema (maybe the larger) that you are going to use for structured output?
You should be aware that your custom Json schema must be aligned with the rules defined in:
https://platform.openai.com/docs/guides/structured-outputs#supported-schemas
When you use a class to define the Json schema, several rules are covered by default:

All fields must be required
additionalProperties: false must always be set in objects
key ordering

Will your custom Json schema accomplish those rules, among others?

Hello, I am aware of the OpenAI structured output rules. My schema will conform to these rules.
I cannot share an example right now (I don't have one ready yet) but I will try to do it.
Thanks


METRON-961: Spooky intermittent exceptions in pcap during sync should be handled and explained

Contributor Comments
Right now, we have a couple scary exceptions that may happen periodically during sync, which should not prevent writing and shouldn't be percolated as an exception.
These exceptions are the ones mentioned in https://issues.apache.org/jira/browse/METRON-960
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

+1 by inspection


Add llamav2 fp8 inference

Add llamav2 fp8 inference
adding kv cache in fp8 option

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

I left a couple of comments.
Additional question: does this only work with the 7B-parameter checkpoint of Llama2 or are the 13B/70B checkpoints compatible too?

It should work but we didn't commit to it.

LGTM

@regisss can we merge the PR?

@regisss can we merge the PR?

I'm going to try it quickly with 1.13, it should be merged in a couple of hours

@bgoldberg-habana I'm getting this error when running the example:
Traceback (most recent call last):
  File "run_generation.py", line 490, in <module>
    main()
  File "run_generation.py", line 294, in main
    generate()
  File "run_generation.py", line 275, in generate
    outputs = model.generate(
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/generation/utils.py", line 680, in generate
    unwrap_deepspeed_model(self).allocate_kv_cache(
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 493, in allocate_kv_cache
    self.model.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 331, in allocate_kv_cache
    layer.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 267, in allocate_kv_cache
    self.self_attn.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 103, in allocate_kv_cache
    self.past_key = torch.zeros(key_shape, dtype=dtype, device=device)
RuntimeError: Expected habana_helpers::is_supported_type(type) to be true, but got false.

It seems to be an issue with torch.float8_e4m3fn taht is not recognized. Do you know what could be the reason?

@bgoldberg-habana I'm getting this error when running the example:
Traceback (most recent call last):
  File "run_generation.py", line 490, in <module>
    main()
  File "run_generation.py", line 294, in main
    generate()
  File "run_generation.py", line 275, in generate
    outputs = model.generate(
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/generation/utils.py", line 680, in generate
    unwrap_deepspeed_model(self).allocate_kv_cache(
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 493, in allocate_kv_cache
    self.model.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 331, in allocate_kv_cache
    layer.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 267, in allocate_kv_cache
    self.self_attn.allocate_kv_cache(batch_size, max_seq_len, inp_seq_len, kv_cache_fp8)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 103, in allocate_kv_cache
    self.past_key = torch.zeros(key_shape, dtype=dtype, device=device)
RuntimeError: Expected habana_helpers::is_supported_type(type) to be true, but got false.

It seems to be an issue with torch.float8_e4m3fn that is not recognized. Do you know what could be the reason?

'which version do you use? 1.12? 1.13?
because it's only part of 1.13 release

Yes 1.13, I'm using this Docker image: vault.habana.ai/gaudi-docker/1.13.0/ubuntu20.04/habanalabs/pytorch-installer-2.1.0:latest

And
HL-SMI Version:                           hl-1.13.0-rc-fw-<IP_ADDRESS>
Driver Version:                                     1.13.0-ee32e42

from hl-smi

is_supported_type

running on gaudi1 or 2?

Ah that must be this! Gaudi1 (DL1 instance on AWS), my Gaudi2 instance won't get 1.13 before next week...
fp8 is only supported on Gaudi2 right?

Okay let's merge then, could you just let me know the throughput you get with the example command line so that I can compare with my runs next week please?

i'll send you all the numbers i  have through @libinta

Thanks!

@bgoldberg-habana When I try the command of the README with Gaudi2 on 1.13, I get the following error:
Traceback (most recent call last):
  File "run_generation.py", line 490, in <module>
    main()
  File "run_generation.py", line 294, in main
    generate()
  File "run_generation.py", line 275, in generate
    outputs = model.generate(
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/generation/utils.py", line 800, in generate
    return self.greedy_search(
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/generation/utils.py", line 1371, in greedy_search
    model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/optimum/habana/transformers/models/llama/modeling_llama.py", line 584, in prepare_inputs_for_generation
    input_ids = input_ids[:, :token_idx]
RuntimeError: Graph compile failed. synStatus=synStatus 26 [Generice failure].

Does it work on your side?

@bgoldberg-habana It's actually just a \ missing at the end of the line where the env variables are declared. Fixed in #586.


Fix networkx forward nightly ci

What do these changes do?

Related issue number

Fixes #1322

Codecov Report

Merging #1387 (f010815) into main (91f4248) will decrease coverage by 3.31%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1387      +/-   ##
==========================================
- Coverage   43.59%   40.27%   -3.32%
==========================================
  Files          88       88
  Lines        9610     9610
==========================================
- Hits         4189     3870     -319
- Misses       5421     5740     +319

Impacted Files
Coverage Δ

python/graphscope/tests/unittest/test_java_app.py
0.00% <0.00%> (-100.00%)
:arrow_down:

python/graphscope/deploy/hosts/cluster.py
31.64% <0.00%> (-55.70%)
:arrow_down:

python/graphscope/analytical/app/java_app.py
24.24% <0.00%> (-44.95%)
:arrow_down:

python/graphscope/framework/utils.py
40.68% <0.00%> (-13.39%)
:arrow_down:

python/graphscope/framework/app.py
67.33% <0.00%> (-7.04%)
:arrow_down:

python/graphscope/framework/graph_schema.py
50.11% <0.00%> (-6.24%)
:arrow_down:

python/graphscope/client/rpc.py
78.22% <0.00%> (-4.84%)
:arrow_down:

python/graphscope/tests/conftest.py
48.21% <0.00%> (-4.65%)
:arrow_down:

python/graphscope/analytical/udf/utils.py
36.17% <0.00%> (-3.20%)
:arrow_down:

python/graphscope/client/session.py
64.41% <0.00%> (-1.69%)
:arrow_down:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91f4248...f010815. Read the comment docs.


Create proposal for defining a dockerfile component on alizer

Which area/kind this issue is related to?

/area alizer
Issue Description

Related: EPIC https://github.com/devfile/api/issues/1159
To improve port detection around dockerfiles, we need to consider cases where a dockerfile may not be inside a detected component. This requires a new component definition which this proposal will focus on.
Target date: July 04

Follow old comments on https://github.com/redhat-developer/alizer/issues/244

PR moved from redhat-developer/alizer to https://github.com/devfile/alizer/pull/3


Request: additional highlight terms

Hello - could you please consider highlighting the following terms:

Parent
Color
InlineSetting
InlinePattern

Thanks!

Merlin hasn't updated this in two years, you'll have a better chance forking it and making the changes yourself. I've been wanting to make an updated fork of this for a while now but just haven't had the time to do it myself.

Added inline setting and pattern
what is there to highlight about parent and color? I did not find anything particular in the docs


statistics: add HotPeerStat.GetBytesRate() & use float64 rates

What problem does this PR solve?

The HotPeerStat has a raw BytesRate and a denoised RollingBytesRate. There  are several places using the raw BytesRate should use the denoised RollingBytesRate actually.
The type of rates like BytesRate and KeysRate should be float64 instead of uint64.

What is changed and how it works?

Add GetBytesRate() for HotPeerStat to make it convenient to use RollingBytesRate and fix the places using wrong bytes rate.
Change type of the rates to float64.

Check List
Tests

Unit test

Code changes

Has HTTP API interfaces change

Related changes

Need to update the documentation

Codecov Report

Merging #1801 into master will decrease coverage by 0.07%.
The diff coverage is 79.59%.

@@            Coverage Diff            @@
##           master   #1801      +/-   ##
=========================================
- Coverage   77.68%   77.6%   -0.08%
=========================================
  Files         166     166
  Lines       16395   16391       -4
=========================================
- Hits        12736   12720      -16
- Misses       2629    2633       +4
- Partials     1030    1038       +8

Impacted Files
Coverage Δ

server/server.go
82.04% <ø> (-0.57%)
:arrow_down:

server/api/trend.go
68.75% <0%> (ø)
:arrow_up:

server/coordinator.go
87.2% <0%> (+1.98%)
:arrow_up:

server/statistics/store.go
96.07% <100%> (ø)
:arrow_up:

server/statistics/hot_peer.go
100% <100%> (ø)
:arrow_up:

server/schedulers/hot_region.go
83.09% <91.66%> (ø)
:arrow_up:

server/statistics/hot_peer_cache.go
89.3% <94.44%> (-0.14%)
:arrow_down:

pkg/etcdutil/etcdutil.go
76.81% <0%> (-14.5%)
:arrow_down:

server/tso/tso.go
78.89% <0%> (-6.43%)
:arrow_down:

server/region_syncer/client.go
78.94% <0%> (-3.95%)
:arrow_down:

... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd68e13...7c7e655. Read the comment docs.

Is this change compatible with TiDB? I see some declaim in infoschema/tables:
var tableTiDBHotRegionsCols = []columnInfo{
	{"TABLE_ID", mysql.TypeLonglong, 21, 0, nil, nil},
	{"INDEX_ID", mysql.TypeLonglong, 21, 0, nil, nil},
	{"DB_NAME", mysql.TypeVarchar, 64, 0, nil, nil},
	{"TABLE_NAME", mysql.TypeVarchar, 64, 0, nil, nil},
	{"INDEX_NAME", mysql.TypeVarchar, 64, 0, nil, nil},
	{"REGION_ID", mysql.TypeLonglong, 21, 0, nil, nil},
	{"TYPE", mysql.TypeVarchar, 64, 0, nil, nil},
	{"MAX_HOT_DEGREE", mysql.TypeLonglong, 21, 0, nil, nil},
	{"REGION_COUNT", mysql.TypeLonglong, 21, 0, nil, nil},
	{"FLOW_BYTES", mysql.TypeLonglong, 21, 0, nil, nil},
}

The type is long long now.

No. I will change the PR description and API documents.

@nolouch most change didn't influenced API doc because of RAML do not support map (there are several FIXMEs in the doc).

Why does this PR have a lot of changes in API docs?

Why does this PR have a lot of changes in API docs?

The api.html is generated by raml2html. Seems there are some minor change in the theme. The real change of API docs is presented in api.raml, only 4 lines.

Why does this PR have a lot of changes in API docs?

The api.html is generated by raml2html. Seems there are some minor change in the theme. The real change of API docs is presented in api.raml, only 4 lines.

I just ran the scripts/build-api.sh based on your PR, it modified the docs/api.html actually.

Why does this PR have a lot of changes in API docs?

The api.html is generated by raml2html. Seems there are some minor change in the theme. The real change of API docs is presented in api.raml, only 4 lines.

I just ran the scripts/build-api.sh based on your PR, it modified the docs/api.html actually.

Fixed.


ISPN-6805 CacheMgmtInterceptor: use a simplified LongAdder

https://issues.jboss.org/browse/ISPN-6805

@danberindei Motivation? Benchmark results?
Note that as the Stripes are allocated in a loop, these probably lie on the same cache line. You can separate them using a hierarchy of classes, or through @sun.misc.Contended but that requires users to set -XX:-RestrictContended (which makes @Contended unusable in the end).

I am also a bit interested into what was behind this.  I know the allocation for the stripes would be less than all of those long adders, but is this solving some additional contention you saw as well?

Motivation? Benchmark results?

The motivation is simple, JFR was showing a lot of CPU spent in LongAdder :) I didn't actually think I could do better, but apparently LongAdder is too slow to scale up the number of cells.
Some performance numbers for replicated mode reads, running on 8-way Opterons:

10 threads
110 threads
210 threads
310 threads

LongAdder
12162261 reqs/s
12441921 reqs/s
12399660 reqs/s
12286400 reqs/s

Stripes
14168233 reqs/s
14781110 reqs/s
13964451 reqs/s
13664435 reqs/s

Note that as the Stripes are allocated in a loop, these probably lie on the same cache line.
You can separate them using a hierarchy of classes (A with padding, B with variables, C with padding), or through @sun.misc.Contended but that requires users to set -XX:-RestrictContended (which makes @Contended unusable in the end).

I'm pretty sure you only need to use multiple classes when you want to separate 2 fields in the same instance. Regardless of how the JVM reorders fields, it's going to use the same layout for all the instances of a class, so it doesn't matter where exactly the padding is.
Anyway, I don't think padding is needed because cache lines are 64 bytes almost everywhere, and a Stripe is 80 bytes + the object header. I can add a few extra longs, but I only really care about hits and hitTimes, so the others already serve as padding :)
I'm a little more worried about the distribution of the thread id... if it turns out that on Windows or some other OS thread ids are all multiples of 8, that's going to cause some trouble ;)

@danberindei Ok, so LongAdder does not generate enough stripes. You could rather isolate your code to another class than put it directly into CacheMgmtInterceptor.
Wrt padding - that means that one Stripe will be on line 0 and 1, and second on line 1 and 2. And you have a conflict on line 1. Could you try the benchmark with @Contended (activated with -XX:-RestrictContended)?
If you're worried about thread ids, you should use java.lang.Thread#threadLocalRandomProbe in the same way as LongAdder does (using Unsafe). That should be truly random.

LGTM. @wburns @rvansa are you ok with merging it?

Retriggering CI

@slaskawi No, I am still not satisfied with the padding and  I would rather use threadLocalRandomProbe than thread Id. Dan updated one of the commits but did not respond to my last comment (and did not address it in the updated commit).

@danberindei are @rvansa 's comments addressed ?

@danberindei Ok, so LongAdder does not generate enough stripes. You could rather isolate your code to another class than put it directly into CacheMgmtInterceptor.

I've updated the PR to add a generic StripedCounters class, but still, most of the code is CacheMgmtInterceptor-specific.

Wrt padding - that means that one Stripe will be on line 0 and 1, and second on line 1 and 2. And you have a conflict on line 1. Could you try the benchmark with @Contended (activated with -XX:-RestrictContended)?

Yes, in theory you can have a conflict, but you usually don't, because a thread will only update 2 of the counters. Still, I read somewhere that Intel prefetches 2 cache lines at a time, so I've added some more fields for padding. I'm having some trouble with JMH at the moment, but I'll get back to you with some more benchmarks. (On my laptop the 2 versions seem much closer than I remember them, and a definitely a lot closer than on the Opterons...)

If you're worried about thread ids, you should use java.lang.Thread#threadLocalRandomProbe in the same way as LongAdder does (using Unsafe). That should be truly random.

That's not really an option, as we want to wean ourselves off of Unsafe for JDK9. Otherwise I could have used Unsafe for accessing the counter fields as well...
Apparently thread id allocation is platform-independent ATM, but I've added some bit twiddling for future-proofing.

Yes, in theory you can have a conflict, but you usually don't, because a thread will only update 2 of the counters. Still, I read somewhere that Intel prefetches 2 cache lines at a time, so I've added some more fields for padding. I'm having some trouble with JMH at the moment, but I'll get back to you with some more benchmarks. (On my laptop the 2 versions seem much closer than I remember them, and a definitely a lot closer than on the Opterons...)

You still have conflicts in theory there, as the field ordering is not guaranteed in the class. If the padding gets in the middle and some hot fields end up on the beginning and end, there will be conflicts (note that while you've made the class occupy at least 16x8 bytes, it is won't be aligned on the cache line). I don't see why you're so much against the padding > fields > padding class hierarchy. But I won't block this PR; I wash my hands on this.

That's not really an option, as we want to wean ourselves off of Unsafe for JDK9. Otherwise I could have used Unsafe for accessing the counter fields as well...

OK, haven't considered JDK 9, that's a beast to come. Fine then.

I would use more meaningful name of the third commit, but otherwise I am satisfied with the padding.

It took a while @rvansa, but I finally ran the local read benchmark with the extra padding you suggested and it seems to make a small, but visible difference:

80 threads, my laptop
80 threads, cluster10

LongAdder
20243520 reqs/s
17190979 reqs/s

Stripes
22011241 reqs/s
17338521 reqs/s

Extra-padding
22212625 reqs/s
17444039 reqs/s

TBH I'm a bit surprised that I'm seeing a 10% difference on my machine now and almost no difference on cluster10, I'm pretty sure it was the other way around when I first made the change :)
I also got the benchmark results from @rmacor, and they still show ~10% improvement in replicated reads throughput:

iteration 1
iteration 5

master
17536470 reqs/s
17853700 reqs/s

ISPN-6805 (without 3rd commit)
19913319 reqs/s
19860132 reqs/s

@rvansa unless you have more comments, I am going to merge this

@tristantarrant :+1:

Pushed to master, thank.


Prevent over-sizing of group heatmaps in nested columns

closes Caleydo/lineup_app#46
prerequisites:

[x] branch is up-to-date with the branch to be merged with, i.e. develop
[x] build is successful
[x] code is cleaned up and formatted
[x] tested with Firefox 52, Firefox 57+, Chrome 64+, MS Edge 16+

Summary
The over-sizing is caused by the increased row height for aggregated groups.
Reducing the row height to the standard 18px will result in a normal heatmap width for nested columns.

As soon as I increase the height the width of the nested heatmaps increases exceeds the column.

By adding a width:100% the heatmaps stay within their column width.

However, now the gap between the nested columns disappeared. Hence, we must subtract the gap from the width to stay inline with the (nested column) grid.

doesn't work for me, in Chrome I get:

i.e Sass is not translating this variable thus the whole rule is invalid and does nothing.

test file: https://github.com/lineupjs/lineupjs/blob/thinkh/nested-column-stack-renderer/demo/nested_bug_pr133.html

Thanks for pointing out. It's fixed now. I needed to escape the Sass variable.

one thing that I didn't get. when you tested this PR with the buggy CSS selector and still the behavior wasn't there anymore. How can this fix, fix the bug or is there a bug at all?

Well, I just didn't test it they way I should have tested it. 😄

can you please update the test file such that I can reproduce the initial bug

I added the groupBy to the HTML file. You can see the effect when disabling the newly added CSS width:

With the width it is correct:


Bright pink and orange glowstone are white at the center

Not sure if this duplicates #15

1.7.10 version is out of date, the rework is written from scratch, so I close this bug. Bright light will be dropped.


flow-remove-types with gulp task?

Any example of using flow-remove-types with gulp task?

Check out https://github.com/leebyron/flow-remove-types#get-started for both a shell transformer (pipe stdin and get transform on stdout) and javascript API (string in, string out) for running flow-remove-types.
I don't have that much experience with gulp, but if you can call shell scripts or run javascript from it, then probably one of those two approaches will work out just fine!
My best guess is to use gulp's through streams API, perhaps something like this?
var through = require('through2')

gulp.task('flow-remove-types', () => {
  return gulp.src('src/**/*.js')
    .pipe(through.obj((file, enc, cb) => {
      file.contents = new Buffer(flowRemoveTypes(file.contents.toString('utf8')))
      cb(null, file);
    }))
})

To anyone who ends up here - v1.1.0 introduced a breaking change which requires the example code above to be slightly different:
var flowRemoveTypes = require('flow-remove-types')
var through = require('through2')

gulp.task('flow-remove-types', () => {
  return gulp.src('src/**/*.js')
    .pipe(through.obj((file, enc, cb) => {
      file.contents = new Buffer(flowRemoveTypes(file.contents.toString('utf8')).toString())
      cb(null, file);
    }))
})


Refactor path/environment preserver, make it always preserve the PATH

Closes #4251
@segiddins @indirect @RochesterinNYC wdyt?

I changed the backup environment variable prefix from _ORIGINAL_ to BUNDLE_ORIG_, as I thought it's good to have a common prefix for all those variables. If you think there's any downside of such a change I will change it back!

Can you look into the failures?

It's quite difficult to understand how the environment is modified throughout the tests :)
But I think it's good now.
Wdyt? And is it okay to change the backup variable to BUNDLE_ORIG_*?

:+1: looks good to me!

@homu r+

:pushpin: Commit 5131fcd has been approved by indirect

:zap: Test exempted - status


Using Knative Sources ns, to allign w/ all the other CMs

Signed-off-by: Matthias Wessendorf<EMAIL_ADDRESS>Fixes #

Proposed Changes

use knative sources namepsace

Release Note

Docs

/lgtm
/approve
Looking at https://github.com/knative-sandbox/eventing-kafka/pull/95


Unable to "npm run build" remix app after upgrading to 1.8.0

What version of Remix are you using?
1.8.0
Steps to Reproduce
1 - Create new remix app by using "npx create-remix@latest"
2 - Choose where to create the application (app directory)
3 - Select any template. I selected the basic Remix app server
4 - Choose Ts or Js
5 - Run "npm install" inside the app directory
6 - Run "npm run dev" and go to the browser to see the app running in development but instead see the error "Error: Cannot find module '/Users/saud/git/new-remix-app/build/index.js'"
7 - Okay so it requires a build. Do "npm run build" and you get the errors,
app/entry.client.tsx:1:29: ERROR: Could not resolve "@remix-run/react"
app/entry.client.tsx:3:28: ERROR: Could not resolve "react-dom/client"
app/root.tsx:9:7: ERROR: Could not resolve "@remix-run/react"
Expected Behavior
The app should build so we can run it in dev mode.
Actual Behavior
It's throwing an error on "npm run build".
Full cli logs:
Building Remix app in production mode...
✘ [ERROR] Could not resolve "@remix-run/react"
app/entry.client.tsx:1:29:
  1 │ import { RemixBrowser } from "@remix-run/react";
    ╵                              ~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@remix-run/react" here because it'snot listed as a dependency of this package:
../.pnp.cjs:73:33:
  73 │           "packageDependencies": [\
     ╵                                  ~~

You can mark the path "@remix-run/react" as external to exclude it from the bundle, which will remove this error.
✘ [ERROR] Could not resolve "react-dom/client"
app/entry.client.tsx:3:28:
  3 │ import { hydrateRoot } from "react-dom/client";
    ╵                             ~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "react-dom" here because it's not listed as a dependency of this package:
../.pnp.cjs:73:33:
  73 │           "packageDependencies": [\
     ╵                                  ~~

You can mark the path "react-dom/client" as external to exclude it from the bundle, which will remove this error.
✘ [ERROR] Could not resolve "@remix-run/react"
app/root.tsx:9:7:
  9 │ } from "@remix-run/react";
    ╵        ~~~~~~~~~~~~~~~~~~

The Yarn Plug'n'Play manifest forbids importing "@remix-run/react" here because it'snot listed as a dependency of this package:
../.pnp.cjs:73:33:
  73 │           "packageDependencies": [\
     ╵                                  ~~

You can mark the path "@remix-run/react" as external to exclude it from the bundle, which will remove this error.
Build failed with 3 errors:
app/entry.client.tsx:1:29: ERROR: Could not resolve "@remix-run/react"
app/entry.client.tsx:3:28: ERROR: Could not resolve "react-dom/client"
app/root.tsx:9:7: ERROR: Could not resolve "@remix-run/react"
Error
at Object.onCompileFailure (/Users/saud/git/new-remix-app/node_modules/@remix-run/dev/dist/cli/commands.js:174:13)
at Object.compile (/Users/saud/git/new-remix-app/node_modules/@remix-run/dev/dist/compiler/remixCompiler.js:33:134)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Object.build (/Users/saud/git/new-remix-app/node_modules/@remix-run/dev/dist/compiler/build.js:33:3)
at async Object.build (/Users/saud/git/new-remix-app/node_modules/@remix-run/dev/dist/cli/commands.js:169:3)
at async Object.run (/Users/saud/git/new-remix-app/node_modules/@remix-run/dev/dist/cli/run.js:448:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ build: remix build
npm ERR! Exit status 1

Hi, and thank you for filing this issue.
If I follow your reproduction steps and it works. At step 6, simply running npm run dev starts the app correctly.
I see later that you're mentioning Yarn PnP, how come?
Could you please elaborate on your setup and provide the full installation log? Here's mine:
🍺  ~ $ npx create-remix@latest
Need to install the following packages:
  create-remix@latest
Ok to proceed? (y) y
? Where would you like to create your app? bug-4764
? What type of app do you want to create? Just the basics
? Where do you want to deploy? Choose Remix App Server if you're unsure; it's easy to change deployment targets. Remix App Server
? TypeScript or JavaScript? TypeScript
? Do you want me to run `npm install`? Yes
npm WARN deprecated<EMAIL_ADDRESS>This functionality has been moved to @npmcli/fs
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated<EMAIL_ADDRESS>https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated<EMAIL_ADDRESS>This package has been deprecated and is no longer maintained. Please use @rollup/plugin-inject.
npm WARN deprecated<EMAIL_ADDRESS>See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated<EMAIL_ADDRESS>Please use @jridgewell/sourcemap-codec instead

added 1054 packages, and audited 1055 packages in 1m

229 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
💿 That's it! `cd` into "/Users/didou/bug-4764" and check the README for development and deploy instructions!
🍺  ~ $ cd bug-4764/
🍺  ~/bug-4764 $ ls
README.md         app               node_modules      package-lock.json package.json      public            remix.config.js   remix.env.d.ts    tsconfig.json
🍺  ~/bug-4764 $ yar
🍺  ~/bug-4764 $ npm run dev

> dev
> remix dev

Remix App Server started at http://localhost:3000 (http://<IP_ADDRESS>:3000)

Yes I noticed the logs it's using yarn. I have no idea why it's using Yarn plug n play while I created the app using npm. So doing "yarn install" fixes the issue. Anyway, it's a problem with my machine and not with remix. Sorry for the trouble.


PPTX that used add_video which added a wav file (mime_type="audio/x-wav" cant' add again?

I have a pptx which used the add_video function to added a audio file.
I delete the audio file with normal Powerpoint apps, and then If I try to add an video again to the same slide,
It shows the following errors.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[156], line 21
     17 output_path = f"pdf2pptx_1699845973_30650/{index}.wav"
     20 # Add the audio file as a movie
---> 21 audio = slide.shapes.add_movie(
     22     output_path,
     23     left=Inches(0.5),  # 0.5 inches from the left
     24     top=Inches(0.5),   # 0.5 inches from the top
     25     width=Inches(1),   # 1 inch wide
     26     height=Inches(1),  # 1 inch tall
     27     poster_frame_image=None,
     28     mime_type='audio/x-wav'
     29 )
     30 # break
     31
     32 # Get the shape id of the audio shape
     33 r_id = audio._element.xpath(f'//p:nvPicPr/p:cNvPr[@name="{index}.wav"]')[0].get('id')

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/shapes/shapetree.py:543, in SlideShapes.add_movie(self, movie_file, left, top, width, height, poster_frame_image, mime_type)
    514 def add_movie(
    515     self,
    516     movie_file,
   (...)
    522     mime_type=CT.VIDEO,
    523 ):
    524     """Return newly added movie shape displaying video in *movie_file*.
    525
    526     **EXPERIMENTAL.** This method has important limitations:
   (...)
    541     a placeholder for the video.
    542     """
--> 543     movie_pic = _MoviePicElementCreator.new_movie_pic(
    544         self,
    545         self._next_shape_id,
    546         movie_file,
    547         left,
    548         top,
    549         width,
    550         height,
    551         poster_frame_image,
    552         mime_type,
    553     )
    554     self._spTree.append(movie_pic)
    555     self._add_video_timing(movie_pic)

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/shapes/shapetree.py:920, in _MoviePicElementCreator.new_movie_pic(cls, shapes, shape_id, movie_file, x, y, cx, cy, poster_frame_image, mime_type)
    910 @classmethod
    911 def new_movie_pic(
    912     cls, shapes, shape_id, movie_file, x, y, cx, cy, poster_frame_image, mime_type
    913 ):
    914     """Return a new `p:pic` element containing video in *movie_file*.
    915
    916     If *mime_type* is None, 'video/unknown' is used. If
    917     *poster_frame_file* is None, the default "media loudspeaker" image is
    918     used.
    919     """
--> 920     return cls(
    921         shapes, shape_id, movie_file, x, y, cx, cy, poster_frame_image, mime_type
    922     )._pic
    923     return

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/util.py:215, in lazyproperty.__get__(self, obj, type)
    210 value = obj.__dict__.get(self.__name__)
    211 if value is None:
    212     # ---on first access, __dict__ item will absent. Evaluate fget()
    213     # ---and store that value in the (otherwise unused) host-object
    214     # ---__dict__ value of same name ('fget' nominally)
--> 215     value = self._fget(obj)
    216     obj.__dict__[self.__name__] = value
    217 return value

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/shapes/shapetree.py:940, in _MoviePicElementCreator._pic(self)
    934 @lazyproperty
    935 def _pic(self):
    936     """Return the new `p:pic` element referencing the video."""
    937     return CT_Picture.new_video_pic(
    938         self._shape_id,
    939         self._shape_name,
--> 940         self._video_rId,
    941         self._media_rId,
    942         self._poster_frame_rId,
    943         self._x,
    944         self._y,
    945         self._cx,
    946         self._cy,
    947     )

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/shapes/shapetree.py:1008, in _MoviePicElementCreator._video_rId(self)
   1001 @property
   1002 def _video_rId(self):
   1003     """Return the rId of RT.VIDEO relationship to video part.
   1004
   1005     For historical reasons, there are two relationships to the same part;
   1006     one is the video rId and the other is the media rId.
   1007     """
-> 1008     return self._video_part_rIds[1]

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/util.py:215, in lazyproperty.__get__(self, obj, type)
    210 value = obj.__dict__.get(self.__name__)
    211 if value is None:
    212     # ---on first access, __dict__ item will absent. Evaluate fget()
    213     # ---and store that value in the (otherwise unused) host-object
    214     # ---__dict__ value of same name ('fget' nominally)
--> 215     value = self._fget(obj)
    216     obj.__dict__[self.__name__] = value
    217 return value

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/shapes/shapetree.py:998, in _MoviePicElementCreator._video_part_rIds(self)
    991 @lazyproperty
    992 def _video_part_rIds(self):
    993     """Return the rIds for relationships to media part for video.
    994
    995     This is where the media part and its relationships to the slide are
    996     actually created.
    997     """
--> 998     media_rId, video_rId = self._slide_part.get_or_add_video_media_part(self._video)
    999     return media_rId, video_rId

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/parts/slide.py:194, in SlidePart.get_or_add_video_media_part(self, video)
    184 def get_or_add_video_media_part(self, video):
    185     """Return rIds for media and video relationships to media part.
    186
    187     A new |MediaPart| object is created if it does not already exist
   (...)
    192     PowerPoint media embedding strategy.
    193     """
--> 194     media_part = self._package.get_or_add_media_part(video)
    195     media_rId = self.relate_to(media_part, RT.MEDIA)
    196     video_rId = self.relate_to(media_part, RT.VIDEO)

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/package.py:44, in Package.get_or_add_media_part(self, media)
     38 def get_or_add_media_part(self, media):
     39     """Return a |MediaPart| object containing the media in *media*.
     40
     41     If a media part for this media bytestream ("file") is already present
     42     in this package, it is reused, otherwise a new one is created.
     43     """
---> 44     return self._media_parts.get_or_add_media_part(media)

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/package.py:207, in _MediaParts.get_or_add_media_part(self, media)
    200 def get_or_add_media_part(self, media):
    201     """Return a |MediaPart| object containing the media in *media*.
    202
    203     If this package already contains a media part for the same
    204     bytestream, that instance is returned, otherwise a new media part is
    205     created.
    206     """
--> 207     media_part = self._find_by_sha1(media.sha1)
    208     if media_part is None:
    209         media_part = MediaPart.new(self._package, media)

File ~/personal_assistant/venv/lib/python3.8/site-packages/pptx/package.py:220, in _MediaParts._find_by_sha1(self, sha1)
    213 """Return |MediaPart| object having *sha1* hash or None if not found.
    214
    215 All media parts belonging to this package are considered. A media
    216 part is identified by the SHA1 hash digest of its bytestream
    217 ("file").
    218 """
    219 for media_part in self:
--> 220     if media_part.sha1 == sha1:
    221         return media_part
    222 return None

AttributeError: 'Part' object has no attribute 'sha1'

Which part should I check for it?

Hmm, looks like a part (file in the .pptx zip archive) is not getting instantiated as a MediaPart (which is what has the .sha1 property) and is defaulting back to the generic Part class.
This is where the Part subclass is determined: https://github.com/scanny/python-pptx/blob/master/pptx/init.py#L51-L60
This is where membership in _MediaParts is determined: https://github.com/scanny/python-pptx/blob/master/pptx/init.py#L51-L60
You can see those determine what counts as a media-part differently.
So I think there's a type that needs to be added to the first list above. Also, in retrospect, it would probably be worth thinking through how to use the same single mechanism for determining what's a media part and what isn't.

@scanny Thank for replying.
I'm adding the audio wav file by using the add_movie method. From your reference, it seems like it works on like mp4, mov and etc. of the movie file.
By adding the audio with mime_type="audio/x-wav" would cause this errors maybe?

Yep, could be. Try adding that type to the first file and see if that fixes it. Interestingly there is no audio/x-wav entry in pptx.opc.constants.CONTENT_TYPES, so you might want to add it there.
I think root cause here is that audio was not considered when adding that feature so we didn't add audio content types or test cases at the time, but I expect much of the implementation is the same between audio and video, not sure about player behavior etc.


Problem when generating bundle file from master

I was trying to use todoApp (https://github.com/vigour-io/todoApp) following the readme file instruction. I run into a problem when I tried to generate the bundle file using the gaston command.
But if I use the #dev branch it works good. Probably something is wrong with master branch ?

for now, let's just use the #dev branch always


fix(migrate): continue with null value instead of  TypeError: Cannot read property 'trim' of undefined.

This can happen in the presence of void elements.
PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/ngneat/transloco/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?
[x] Bugfix

Does this PR introduce a breaking change?
[ ] Yes
[x] No

@fowl2 Thanks for the PR, what was the case that you encountered that?

IIRC, I had a void element with a i18n attribute. An  I think. Technically illegal html, but it was there and stopped the whole migration.

Can you please add a comment with the case? to make this change clearer


Support Update By Query with slices auto

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html#docs-update-by-query-slice  According to the doc, we should be able to use the value auto with the param slices

@Philippus I don't understand the error on the scala-2_12 CI. Can you help me?

Actually, it partially breaks compatibility because I assume most of people use the DSL
def slices(slices: Int): UpdateByQueryRequest = copy(slices = Some(NumericSlices(slices))).
We can also add an implicit conversion Int -> Slices in the scope to avoid breaking changes.
The drawback having 2 different params for the slices is that a user can use both in same time, so which one should we keep? It can be confusing.
What do you think?

We can add a new parameter like numericOrAutoSlices: Option[Slices] = None and deprecate the old parameter. The new parameter should take precedence over the deprecated one. The methods then can look like this:
  def slices(slices: Int): ReindexRequest = copy(numericOrAutoSlices = NumericSlices(slices).some)
  def slicesAuto(): ReindexRequest = copy(numericOrAutoSlices = AutoSlices.some)

and in the handlers we could do something like this:
      request.numericOrAutoSlices.foreach {
        case AutoSlices            => params.put("slices", "auto")
        case NumericSlices(slices) => params.put("slices", slices.toString)
      }
      if (request.numericOrAutoSlices.isEmpty)
        request.slices.foreach(params.put("slices", _))