Clarify use of enum LRAStatus as entity parameter

Fixes #347

@xstefank again, since this change is not contentious, I think we can merge with just the one approval.

@mmusgrov sure, I was just waiting for more reviews.


Ability to completely disable Issues, Pull Requests, Milestones

Feature Description
In the config file, it is possible to disable the Stars function so that it is impossible to put stars to repositories.
It would be nice if it were possible to disable the functions specified in the title if you host your own Gitea and do not use these functions.
Screenshots

This would be great to have!

Duplicate of https://github.com/go-gitea/gitea/issues/4750?


[Word] (typo) Fix

Docs Build status updates of commit 9a77c0e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/reference/requirement-sets/word-api-requirement-sets.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


"Failed to find user by email" error

Hi all,
I am trying to run load-test using manual loadtesting against an existing cluster strategy.
When I run the loadtest all command,I get the following errror(repeated several times with different mail ids) :
{"level":"error","ts":1564551977.214884,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user499@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
Please provide any possible solutions.

@reflog would you have thoughts on why this might occur?

@sourabkumarkeshri - any chance you post the full log? It's hard to investigate based on just one line

@reflog - Here is the full log.And the email error is repeated many times(say about 500 times) and then the final messsage is displayed.
sourabkumarkeshri-C02YQAS7LVCG:mattermost-load-test sourabkumarkeshri$ loadtest all
{"level":"info","ts":1565009526.252393,"caller":"loadtest/loadtest.go:111","msg":"Running test","test":"all"}
{"level":"info","ts":1565009529.677979,"caller":"loadtest/run.go:57","msg":"Registered loadtest instance","instance_id":"ahe7youzppfxdq7pco46o51j6r","entity_start_num":0,"seed":1565009529000}
{"level":"info","ts":1565009529.678061,"caller":"loadtest/run.go:63","msg":"Settings","tag":"report","configuration":{"LoadtestEnviromentConfig":{"NumTeams":1,"NumChannelsPerTeam":300,"NumPrivateChannelsPerTeam":100,"NumDirectMessageChannels":2000,"NumGroupMessageChannels":50,"NumUsers":1000,"NumTeamSchemes":1,"NumChannelSchemes":1,"NumEmoji":2000,"NumPlugins":1,"PercentHighVolumeChannels":0.2,"PercentMidVolumeChannels":0.5,"PercentLowVolumeChannels":0.3,"PercentUsersHighVolumeChannel":0.1,"PercentUsersMidVolumeChannel":0.003,"PercentUsersLowVolumeChannel":0.0002,"PercentHighVolumeTeams":0.2,"PercentMidVolumeTeams":0.5,"PercentLowVolumeTeams":0.3,"PercentUsersHighVolumeTeams":0.9,"PercentUsersMidVolumeTeams":0.5,"PercentUsersLowVolumeTeams":0.1,"HighVolumeTeamSelectionWeight":3,"MidVolumeTeamSelectionWeight":2,"LowVolumeTeamSelectionWeight":1,"HighVolumeChannelSelectionWeight":1,"MidVolumeChannelSelectionWeight":3,"LowVolumeChannelSelectionWeight":1,"NumPosts":20000000,"PostTimeRange":2600000,"ReplyChance":0.3,"PercentCustomSchemeTeams":0.2,"PercentCustomSchemeChannels":0.1},"ConnectionConfiguration":{"ServerURL":"http://localhost:8065","WebsocketURL":"ws://localhost:8065","PProfURL":"http://localhost:8067/debug/pprof","DriverName":"mysql","DataSource":"mmuser:mostest@tcp(localhost:3306)/mattermost_test?charset=utf8mb4,utf8&readTimeout=20s&writeTimeout=20s&timeout=20s","DBEndpoint":"","LocalCommands":true,"SSHHostnamePort":"<IP_ADDRESS>:22","SSHUsername":"mmuser","SSHPassword":"null","SSHKey":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test/.ssh/id_rsa","MattermostInstallDir":"/Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test","ConfigFileLoc":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/config/config.json","AdminEmail":"sourab@sourab.com","AdminPassword":"So@12345678","SkipBulkload":true,"MaxIdleConns":100,"MaxIdleConnsPerHost":128,"IdleConnTimeoutMilliseconds":90000},"UserEntitiesConfiguration":{"TestLengthMinutes":20,"NumActiveEntities":500,"ActionRateMilliseconds":240000,"ActionRateMaxVarianceMilliseconds":1500,"EnableRequestTiming":false,"ChannelLinkChance":0.02,"UploadImageChance":0.01,"LinkPreviewChance":0.2,"CustomEmojiChance":0.2,"CustomEmojiReactionChance":0.05,"SystemEmojiReactionChance":0.1,"NeedsProfilesByIdChance":0.22,"NeedsProfilesByUsernameChance":0.38,"NeedsProfileStatusChance":0.8,"DoStatusPolling":true,"RandomizeEntitySelection":false},"ResultsConfiguration":{"PProfDelayMinutes":15,"PProfLength":240},"LogSettings":{"EnableConsole":true,"ConsoleJson":true,"ConsoleLevel":"INFO","EnableFile":true,"FileJson":true,"FileLevel":"INFO","FileLocation":"loadtest.log"}},"instance_id":"ahe7youzppfxdq7pco46o51j6r"}
{"level":"info","ts":1565009529.6783879,"caller":"loadtest/run.go:74","msg":"Setting up server."}
{"level":"info","ts":1565009529.67842,"caller":"loadtest/setup_server.go:35","msg":"Connecting to local app server"}
{"level":"info","ts":1565009529.6787689,"caller":"loadtest/setup_server.go:45","msg":"Testing ability to run commands."}
{"level":"info","ts":1565009529.678838,"caller":"loadtest/local_command_runner.go:28","msg":"Running local command: ./bin/mattermost version"}
{"level":"error","ts":1565009529.680531,"caller":"loadtest/setup_server.go:47","msg":"Unable to connect issue mattermost commands. Continuing anyway... Got Output: fork/exec ./bin/mattermost: no such file or directory : "}
{"level":"info","ts":1565009529.685229,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009529.765787,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009530.769669,"caller":"loadtest/setup_server.go:62","msg":"Checking configuration parameters."}
{"level":"info","ts":1565009530.772255,"caller":"loadtest/setup_server.go:246","msg":"EnableOpenServer is true"}
{"level":"info","ts":1565009530.772313,"caller":"loadtest/setup_server.go:257","msg":"MaxUsersPerTeam is 50000"}
{"level":"info","ts":1565009530.7723389,"caller":"loadtest/setup_server.go:268","msg":"MaxChannelsPerTeam is 50000"}
{"level":"info","ts":1565009530.772359,"caller":"loadtest/setup_server.go:279","msg":"EnableIncomingWebhooks is true"}
{"level":"info","ts":1565009530.772382,"caller":"loadtest/setup_server.go:281","msg":"Disabling only admin integrations for loadtest. 1/2"}
{"level":"info","ts":1565009530.784023,"caller":"loadtest/setup_server.go:312","msg":"Disabling only admin integrations for loadtest. 2/2"}
{"level":"info","ts":1565009530.789064,"caller":"loadtest/setup_server.go:344","msg":"PluginSettings.Enable is true"}
{"level":"info","ts":1565009530.789106,"caller":"loadtest/setup_server.go:349","msg":"PluginSettings.EnableUploads is true"}
{"level":"info","ts":1565009530.7891352,"caller":"loadtest/setup_server.go:68","msg":"Setting up plugins."}
{"level":"info","ts":1565009530.789231,"caller":"loadtest/setup_server.go:79","msg":"Checking if plugin is installed: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.7927208,"caller":"loadtest/setup_server.go:86","msg":"Plugin found, removing: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794621,"caller":"loadtest/setup_server.go:94","msg":"Removed plugin: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794658,"caller":"loadtest/setup_server.go:100","msg":"Uploading plugin."}
{"level":"info","ts":1565009530.852906,"caller":"loadtest/setup_server.go:105","msg":"Enabling plugin."}
{"level":"info","ts":1565009530.896698,"caller":"loadtest/setup_server.go:111","msg":"Generating users for loadtest."}
{"level":"info","ts":1565009530.925386,"caller":"loadtest/setup_server.go:177","msg":"Clearing caches"}
{"level":"info","ts":1565009530.943114,"caller":"loadtest/setup_server.go:190","msg":"Found teams","teams":4}
{"level":"info","ts":1565009530.945677,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"haha"}
{"level":"info","ts":1565009530.945716,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"haha","channels":2}
{"level":"info","ts":1565009530.947923,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"okay"}
{"level":"info","ts":1565009530.947965,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"okay","channels":3}
{"level":"info","ts":1565009530.95008,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"ok"}
{"level":"info","ts":1565009530.9501178,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"ok","channels":2}
{"level":"info","ts":1565009530.953275,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"team-2"}
{"level":"info","ts":1565009530.953395,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"team-2","channels":4}
{"level":"info","ts":1565009530.956019,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009531.0372639,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009532.037778,"caller":"loadtest/run.go:136","msg":"Logging in as users."}
{"level":"error","ts":1565009532.0444129,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user225@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.0489442,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user330@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048983,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user150@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048961,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user375@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.04912,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user60@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
Error: run test failed: Failed to login as any users
Usage:
loadtest all [flags]
Flags:
-h, --help   help for allo`urabkumarkeshri-C02YQAS7LVCG:mattermost-load-test sourabkumarkeshri$ loadtest all
{"level":"info","ts":1565009526.252393,"caller":"loadtest/loadtest.go:111","msg":"Running test","test":"all"}
{"level":"info","ts":1565009529.677979,"caller":"loadtest/run.go:57","msg":"Registered loadtest instance","instance_id":"ahe7youzppfxdq7pco46o51j6r","entity_start_num":0,"seed":1565009529000}
{"level":"info","ts":1565009529.678061,"caller":"loadtest/run.go:63","msg":"Settings","tag":"report","configuration":{"LoadtestEnviromentConfig":{"NumTeams":1,"NumChannelsPerTeam":300,"NumPrivateChannelsPerTeam":100,"NumDirectMessageChannels":2000,"NumGroupMessageChannels":50,"NumUsers":1000,"NumTeamSchemes":1,"NumChannelSchemes":1,"NumEmoji":2000,"NumPlugins":1,"PercentHighVolumeChannels":0.2,"PercentMidVolumeChannels":0.5,"PercentLowVolumeChannels":0.3,"PercentUsersHighVolumeChannel":0.1,"PercentUsersMidVolumeChannel":0.003,"PercentUsersLowVolumeChannel":0.0002,"PercentHighVolumeTeams":0.2,"PercentMidVolumeTeams":0.5,"PercentLowVolumeTeams":0.3,"PercentUsersHighVolumeTeams":0.9,"PercentUsersMidVolumeTeams":0.5,"PercentUsersLowVolumeTeams":0.1,"HighVolumeTeamSelectionWeight":3,"MidVolumeTeamSelectionWeight":2,"LowVolumeTeamSelectionWeight":1,"HighVolumeChannelSelectionWeight":1,"MidVolumeChannelSelectionWeight":3,"LowVolumeChannelSelectionWeight":1,"NumPosts":20000000,"PostTimeRange":2600000,"ReplyChance":0.3,"PercentCustomSchemeTeams":0.2,"PercentCustomSchemeChannels":0.1},"ConnectionConfiguration":{"ServerURL":"http://localhost:8065","WebsocketURL":"ws://localhost:8065","PProfURL":"http://localhost:8067/debug/pprof","DriverName":"mysql","DataSource":"mmuser:mostest@tcp(localhost:3306)/mattermost_test?charset=utf8mb4,utf8&readTimeout=20s&writeTimeout=20s&timeout=20s","DBEndpoint":"","LocalCommands":true,"SSHHostnamePort":"<IP_ADDRESS>:22","SSHUsername":"mmuser","SSHPassword":"null","SSHKey":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test/.ssh/id_rsa","MattermostInstallDir":"/Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test","ConfigFileLoc":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/config/config.json","AdminEmail":"sourab@sourab.com","AdminPassword":"So@12345678","SkipBulkload":true,"MaxIdleConns":100,"MaxIdleConnsPerHost":128,"IdleConnTimeoutMilliseconds":90000},"UserEntitiesConfiguration":{"TestLengthMinutes":20,"NumActiveEntities":500,"ActionRateMilliseconds":240000,"ActionRateMaxVarianceMilliseconds":1500,"EnableRequestTiming":false,"ChannelLinkChance":0.02,"UploadImageChance":0.01,"LinkPreviewChance":0.2,"CustomEmojiChance":0.2,"CustomEmojiReactionChance":0.05,"SystemEmojiReactionChance":0.1,"NeedsProfilesByIdChance":0.22,"NeedsProfilesByUsernameChance":0.38,"NeedsProfileStatusChance":0.8,"DoStatusPolling":true,"RandomizeEntitySelection":false},"ResultsConfiguration":{"PProfDelayMinutes":15,"PProfLength":240},"LogSettings":{"EnableConsole":true,"ConsoleJson":true,"ConsoleLevel":"INFO","EnableFile":true,"FileJson":true,"FileLevel":"INFO","FileLocation":"loadtest.log"}},"instance_id":"ahe7youzppfxdq7pco46o51j6r"}
{"level":"info","ts":1565009529.6783879,"caller":"loadtest/run.go:74","msg":"Setting up server."}
{"level":"info","ts":1565009529.67842,"caller":"loadtest/setup_server.go:35","msg":"Connecting to local app server"}
{"level":"info","ts":1565009529.6787689,"caller":"loadtest/setup_server.go:45","msg":"Testing ability to run commands."}
{"level":"info","ts":1565009529.678838,"caller":"loadtest/local_command_runner.go:28","msg":"Running local command: ./bin/mattermost version"}
{"level":"error","ts":1565009529.680531,"caller":"loadtest/setup_server.go:47","msg":"Unable to connect issue mattermost commands. Continuing anyway... Got Output: fork/exec ./bin/mattermost: no such file or directory : "}
{"level":"info","ts":1565009529.685229,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009529.765787,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009530.769669,"caller":"loadtest/setup_server.go:62","msg":"Checking configuration parameters."}
{"level":"info","ts":1565009530.772255,"caller":"loadtest/setup_server.go:246","msg":"EnableOpenServer is true"}
{"level":"info","ts":1565009530.772313,"caller":"loadtest/setup_server.go:257","msg":"MaxUsersPerTeam is 50000"}
{"level":"info","ts":1565009530.7723389,"caller":"loadtest/setup_server.go:268","msg":"MaxChannelsPerTeam is 50000"}
{"level":"info","ts":1565009530.772359,"caller":"loadtest/setup_server.go:279","msg":"EnableIncomingWebhooks is true"}
{"level":"info","ts":1565009530.772382,"caller":"loadtest/setup_server.go:281","msg":"Disabling only admin integrations for loadtest. 1/2"}
{"level":"info","ts":1565009530.784023,"caller":"loadtest/setup_server.go:312","msg":"Disabling only admin integrations for loadtest. 2/2"}
{"level":"info","ts":1565009530.789064,"caller":"loadtest/setup_server.go:344","msg":"PluginSettings.Enable is true"}
{"level":"info","ts":1565009530.789106,"caller":"loadtest/setup_server.go:349","msg":"PluginSettings.EnableUploads is true"}
{"level":"info","ts":1565009530.7891352,"caller":"loadtest/setup_server.go:68","msg":"Setting up plugins."}
{"level":"info","ts":1565009530.789231,"caller":"loadtest/setup_server.go:79","msg":"Checking if plugin is installed: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.7927208,"caller":"loadtest/setup_server.go:86","msg":"Plugin found, removing: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794621,"caller":"loadtest/setup_server.go:94","msg":"Removed plugin: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794658,"caller":"loadtest/setup_server.go:100","msg":"Uploading plugin."}
{"level":"info","ts":1565009530.852906,"caller":"loadtest/setup_server.go:105","msg":"Enabling plugin."}
{"level":"info","ts":1565009530.896698,"caller":"loadtest/setup_server.go:111","msg":"Generating users for loadtest."}
{"level":"info","ts":1565009530.925386,"caller":"loadtest/setup_server.go:177","msg":"Clearing caches"}
{"level":"info","ts":1565009530.943114,"caller":"loadtest/setup_server.go:190","msg":"Found teams","teams":4}
{"level":"info","ts":1565009530.945677,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"haha"}
{"level":"info","ts":1565009530.945716,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"haha","channels":2}
{"level":"info","ts":1565009530.947923,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"okay"}
{"level":"info","ts":1565009530.947965,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"okay","channels":3}
{"level":"info","ts":1565009530.95008,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"ok"}
{"level":"info","ts":1565009530.9501178,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"ok","channels":2}
{"level":"info","ts":1565009530.953275,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"team-2"}
{"level":"info","ts":1565009530.953395,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"team-2","channels":4}
{"level":"info","ts":1565009530.956019,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009531.0372639,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009532.037778,"caller":"loadtest/run.go:136","msg":"Logging in as users."}
{"level":"error","ts":1565009532.0444129,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user225@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.0489442,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user330@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048983,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user150@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048961,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user375@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.04912,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user60@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
Error: run test failed: Failed to login as any users
Usage:
loadtest all [flags]
Flags:
-h, --help   help for all

@reflog - Here is the full log.And the error email is repeated several times(say about 500) and then the final message is displayed.
`sourabkumarkeshri-C02YQAS7LVCG:mattermost-load-test sourabkumarkeshri$ loadtest all
{"level":"info","ts":1565009526.252393,"caller":"loadtest/loadtest.go:111","msg":"Running test","test":"all"}
{"level":"info","ts":1565009529.677979,"caller":"loadtest/run.go:57","msg":"Registered loadtest instance","instance_id":"ahe7youzppfxdq7pco46o51j6r","entity_start_num":0,"seed":1565009529000}
{"level":"info","ts":1565009529.678061,"caller":"loadtest/run.go:63","msg":"Settings","tag":"report","configuration":{"LoadtestEnviromentConfig":{"NumTeams":1,"NumChannelsPerTeam":300,"NumPrivateChannelsPerTeam":100,"NumDirectMessageChannels":2000,"NumGroupMessageChannels":50,"NumUsers":1000,"NumTeamSchemes":1,"NumChannelSchemes":1,"NumEmoji":2000,"NumPlugins":1,"PercentHighVolumeChannels":0.2,"PercentMidVolumeChannels":0.5,"PercentLowVolumeChannels":0.3,"PercentUsersHighVolumeChannel":0.1,"PercentUsersMidVolumeChannel":0.003,"PercentUsersLowVolumeChannel":0.0002,"PercentHighVolumeTeams":0.2,"PercentMidVolumeTeams":0.5,"PercentLowVolumeTeams":0.3,"PercentUsersHighVolumeTeams":0.9,"PercentUsersMidVolumeTeams":0.5,"PercentUsersLowVolumeTeams":0.1,"HighVolumeTeamSelectionWeight":3,"MidVolumeTeamSelectionWeight":2,"LowVolumeTeamSelectionWeight":1,"HighVolumeChannelSelectionWeight":1,"MidVolumeChannelSelectionWeight":3,"LowVolumeChannelSelectionWeight":1,"NumPosts":20000000,"PostTimeRange":2600000,"ReplyChance":0.3,"PercentCustomSchemeTeams":0.2,"PercentCustomSchemeChannels":0.1},"ConnectionConfiguration":{"ServerURL":"http://localhost:8065","WebsocketURL":"ws://localhost:8065","PProfURL":"http://localhost:8067/debug/pprof","DriverName":"mysql","DataSource":"mmuser:mostest@tcp(localhost:3306)/mattermost_test?charset=utf8mb4,utf8&readTimeout=20s&writeTimeout=20s&timeout=20s","DBEndpoint":"","LocalCommands":true,"SSHHostnamePort":"<IP_ADDRESS>:22","SSHUsername":"mmuser","SSHPassword":"null","SSHKey":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test/.ssh/id_rsa","MattermostInstallDir":"/Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/dist/mattermost-load-test","ConfigFileLoc":"Users/sourabkumarkeshri/gocode/src/github.com/mattermost/mattermost-server/config/config.json","AdminEmail":"sourab@sourab.com","AdminPassword":"So@12345678","SkipBulkload":true,"MaxIdleConns":100,"MaxIdleConnsPerHost":128,"IdleConnTimeoutMilliseconds":90000},"UserEntitiesConfiguration":{"TestLengthMinutes":20,"NumActiveEntities":500,"ActionRateMilliseconds":240000,"ActionRateMaxVarianceMilliseconds":1500,"EnableRequestTiming":false,"ChannelLinkChance":0.02,"UploadImageChance":0.01,"LinkPreviewChance":0.2,"CustomEmojiChance":0.2,"CustomEmojiReactionChance":0.05,"SystemEmojiReactionChance":0.1,"NeedsProfilesByIdChance":0.22,"NeedsProfilesByUsernameChance":0.38,"NeedsProfileStatusChance":0.8,"DoStatusPolling":true,"RandomizeEntitySelection":false},"ResultsConfiguration":{"PProfDelayMinutes":15,"PProfLength":240},"LogSettings":{"EnableConsole":true,"ConsoleJson":true,"ConsoleLevel":"INFO","EnableFile":true,"FileJson":true,"FileLevel":"INFO","FileLocation":"loadtest.log"}},"instance_id":"ahe7youzppfxdq7pco46o51j6r"}
{"level":"info","ts":1565009529.6783879,"caller":"loadtest/run.go:74","msg":"Setting up server."}
{"level":"info","ts":1565009529.67842,"caller":"loadtest/setup_server.go:35","msg":"Connecting to local app server"}
{"level":"info","ts":1565009529.6787689,"caller":"loadtest/setup_server.go:45","msg":"Testing ability to run commands."}
{"level":"info","ts":1565009529.678838,"caller":"loadtest/local_command_runner.go:28","msg":"Running local command: ./bin/mattermost version"}
{"level":"error","ts":1565009529.680531,"caller":"loadtest/setup_server.go:47","msg":"Unable to connect issue mattermost commands. Continuing anyway... Got Output: fork/exec ./bin/mattermost: no such file or directory : "}
{"level":"info","ts":1565009529.685229,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009529.765787,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009530.769669,"caller":"loadtest/setup_server.go:62","msg":"Checking configuration parameters."}
{"level":"info","ts":1565009530.772255,"caller":"loadtest/setup_server.go:246","msg":"EnableOpenServer is true"}
{"level":"info","ts":1565009530.772313,"caller":"loadtest/setup_server.go:257","msg":"MaxUsersPerTeam is 50000"}
{"level":"info","ts":1565009530.7723389,"caller":"loadtest/setup_server.go:268","msg":"MaxChannelsPerTeam is 50000"}
{"level":"info","ts":1565009530.772359,"caller":"loadtest/setup_server.go:279","msg":"EnableIncomingWebhooks is true"}
{"level":"info","ts":1565009530.772382,"caller":"loadtest/setup_server.go:281","msg":"Disabling only admin integrations for loadtest. 1/2"}
{"level":"info","ts":1565009530.784023,"caller":"loadtest/setup_server.go:312","msg":"Disabling only admin integrations for loadtest. 2/2"}
{"level":"info","ts":1565009530.789064,"caller":"loadtest/setup_server.go:344","msg":"PluginSettings.Enable is true"}
{"level":"info","ts":1565009530.789106,"caller":"loadtest/setup_server.go:349","msg":"PluginSettings.EnableUploads is true"}
{"level":"info","ts":1565009530.7891352,"caller":"loadtest/setup_server.go:68","msg":"Setting up plugins."}
{"level":"info","ts":1565009530.789231,"caller":"loadtest/setup_server.go:79","msg":"Checking if plugin is installed: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.7927208,"caller":"loadtest/setup_server.go:86","msg":"Plugin found, removing: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794621,"caller":"loadtest/setup_server.go:94","msg":"Removed plugin: com.mattermost.sample-plugin"}
{"level":"info","ts":1565009530.794658,"caller":"loadtest/setup_server.go:100","msg":"Uploading plugin."}
{"level":"info","ts":1565009530.852906,"caller":"loadtest/setup_server.go:105","msg":"Enabling plugin."}
{"level":"info","ts":1565009530.896698,"caller":"loadtest/setup_server.go:111","msg":"Generating users for loadtest."}
{"level":"info","ts":1565009530.925386,"caller":"loadtest/setup_server.go:177","msg":"Clearing caches"}
{"level":"info","ts":1565009530.943114,"caller":"loadtest/setup_server.go:190","msg":"Found teams","teams":4}
{"level":"info","ts":1565009530.945677,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"haha"}
{"level":"info","ts":1565009530.945716,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"haha","channels":2}
{"level":"info","ts":1565009530.947923,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"okay"}
{"level":"info","ts":1565009530.947965,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"okay","channels":3}
{"level":"info","ts":1565009530.95008,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"ok"}
{"level":"info","ts":1565009530.9501178,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"ok","channels":2}
{"level":"info","ts":1565009530.953275,"caller":"loadtest/setup_server.go:207","msg":"Found town-square","team":"team-2"}
{"level":"info","ts":1565009530.953395,"caller":"loadtest/setup_server.go:213","msg":"Found team channels","team":"team-2","channels":4}
{"level":"info","ts":1565009530.956019,"caller":"loadtest/client.go:85","msg":"Successfully pinged server","server_url":"http://localhost:8065"}
{"level":"info","ts":1565009531.0372639,"caller":"loadtest/client.go:114","msg":"Successfully logged in","email":"sourab@sourab.com","roles":"system_admin system_user"}
{"level":"info","ts":1565009532.037778,"caller":"loadtest/run.go:136","msg":"Logging in as users."}
{"level":"error","ts":1565009532.0444129,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user225@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.0489442,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user330@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048983,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user150@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.048961,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user375@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
{"level":"error","ts":1565009532.04912,"caller":"loadtest/client.go:48","msg":"Failed to find user by email","email":"success+user60@simulator.amazonses.com","error":": Unable to find the user<EMAIL_ADDRESS>sql: no rows in result set"}
Error: run test failed: Failed to login as any users
Usage:
loadtest all [flags]
Flags:
-h, --help   help for all`

@sourabkumarkeshri - there are several issues here:
{"level":"info","ts":1565009529.678838,"caller":"loadtest/local_command_runner.go:28","msg":"Running local command: ./bin/mattermost version"}
{"level":"error","ts":1565009529.680531,"caller":"loadtest/setup_server.go:47","msg":"Unable to connect issue mattermost commands. Continuing anyway... Got Output: fork/exec ./bin/mattermost: no such file or directory : "}

This means that your mattermost installation cannot be access by loadtest. Usually - ssh issue.
I've alose noticed that your SSHKey config param is missing a leading '/' character. That might be it.

Thanks @reflog .
I corrected the SSHKey config param as you said.
But the issue still exists with the same error.

Hm. I see that you also have SkipBulkLoad set to true. Any specific reason? Because these users that the loadtest can't find (like<EMAIL_ADDRESS>are being generated by bulk loading


CI failed because it's unable to install make

Failure logs:
Failures
 - make - make not installed. The package was not found with the source(s) listed.
 Source(s): 'https://chocolatey.org/api/v2/'
 NOTE: When you specify explicit sources, it overrides default sources.
If the package version is a prerelease and you didn't specify `--pre`,
 the package may not be found.
Please see https://chocolatey.org/docs/troubleshooting for more
 assistance.
The command "(
    set -Eeuo pipefail
    if [[ "$TRAVIS_OS_NAME" == "windows" ]]; then
      choco install -y make mingw
      export PATH="/c/tools/mingw64/bin:/c/ProgramData/chocolatey/bin;$PATH"
    fi
  )
  " failed and exited with 1 during .

/kind failing-test

no errors found recently
/close


[SPARK-11437] [PySpark] Don't .take when converting RDD to DataFrame with provided schema

When creating a DataFrame from an RDD in PySpark, createDataFrame calls .take(10) to verify the first 10 rows of the RDD match the provided schema. Similar to https://issues.apache.org/jira/browse/SPARK-8070, but that issue affected cases where a schema was not provided.
Verifying the first 10 rows is of limited utility and causes the DAG to be executed non-lazily. If necessary, I believe this verification should be done lazily on all rows. However, since the caller is providing a schema to follow, I think it's acceptable to simply fail if the schema is incorrect.
@marmbrus We chatted about this at SparkSummitEU. @davies you made a similar change for the infer-schema path in https://github.com/apache/spark/pull/6606

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Test build #1969 has started for PR 9392 at commit a7c395f.

LGTM

Test build #1969 has finished for PR 9392 at commit a7c395f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged into master, thanks!


A question about equation 4

Hi,
Your work is great. I have a little question about equation 4. In your paper, you say: For a negative teacher response, if the student response is higher than the target value, it should be reduced, but if the student response is lower than the target value, it doesn’t need to be increased since negatives are equally blocked by ReLU regardless of their values.
If a student response is negative but higher than the target value, the loss still exists according to equation (4). But the negative student response could be equally blocked by ReLU.  I do not understand the reason to put positive loss in the above scenario. Could you please give me some help? Thanks a lot.

Hi
The reason is not simple but I will do my best to explain it.
At first, distillation try to make teacher response with inferior student. Therefore, distillation loss decrease during training but, the loss can't be zero in general cases.
Second, gradients of L2 loss decrease when the distance is close to zero.
If we use L2 distance to zero (instead of negative target), student responses will be reduced until zero. but, it doesn't need to be below zero. Because gradient for small positive response is also very small and there are many gradients from other non-zero loss part.
Actually, the zero target value is not good to make negative student response in our experiments.
Therefore, we need small negative target instead of zero to make negative student response.
So, we make a negative margin in eq 2 and use the L2 loss to the margin in eq 3. It is an approximation of L0 loss function only for negative responses.
This topic was discussed in my previous work "Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons". So, I didn't include details about it on the paper. I'm sorry for that.

Thanks for your patient reply. It is very helpful for me to understand the idea of the paper.


Uploading image taking too much memory on version 2.2.0 and newer while its normal on version 2.0.6 and below

Details:

I am using image_picker: ^0.8.1+2 for capturing image in full resolution.
Storing image file on the temporary directory.
Uploading it to the server using http.MultipartRequest("POST", uri).
In response, the server returns the link of that stored image.
Now using this link, I am showing images in app using Image.network. (even if i comment this memory issue is still there)
I am repeating these steps for each image.

Before uploading images, the app takes 50-60MB of memory, but after just 5-6 images, it takes 400-500MB of memory and crashes on devices with low RAM.
This is happening on Flutter version 2.2.0 and later while on versions before that, it's working fine.
Flutter doctor -v response:
[✓] Flutter (Channel stable, 2.2.3, on Linux, locale en_IN)
• Flutter version 2.2.3 at /home/daffolap-951/flutter
• Framework revision f4abaa0735 (12 days ago), 2021-07-01 12:46:11 -0700
• Engine revision 241c87ad80
• Dart version 2.13.4
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at /home/daffolap-951/Android/Sdk
• Platform android-30, build-tools 30.0.2
• Java binary at: /snap/android-studio/113/android-studio/jre/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.8+0-b944-P17168821)
• All Android licenses accepted.
[✓] Chrome - develop for the web
• Chrome at google-chrome
[✓] Android Studio (version 4.2)
• Android Studio at /snap/android-studio/113/android-studio
• Flutter plugin version 58.0.1
• Dart plugin version 202.8531
• Java version OpenJDK Runtime Environment (build 11.0.8+0-b944-P17168821)
[✓] VS Code (version 1.53.2)
• VS Code at /usr/share/code
• Flutter extension version 3.20.0
[✓] Connected device (2 available)
• RMX2081 (mobile) • 13ec0c2f • android-arm64  • Android 11 (API 30)
• Chrome (web)     • chrome   • web-javascript • Google Chrome 91.0.4472.114
• No issues found!

Flutter version 2.2.3

Flutter version 2.0.6

Hi @shivamsahu15,
Thanks for filing the issue, Could please share a minimal reproducible code to better investigate the issue.
Thank you.

Hi, I am able to reproduce the same problem using this code.
main.dart
`import 'package:flutter/material.dart';
import 'package:gallery/home_screen.dart';
void main() {
runApp(MyApp());
}
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: HomeScreen(),
);
}
}`
home_screen.dart
`import 'package:flutter/material.dart';
import 'package:gallery/image_screen.dart';
class HomeScreen extends StatefulWidget {
const HomeScreen({Key? key}) : super(key: key);
@override
_HomeScreenState createState() => _HomeScreenState();
}
class _HomeScreenState extends State {
@override
Widget build(BuildContext context) {
return ListView.builder(itemBuilder: (context,index){
return ImageScreenButton(index: index);
}, itemCount: 10);
}
}
class ImageScreenButton extends StatelessWidget {
final int index;
const ImageScreenButton({Key? key, required this.index}) : super(key: key);
@override
Widget build(BuildContext context) {
return Padding(
padding: const EdgeInsets.all(16.0),
child: ElevatedButton(
onPressed: () {
Navigator.push(
context, MaterialPageRoute(builder: (context) => ImageScreen(screenIndex: index,)));
},
child: Text('Image List $index')),
);
}
}
`
image_screen.dart
`import 'package:flutter/material.dart';
class ImageScreen extends StatefulWidget {
final int screenIndex;
const ImageScreen({Key? key,required this.screenIndex}) : super(key: key);
@override
_ImageScreenState createState() => _ImageScreenState();
}
class _ImageScreenState extends State {
List imageUrl = [
'https://images.pexels.com/photos/257360/pexels-photo-257360.jpeg?auto=compress&cs=tinysrgb&dpr=3&h=750&w=1260',
'https://images.pexels.com/photos/417074/pexels-photo-417074.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/443446/pexels-photo-443446.jpeg?auto=compress&cs=tinysrgb&dpr=3&h=750&w=1260',
'https://images.pexels.com/photos/3617500/pexels-photo-3617500.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/33109/fall-autumn-red-season.jpg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/247431/pexels-photo-247431.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/2754200/pexels-photo-2754200.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/1586298/pexels-photo-1586298.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/3690511/pexels-photo-3690511.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/33688/delicate-arch-night-stars-landscape.jpg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940'
];
List imageUrl2 = [
'https://images.pexels.com/photos/2753453/pexels-photo-2753453.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260',
'https://images.pexels.com/photos/1499629/pexels-photo-1499629.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/169789/pexels-photo-169789.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/1205302/pexels-photo-1205302.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/2682770/pexels-photo-2682770.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/2908971/pexels-photo-2908971.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/4695929/pexels-photo-4695929.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/2092895/pexels-photo-2092895.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/2579912/pexels-photo-2579912.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/1819656/pexels-photo-1819656.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
'https://images.pexels.com/photos/4549674/pexels-photo-4549674.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940'
];
@override
void dispose() {
imageCache!.clear();
super.dispose();
}
@override
Widget build(BuildContext context) {
return PageView.builder(itemBuilder: (context,index){
return Image.network(widget.screenIndex%2==0?imageUrl[index]:imageUrl2[index]);
},itemCount: widget.screenIndex%2==0?imageUrl.length:imageUrl2.length,);
}
}
`
On flutter 2.0.6

On flutter 2.2.3

I've also made a repo of this code, if you want
https://github.com/shivamsahu15/flutter-issue-sample/

Hi @shivamsahu15, I looked at your repo, I could not see any image picker related code which is basically uploading the images to the server as described in the original post. I ran the sample app but could not see memory usage going beyond 60mb on an average the most it went up to was 75 mb for dart/flutter. Let me know if I am missing anything here.

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.4 20F71 darwin-arm, locale en-GB)
    • Flutter version 2.2.3 at /Users/mahesh/Documents/flutter
    • Framework revision f4abaa0735 (4 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.1.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 57.0.5
    • Dart plugin version 211.7233

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.24.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 91.0.4472.164

• No issues found!

Thank you

just run it on flutter version 2.0.6, the Dart/Flutter will take less than 15-20 MB, I am concerned with the difference in Dart/Flutter memory between these flutter versions.

@shivamsahu15, I ran your sample code on v2.0.6 on Xiaomi k2O pro, But I don't see much difference in the memory usage than the one posted here

You also mentioned that you are experiencing the crash on low ram devices could you please share what device did you test on with API version and the amount of ram the device has.
[✓] Flutter (Channel unknown, 2.0.6, on macOS 11.4 20F71 darwin-arm, locale en-GB)
    • Flutter version 2.0.6 at /Users/mahesh/Documents/flutter
    • Framework revision 1d9032c7e1 (3 months ago), 2021-04-29 17:37:58 -0700
    • Engine revision 05e680e202
    • Dart version 2.12.3

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] IntelliJ IDEA Community Edition (version 2021.1.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 57.0.5
    • Dart plugin version 211.7233

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.24.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.107

• No issues found!

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.


ROX-9652: Version port to Postgres

Description
A detailed explanation of the changes in your PR.
Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.
In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

Okay this one is unmergeable until we figure out the migrator

@connorgorman I checked with @c-du and she currently anticipates version working basically the same as it does before postgres where we simply use a sequence number.  Obviously that may evolve, but I think a simple port, will suffice initially.

/test push-images
/test gke-postgres-qa-e2e-tests


[$125] Web - Inconsistent behavior of tag disappearing and reappearing when requesting money

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.55-0
Reproducible in staging?: y
Reproducible in production?: y
Issue reported by: Applause - Internal team
Action Performed:
Preconditions:
In OldDot under admin, create a Collect group policy, enable tags and add the tag "Child: Tom", and it should be the only tag in the policy. Add the employee to the policy. Setup OldDot Collect Policy:
https://sites.google.com/applausemail.com/applause-expensifyproject/wiki-guides/newdot-categories?authuser=0

Open the WS chat report
Click on the plus icon inside the compose box > Request money > Enter an amount > Click on the tag
Select the tag "Child: Tom" (Note that it is already selected by default)
Observe the behavior of the "Tag" field after you selected the tag and went back
Click on FAB > Request money > Enter an amount > Select the collect policy created in OD
Click on the Tag field (note that the "Child: Tom" tag is already selected)
Click on the "Child: Tom" tag again
Observe the behavior of the tag field and compare it with the behavior seen on step 4

Expected Result:
The behavior seen on step 4 should be the same for step 8 as well
Actual Result:
The tag field first becomes empty and there is a noticeable delay before the selected tag "Child: Tom" shows up again. This behavior is different from the one seen on step 4, where there is no noticeable delay.
Workaround:
n/a
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/67b27697-92b0-43ed-b620-3ad3a2788daa
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~010aab18242acca3aa
Upwork Job ID:<PHONE_NUMBER>762127360
Last Price Increase: 2024-04-08

@joekaufmanexpensify I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors.

We think that this bug might be related to #wave-collect - Release 1

Proposal
Please re-state the problem that we are trying to solve in this issue.
The tag field first becomes empty and there is a noticeable delay before the selected tag "Child: Tom" shows up again. This behavior is different from the one seen on step 4, where there is no noticeable delay.
What is the root cause of that problem?
We have a useEffect here that auto select the tag if the tag required and there's only one tag to select.
https://github.com/Expensify/App/blob/8970d088ab3a31ff1f625f254dcb4c6e32b452ee/src/components/MoneyTemporaryForRefactorRequestConfirmationList.js#L506-L515
But In IOURequestStepTag we have the logic to de-select the tag if we select the selected tag. So after going back to confirm page the tag will disappear and then reappear immediately
This bug is the same for category
What changes do you think we should make in order to solve the problem?
Since we auto select the tag if it's required and  there's only one tag to select, we should disable user go to IOURequestStepTag page in this case. That makes more seen because we can't edit this field, it's the same behavior the amount field in distance request flow. To do that we can create variable like isAutoSelectTag that will be true if the tag if it's required and  there's only one tag to select
const [updatedTagsString, isAutoSelectTags] = useMemo(() => {
        let updatedTagsString = TransactionUtils.getTag(transaction);
        let isAutoSelectTags = {};
        policyTagLists.forEach((tagList, index) => {
            const enabledTags = _.filter(tagList.tags, (tag) => tag.enabled);
            const isTagListRequired = isUndefined(tagList.required) ? false : tagList.required && canUseViolations;
            if (!isTagListRequired || enabledTags.length !== 1 || TransactionUtils.getTag(transaction, index)) {
                isAutoSelectTags[tagList.name] = isTagListRequired && enabledTags.length === 1;
                return;
            }

            isAutoSelectTags[tagList.name] = true;
            updatedTagsString = IOUUtils.insertTagIntoTransactionTagsString(updatedTagsString, enabledTags[0] ? enabledTags[0].name : '', index);
        });
        return [updatedTagsString, isAutoSelectTags];
    }, [policyTagLists, transaction, policyTags, isTagRequired, canUseViolations]);

useEffect(() => {
    if (!isTagRequired || !isAutoSelectTags) {
        return ;
    }
    if (updatedTagsString !== TransactionUtils.getTag(transaction) && updatedTagsString) {
        IOU.setMoneyRequestTag(transaction.transactionID, updatedTagsString);
    }
}, [isTagRequired, updatedTagsString, isAutoSelectTags]);

Update the condition to show the right icon here to !isReadOnly && !isAutoSelectTags
https://github.com/Expensify/App/blob/8970d088ab3a31ff1f625f254dcb4c6e32b452ee/src/components/MoneyTemporaryForRefactorRequestConfirmationList.js#L815
And update the onPress to do nothing if we auto select the tag
https://github.com/Expensify/App/blob/8970d088ab3a31ff1f625f254dcb4c6e32b452ee/src/components/MoneyTemporaryForRefactorRequestConfirmationList.js#L819
onPress={() => {
    if (isAutoSelectTags[name]) {
        return;
    }
    Navigation.navigate(
        ROUTES.MONEY_REQUEST_STEP_TAG.getRoute(CONST.IOU.ACTION.CREATE, iouType, index, transaction.transactionID, reportID, Navigation.getActiveRouteWithoutParams()),
    )
}}

Do the same way for category
What alternative solutions did you explore? (Optional)
NA

@luacmartins could you reproduce this? I'm not able to reproduce the behavior shown in OP at all. Maybe the performance is a bit slower on the second attempt to request, but nothing like the above video. If this is just impacting one tag, feels like pretty minor edge case, no? Not sure like it feels worth prioritizing.
https://github.com/Expensify/App/assets/54866469/3600a793-b8e8-4a83-9790-08b381cd9c60

I haven't tried to reproduce, but it seems like it might happen on slower connections so maybe we can try to throttle it in the browser inspector. Also, I looked at the issue and this doesn't seem related to simplified collect, so I'm removing that from the project.

So it seems like this is the expected behavior? Tags are required and we have only one tag, so it should be selected and user's can't deselect it. Maybe in that case we should disable the editing of tags altogether.

I meant if tags are required and users only have a single tag, we auto select it and that the MenuItem should be disabled in that case and users shouldn't even be able to see the tags selection page. @trjExpensify can you confirm?

We definitely could do that, though my 2c is not sure it's really worth prioritizing that change. Do we think it's going to be common that a workspace is going to be using tags, has them required, and only has one enabled? That doesn't strike me as a super common configuration.
Even if it does come up, the product works now. You can't remove a tag when one is required. I'd probably vote to do nothing, see if this impacts anyone, and then fix it then (if so).

@luacmartins @joekaufmanexpensify I think we should disable user go to the tag/category selection page in confirm page when it's auto selected.

It's the same behavior when we create a distance request the amount field also can't be edited in confirm page.

https://github.com/Expensify/App/issues/38744#issuecomment-2015801370

I meant if tags are required and users only have a single tag, we auto select it and the MenuItem should be disabled, so users shouldn't even be able to see the tags selection page. @trjExpensify can you confirm?

Waiting for confirmation about the Expected Result from @trjExpensify. Should we also chime in the Design team?

@Krishna2323 @nkdengineer Would you want to update your proposals to match the expectations.

Proposals pending update

I'm going to defer Carlos on the difficulty of fixing the bug. But looking at OP, it's a pretty niche case. We are generally trying to be better about adjusting the bounty to the difficulty of the issue. So unless Carlos thinks it's a difficult bug, I am probably inclined to leave it as is.

I feel like the solution to this issue is pretty simple so I think the current bounty is applicable here.

@Krishna2323 Can you please add more details to your solution.

The useEffect should run only once. For that we can create a ref to track or we can remove the dependencies. We also need to do the same with other fields like category.

I am not sure if this will solve the issue.
Similarly, for your alternative solution, more details will be appreciated. It gives general idea of the solution.

We will create a ref isAutoSelectedTag/isAutoSelectedCategory and set it to true when we automatically select a tag and in useEffect we will check if sAutoSelectedTag/isAutoSelectedCategory.current is true or not and if true we will return.

Rather than this, can we check in the submit function, if tags are required and only one tag is present, don't de-select the tag. Just a thought. What do you think?

Optionally, we can only auto select tad if it is required.

I think we already do that. We auto select tag and category only if it is required. And we would want to continue the same behavior.

I think my alternative proposal already covers the all required parts, we just need to check if tags are required and if the length is 1, in that case we will navigateBack. Pls let me know if you want to see all the code changes.
We need to make changes here:

More information and update in alternative proposal will be very helpful. Please remove the bits which are not required and include your solution. Please do state where will you make those changes and what approach will you use.
Your alternative solution says we will require ref and useEffect, right? My suggestion is

Rather than this, can we check in the submit function, if tags are required and only one tag is present, don't de-select the tag. Just a thought. What do you think?

Please update your solution and let me know.

Still discussing proposals

Issue not reproducible during KI retests. (First week)

@Krishna2323 Any update on this?

Seems like maybe this can't be reproduced anymore? Going to work on verifying that

Pls ignore the alternative section, we don't need to disallow user to deselect the tag even if it is one and required because when we have multiple tags and it is required, we don't run any validation and user can create the request without the tag.

I kind of agree with you. @Krishna2323 Do we allow user to deselect tag if there is only one tag from Transaction Previews?
@luacmartins @trjExpensify Do you think we should allow user to deselect tag even if it is one and is required as we do allow user to create the request without tag when there are multiple tags?

@luacmartins @trjExpensify Do you think we should allow user to deselect tag even if it is one and is required as we do allow user to create the request without tag when there are multiple tags?

which then would add a violation, right? Yeah, @luacmartins, I'm kind of souring on not letting them deselect when there's one value and required, as you can when there are multiple values and required. So let's just lean on the violation consistently in both cases. 👍

Yea, I agree we could let them deselect and just have the violation show up and be consistent on all flows.

@Krishna2323 IMO removing the dependencies from the second argument of the useEffect and simply passing [] should do the job
https://github.com/Expensify/App/blob/9c8ac2975aa898a2fe90da287a4e11b187b4e5b1/src/components/MoneyTemporaryForRefactorRequestConfirmationList.js#L521
- }, [policyTagLists, transaction, policyTags, isTagRequired, canUseViolations]);
+ }, []

@Krishna2323  Would you like to update your proposal to reflect this change?

Issue not reproducible during KI retests. (Second week)

Proposal from @Krishna2323 looks good to me.
🎀 👀 🎀 C+ reviewed

Great, let's just proceed with removing the dependencies array. Thanks for the convo here!

Great. TY!

All set to issue payment! We need to issue the following payments:

@Krishna2323 - $125 for PR via Upwork.
@sobitneupane - $125 for C+ via NewDot.

@Krishna2323 $125 sent and contract ended!

@sobitneupane please request $125 via NewDot, and confirm here once complete.

Bumped @sobitneupane in slack here.

Requested payment in newDot

Great. TY! All set. Payment summary is here.

$125 approved for @sobitneupane


Instructor Notes

This PR implements arbitrary instructor notes on a per-content page basis. The notes can be accessed via icons on the path admin page or on the individual content piece edit pages. They are viewable on the content view pages as well.
Note that these are only stored in localStorage and thus not shareable among instructors directly. However, there is an import/export feature for doing so (a stopgap solution for now).

Looks good :shipit:


#5186: UTWH neighbourhood distance >2 support

Config data has been made completely generic such that cores can address
any other core, i.e. any neighborhood distance.  There is now a single
UTWH kernel that is parameterized via compile time args to specify which
of, padding, local, remote data each risc DM core should perform.
Padding config data is still special cased since its access pattern is
different (single source row, always local).
Perf is better for almost every case in resnet.

@mywoodstock / @tt-nshanker, I have both the new and old paths supported!  To simplify the diff process I figured I'd commit the additions first, and then remove the old path as a separate change.  LMK if you want me to do otherwise.
@tt-nshanker, also if you don't mind taking a look to see if I setup the program caching correctly.

Follow on removal of fixed neighborhood path: https://github.com/tenstorrent-metal/tt-metal/pull/5650

Closing, PR moved here https://github.com/tenstorrent-metal/tt-metal/pull/5650


[BUG] winget source won't appear if path has space + typo

Please confirm these before moving forward

[X] I have searched for my issue and not found a work-in-progress/duplicate/resolved issue.
[X] I have tested that this issue has not been fixed in the latest (beta or stable) release.
[X] I have checked the FAQ section for solutions
[X] This issue is about a bug (if it is not, please use the correct template)

Describe your issue
bucket is used instead of source and if the path has space the sources aren't shown

changing the path to C:\Users\Pedro\Downloads\WingetUI\PackageManagers\winget-cli_x64\winget.exe avoids the problem. running winget.exe source list  from the folder of the image also shows the sources correctly
Steps to reproduce the issue
No response
WingetUI Log
not needed I guess

Package Managers Logs
not needed I guess

Relevant information
No response
Screenshots and videos
No response

Yes, the bundled one
De: Martí Climent @.>
Enviada: 20 September 2023 13:53
Para: marticliment/WingetUI @.>
Cc: pedronovaes @.>; Author @.>
Assunto: Re: [marticliment/WingetUI] [BUG] winget source won't appear if path has space + typo (Issue #1376)
if the path has space
You mean the path where winget is located?
—
Reply to this email directly, view it on GitHub https://github.com/marticliment/WingetUI/issues/1376#issuecomment-1727573571 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ARTBYHTMSB22ACPEZ7XZPILX3LKIDANCNFSM6AAAAAA464PFUU .
You are receiving this because you authored the thread.  https://github.com/notifications/beacon/ARTBYHVKEZ3JJLR4RRHC6TDX3LKIDA5CNFSM6AAAAAA464PFUWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTG7CXEG.gif Message ID: @.*** @.***> >

This issue was fixed on 98647fb


Returning backend.set_learning_phase

The main problem is outlined here.
In short:
Is it possible to return keras.backend.set_learning_phase and keras.backend.learning_phase? Or another functionality to disable training mode for pre-trained model used like this:
...
inp = layers.Input((None, None, 3))

backbone = applications.vgg16.VGG16(include_top=False,
                                    input_tensor=inp,
                                    weights=None)
x = backbone.output
...

I created the following class for these purposes, but it no longer works:
class TrainingMode:
    def __init__(self, training_mode=False):
        assert isinstance(training_mode, bool)
        self.learning_phase = 0 if training_mode else 1

    def __enter__(self):
        self.external_learning_phase = backend.learning_phase()
        backend.set_learning_phase(self.learning_phase)

    def __exit__ (self, type, value, traceback):
        backend.set_learning_phase(self.external_learning_phase)

Thank you.

This is a Keras 1 concept that was deprecated many years ago (around the release of TF 2), and was entirely removed from Keras 3. Instead, you can pass training=True/False when calling layers.

When I use
from tf_keras import applications, layers, models, utils

inp = layers.Input((None, None, 3))
backbone = applications.vgg16.VGG16(include_top=False,
                                    weights=None)
x = backbone.call(inp, training=False)

backbone.trainable = False

x = layers.GlobalAveragePooling2D()(x)
x = layers.Dense(10, activation='relu')(x)
out = layers.Dense(1, activation='sigmoid')(x)

model = models.Model(inp, out)

model.summary()

everything works fine, but if I replace tf_keras(2.16.0) with keras(3.3.3), I get:
> ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces `keras.layers` and `keras.operations`). You are likely doing something like:
>
> ```
> x = Input(...)
> ...
> tf_fn(x)  # Invalid.
> ```
>
> What you should do instead is wrap `tf_fn` in a layer:
>
> ```
> class MyLayer(Layer):
>     def call(self, x):
>         return tf_fn(x)
>
> x = MyLayer()(x)

What am I doing wrong?


pytest doesn't return print statements when passing the "-s" flag

What is happening?
Typically, for debugging tests, I will place print statements in my code and see what they return in the CLI. However, pytest doesn't seem to be returning these print statements anymore.
How can we reproduce the issue?
Add a print statement to any test.
What should happen?
It should return the print statement, but won't.
Environment
N/A
Any additional context?
N/A

It looks like pytest-xdist just doesn't support the -s flag -- see discussion in https://github.com/pytest-dev/pytest-xdist/issues/354.
But there might be workarounds...

Yeah it looks like the print statement needs to send the message to stderr for it to show.
You can add something like print("Message here", file=sys.__stderr__) or you can run the test manually with pytest (without running ./checks/pytest_.py) to avoid the multi threaded tests.
We can also add an flag to run without xdist if needed, but I think the above solutions are easier.

You can add something like print("Message here", file=sys.__stderr__)

I was using print("Message here", file=sys.stderr) initially which didn't seem to work, but can confirm print("Message here", file=sys.__stderr__) does work. Though as you mention, I usually just use pytest ... for specific debugging.

We can also add an flag to run without xdist if needed, but I think the above solutions are easier.
Requested in https://github.com/Infleqtion/client-superstaq/issues/990

linking @PalashGoiporia's proposal (which makes sense to me): https://github.com/Infleqtion/client-superstaq/issues/990#issuecomment-2231300487
(imo we should also treat the --incremental flag like passing in specific tests)


chatbot fails with error WindowsPath found

Looking into gs://shark_tank/llama2_7b/unsharded/vmfb/llama2_7b_int4_vulkan_rdna3_7900.vmfb
[DEBUG] vmfb not found (search path: llama2_7b_int4_vulkan_rdna3_7900.vmfb)
Looking into gs://shark_tank/llama2_7b/unsharded/mlir/llama2_7b_int4.mlirbc
[DEBUG] mlir found at F:\Download\Web\llama2_7b_int4.mlir
vulkan
Configuring for device:vulkan
Using target triple -iree-vulkan-target-triple=rdna3-7900-windows from command line args
Traceback (most recent call last):
File "gradio\queueing.py", line 388, in call_prediction
File "gradio\route_utils.py", line 219, in call_process_api
File "gradio\blocks.py", line 1437, in process_api
File "gradio\blocks.py", line 1123, in call_function
File "gradio\utils.py", line 503, in async_iteration
File "gradio\utils.py", line 496, in anext
File "anyio\to_thread.py", line 33, in run_sync
File "anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
File "anyio_backends_asyncio.py", line 807, in run
File "gradio\utils.py", line 479, in run_sync_iterator_async
File "gradio\utils.py", line 629, in gen_wrapper
File "ui\stablelm_ui.py", line 228, in chat
File "apps\language_models\scripts\vicuna.py", line 1302, in init
File "apps\language_models\scripts\vicuna.py", line 1757, in compile
File "shark\shark_inference.py", line 206, in save_module
return export_iree_module_to_vmfb(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "shark\iree_utils\compile_utils.py", line 504, in export_iree_module_to_vmfb
flatbuffer_blob = compile_module_to_flatbuffer(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "shark\iree_utils\compile_utils.py", line 324, in compile_module_to_flatbuffer
flatbuffer_blob = ireec.compile_file(
^^^^^^^^^^^^^^^^^^^
File "iree\compiler\tools\core.py", line 256, in compile_file
File "iree\compiler\tools\binaries.py", line 200, in invoke_immediate
File "iree\compiler\tools\binaries.py", line 81, in init
TypeError: sequence item 1: expected str instance, WindowsPath found

nodai_shark_studio_20231009_984.exe

This is fixed by https://github.com/nod-ai/SHARK/pull/1901, though I thought that stable release didn't have this issue. Sorry about that.
Chatbot should be quite stable and performant on our latest pre-release: https://github.com/nod-ai/SHARK/releases/download/20231110.1021/nodai_shark_studio_20231110_1021.exe

(you can reuse the llama2 .mlir so you dont have to re-download it, just drop it in the same folder as the new .exe)

Closing this issue for now. We can reopen it if you see this issue on versions after #1901 .


Loss drops and stays at zero

Currently I am working on a medium size project where I need to fine-tune several models.
Right now I am training DeepSeekCoder 6.7b and while training I noticed a strange behavior, where the training loss drops directly to 0 and stays there, no matter how long the training is continued.
The dataset used contains 10k samples. Each sample results in 1024 tokens without the need of a padding.
The dataset was deduplicated using MinHash with a threshold of 0.7.
I use rslora in each linear layers except the head, because the goal is to improve the performance of another lesser known programming language.
Quantization: 4bit
Learning rate: 2e-5
Lr scheduler: Constant and Cosine tried out
Rank: 126 and alpha of 16 (since more would not fit into the VRAM)
Batch size: Tried 16 and 32
Lora dropout: 0.1
Weight decay: 0.01
Max grad norm: 0.3 (which might be overblown, not that I think about it)
When using a batch size of 16 it takes around 114 steps to go from a loss of 1.2 to 0.8 and then it drops to 0.0
When using a batch size of 32 it takes around 57 steps to go from a loss of 1.2 to 1.0 and then it drops to 0.0 again
In another experiment I tried to lower the rank to 64 where the first 200 steps didn't result in a loss of 0.0, but with that approach the loss did not improve significantly anymore.
Might this be a problem with the loss calculation of the library, or am I using it wrong?

Try adding a validation dataset to see if the val loss is actually decreasing - a loss of 0 could mean NaN gradients (try decreasing learning rate to 2e-6 for eg), or it could mean overfitting

Oh my if the eval loss goes to NaN, it means it diverged. Set max_grad_norm = 0.3 in the training args

I was under the assumption that the eval_loss goes to NaN because the grad_norm went to NaN in the step before. I will give it a try. Thank you for the response

Somehow this situation got stranger. I noticed that there is a special token which I do not want to train, therefore I remove it using this code:
def remove_name_tags(example):
example["content"] = example["content"].replace("", "")
return example
data["train"] = data["train"].map(remove_name_tags)
(This train column is later split into train and test)
Now the gradient often reaches a NaN after the first step, and if it does not, then the first validation is guaranteed to be a NaN.
I reduced the model complexity further and reduced the lr to no avail

In the past two weeks I haven't had much time to solve the issue.
Yesterday however, I noticed, that I forgot to set the fp16 variable to True. The reason for instability in training therefore can't be more obvious.
Thank you for this awesome project and your help.

Oh great you solved it!


Not showing all nodes

Hi,
I'm facing the following issue that even if I've configured the bootstrap.servers property with a list including all Kafka nodes, in the GUI sometimes I see only two of the nodes and after restarting the other node that I've not seen previously.
I should mention that KafkaHQ is deployed on the same machine with the node that is not shown in the list.
My application.yaml file is the following one:
kafkahq:
connections:
cluster-1:
properties:
bootstrap.servers: "<IP_ADDRESS>:9092,<IP_ADDRESS>:9092,<IP_ADDRESS>:9092"
schema-registry:
url: "http://<IP_ADDRESS>:8081"
Topic list display options (optional)
topic:
page-size: 25 # number of topics per page (default : 25)
default-view: HIDE_INTERNAL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM)
internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)
- "^_.$"
- "^._schemas$"
- "^.connect-config$"
- "^.connect-offsets$1"
- "^.connect-status$"
stream-regexps: # list of regexp to be considered as internal stream topic
- "^.-changelog$"
- "^.-repartition$"
- "^.-rekey$"
Topic display data options (optional)
topic-data:
sort: OLDEST # default sort order (OLDEST, NEWEST) (default: OLDEST)
size: 50 # max record per page (default: 50)
poll-timeout: 1000 # The time, in milliseconds, spent waiting in poll if data is not available in the buffer.
Schama list display options (optional)
schema:
page-size: 25 # number of schemas per page (default : 25)
Auth & Roles (optional)
security:
default-roles: # Roles available for all the user even unlogged user
- topic/read
- topic/insert
- topic/delete
- topic/config/update
- node/read
- node/config/update
- topic/data/read
- topic/data/insert
- topic/data/delete
- group/read
- group/delete
- group/offsets/update
- registry/read
- registry/insert
- registry/update
- registry/delete
- registry/version/delete
- connect/read
- connect/insert
- connect/update
- connect/delete
- connect/state/update
# Basic auth configuration
basic-auth:
  user: # Username
    password: pass # Password in sha256
    roles: # Role for current users
      - topic/read
      - group/read
      - group/delete

The problem occurred because the properties: listeners & advertised.listeners were not set for the specific node. But it is strange because at a certain point in time KafkaHQ showed me only this specific node without the other two nodes.

Hello,
sorry for the late response.
I think it's normal as I understand connection protocol for kafka, I will pick 1 random server on the list.

If it pick a misconfigured node, kafka cli will only see this one
If it pick one of 2 others one, kafka cli will only see 2 nodes and not the misconfigured.


ERROR [ExceptionsHandler] An API error occurred: not_authed

Hi Everyone,
Hope you guys are doing great. I am using this package to send messages to the slack using SlackServide.postMessage() method. While calling it I am getting this issue. Can anyone help to resolve that issue?
Here is the sample code:
constructor(private readonly slackService: SlackService) {}

this.slackService.postMessage(payload);

Hello,
Which version are you using? API, Webhook or Google Cloud Logger?

I'm using API version.

Okey, have you made sure the API key is valid? Maybe using curl or Postman just to test?

Yes, I am using the BOT Token to send the message. I verified that using Postman it's working fine. Here is how actually I configured it with the module.
SlackModule.forRoot({
      type: 'api',
      token: process.env.SLACK_BOT_TOKEN,
      isGlobal: true,
    }),


Netlify delployment error

Describe the bug
I haven't been able to create the Netlify preview link this week. I have previously used it successfully but at some point last week it stopped working.
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-sdm8kn
Steps to reproduce
Creat preview with Netlify
Expected behavior
A link similar to
https://relaxed-blancmange-91b99d.netlify.app/
Screen Recording / Screenshot

Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 124
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 104808268,
  "usedJSHeapSize": 97322364,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = jhillbht
Chat ID = 5e5ca95950f7

Additional context
Error:
An unknown error occured while running an action.

Hi @jhillbht,
Appreciate the feedback! The team is aware of this issue and tracking it here: #499.


Stack based builds in Github Actions broken

See https://github.com/input-output-hk/bech32/actions/runs/615972176

Do you have a little bit more context? Any particular branch or revision you've encountered this? Is it related to the ghc-8.10.2 bump?
(PS: please don't assign tickets to me ^.^)


attach_disk() /detach_disk() multipath support BasedOnShellScript

In this PR, a bunch of files had been changed in order to support multipath in SL eCPI,  what follows are the steps involving attach volume with multipath enabled

vm.hasMulitPathToolBasedOnShellScript
vm.getAllowedHostCredential
vm.backupOpenIscsiConfBasedOnShellScript
vm.writeOpenIscsiInitiatornameBasedOnShellScript
vm.writeOpenIscsiConfBasedOnShellScript
vm.restartOpenIscsiBasedOnShellScript
vm.discoveryOpenIscsiTargetsBasedOnShellScript
vm.getIscsiDeviceNamesBasedOnShellScript

Hi @maximilien ,
I had uploaded the changes related with attach_disk() /detach_disk() multipath support, could you please review the change set, thanks!

OK cool. Cheers :beers:


_SUBSCRIPTION_CHANGED  called after STOP_SUBSCRIPTION

I'm pretty sure I've got things wire up properly but the changed and ready events are being called after I stop a subscription. My STOP_SUBSCRIPTION reducer removed the currentUser from the store, but then after that, the _SUBSCRIPTION_CHANGED is called again, re-populating the currentUser from the payload.
Is this expected behaviour? Have I done something wrong?
Great package, let me know if you need a PR to fix this. Thanks!

Hi Alex, please share your code so we can figure out what's happening!

Hey @samybob1, thanks for your response. So essentially I'm subscribing to the user collection which works perfectly:
startSubscription({
  key: USER_SELF_SUB,
  get: () => User.find({ _id }).fetch(),
  subscribe: () => Meteor.subscribe(USER_SELF_SUB, _id),
});

But when logging out, I call stopSubscription:
    Meteor.logout(err => {
      if (!err) {
        // Stop watching the redux-middleware subscription
        dispatch(stopSubscription(USER_SELF_SUB));
        dispatch(push('/login'));
      }
    });

and see the STOP_SUBSCRIPTION action (the reducer un-sets the currentUser from the store, but then, after that,  I see, USER_SELF_SUBSCRIPTION_CHANGED with payload: Array(1) which re-sets the currentUser which obviously isn't what I want!

I thought I'd drop in my whole AppContainer file so you can see how I'm starting the subscription. I think this is where the problem lies...
import { Meteor } from 'meteor/meteor';
import { compose } from 'redux';
import { connect } from 'react-redux';
import { withRouter } from 'react-router-dom';
import { withTracker } from 'meteor/react-meteor-data';

import App from '../components/App';
import { fetchUser } from '../modules/actions';
import { SET_LOADING_STATE } from '../modules';

const mapStateToProps = ({ app }) => ({
  loading: app.loading,
  currentUser: app.currentUser,
});

const mapDispatchToProps = dispatch => ({
  init: _id => dispatch(fetchUser(_id)),
  stopLoading: () =>
    dispatch({
      type: SET_LOADING_STATE,
      state: false,
    }),
});

// We need to do this crap becuase we need to wait for
// Meteor to get the current session otherwise Meteor.userId
// isn't available
// https://docs.meteor.com/api/accounts.html#Meteor-loggingIn
const meteorData = withTracker(({ currentUser, init, stopLoading }) => {
  // wait for Meteor to find current user session
  // and then init the store
  if (!currentUser && !Meteor.loggingIn()) {
    if (Meteor.userId()) {
      init(Meteor.userId());
    } else {
      // We get to here when it is an anonymous user.
      // We don't need to load anything in this case,
      // so just stop the loading state
      stopLoading();
    }
  }

  // We have to return an object for withTracker to work
  return {};
});

const reduxData = connect(mapStateToProps, mapDispatchToProps);

export default compose(withRouter, reduxData, meteorData)(App);

It must be something to do with withTracker re-running after the Meteor.logout(). Hmm...

Nope, it's not that as START_SUBSCRIPTION isn't being called again. Sorry for the spam updates but it's helping me debug it step by step!

Here's the actions that get triggered on logout:
STOP_SUBSCRIPTION @ 22:29:43.271
APP/SET_LOADING_STATE @ 22:29:43.271
APP/SET_LOADING_STATE @ 22:29:43.320
@@router/LOCATION_CHANGE @ 22:29:43.357
USER_SELF_SUBSCRIPTION_CHANGED @ 22:29:43.365
USER_SELF_SUBSCRIPTION_READY @ 22:29:43.381

I'm stuck now, not sure what else I can check...

Okay so I came up with a fix but I still don't think meteor-redux-middlewares is behaving as I'd like...
I moved the Meteor.userId() into the actual thunk rather than just passing in a string, so the the subscription has a reactive userId, which when Meteor.logout() is called, becomes null and the subscription returns Array(0) which is fine.
I'm still not sure why USER_SELF_SUBSCRIPTION_READY and CHANGED are bring called post-STOP_SUBSCRIPTION... Maybe you have some insight?
const fetchUser = () =>
  startSubscription({
    key: USER_SELF_SUB,
    get: () => User.find({ _id: Meteor.userId() }).fetch(),
    subscribe: () => Meteor.subscribe(USER_SELF_SUB, Meteor.userId()),
  });

Okay... So I woke up this morning with the realisation that meteor-redux-middlewares runs its own internal withTracker and so the Meteor.userId() is reactive and I can just start the subscription on App.componentDidMount() and let it sit listening for changes on Meteor.userId(). There is no need for any of my custom withTracker + redux composing stuff! Closing now <3


Stricter type checking

Why was this change made?
a comment related to #107, and some slightly stricter sorbet type checking
How was this change tested?
unit tests
Which documentation and/or configurations were updated?
auto-gen sorbet configs

@jmartin-sul rebase please.

rebased and merged


Enable user to specify proxy settings

A customer needs to specify http_proxy and no_proxy settings for compute resources like Lambda functions, ECS containers, EC2 instances, etc. so that network traffic will be properly routed through the http proxy managed by their sponsor agency's cloud hosting division and they will be able to use LISA.

Hey @jebbens, thank you for bringing this to our attention! We will review this PFR for potential inclusion in our next release. I will keep you up to date here!

I recommend allowing a user to provide values for two variables, https_proxy and no_proxy. The first can be defined as: "Optional URL, including port, of an https proxy, e.g. Squid". The second can be defined as "An optional comma-separated list of endpoints, hostnames, or IP addresses whose traffic should not flow through the HTTP proxy."
If these values are present, they should be added as environment variables for any Lambda functions using the Python runtime and are attached to a VPC as HTTPS_PROXY and NO_PROXY. I do not know how/if other Lambda runtimes address this.
EC2 instances must also be configured to use these values, if present. Here is some example code to configure EKS nodes, but this will need to be modified per instructions on using an http proxy with ECS nodes.
TOKEN=$(curl -X PUT "http://<IP_ADDRESS>/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 3600")
REGION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://<IP_ADDRESS>/latest/meta-data/placement/region)
DOMAIN=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://<IP_ADDRESS>/latest/meta-data/services/domain)
PARTITION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://<IP_ADDRESS>/latest/meta-data/services/partition)
INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://<IP_ADDRESS>/latest/meta-data/instance-id)

PROXY_URL=${outbound_proxy_url}
if [[ -n "$PROXY_URL" ]]; then
  # https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-proxy.html
  export HTTPS_PROXY="$PROXY_URL" # enables calls to service APIs & IMDS

  EKS_IPv4_RANGE=$(aws eks describe-cluster --region $REGION --name ${cluster_name} --query 'cluster.kubernetesNetworkConfig.serviceIpv4Cidr')
  MAC=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://<IP_ADDRESS>/latest/meta-data/mac/)
  VPC_CIDR=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" "http://<IP_ADDRESS>/latest/meta-data/network/interfaces/macs/$MAC/vpc-ipv4-cidr-blocks" | xargs | tr ' ' ',')

  NO_PROXY_LIST=$EKS_IPv4_RANGE,$VPC_CIDR,localhost,<IP_ADDRESS>,<IP_ADDRESS>,.internal,.eks.$DOMAIN,${no_proxy_endpoints}
  NO_PROXY_LIST=$(sed "s/,$//" <<< "$NO_PROXY_LIST")

  # Set proxy for future processes
  cloud-init-per instance env_proxy_config cat <<EOF >> /etc/environment
http_proxy="$PROXY_URL"
https_proxy="$PROXY_URL"
no_proxy="$NO_PROXY_LIST"
HTTP_PROXY="$PROXY_URL"
HTTPS_PROXY="$PROXY_URL"
NO_PROXY="$NO_PROXY_LIST"
AWS_DEFAULT_REGION="$REGION"
EOF

  # Configure containerd for the proxy
  mkdir -p /etc/systemd/system/containerd.service.d
  cloud-init-per instance docker_proxy_config tee <<EOF /etc/systemd/system/containerd.service.d/http-proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # Configure the sandbox-image for the proxy
  mkdir -p /etc/systemd/system/sandbox-image.service.d
  cloud-init-per instance docker_proxy_config tee <<EOF /etc/systemd/system/sandbox-image.service.d/http-proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # Configure the kubelet for the proxy
  cloud-init-per instance kubelet_proxy_config tee <<EOF /etc/systemd/system/kubelet.service.d/proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-proxy-with-ssm-agent.html#ssm-agent-proxy-systemd
  mkdir /etc/systemd/system/amazon-ssm-agent.service.d
  cat <<EOF >> /etc/systemd/system/amazon-ssm-agent.service.d/override.conf
[Service]
EnvironmentFile=/etc/environment
EOF

  # Reload the daemon to reflect proxy configurations at launch of instance; restart ssm agent
  cloud-init-per instance reload_daemon systemctl daemon-reload
  cloud-init-per instance restart_ssm systemctl restart amazon-ssm-agent

fi