added import in order for alloca to work on macos

without this modification, I couldn't get raylib running on macos catalina due to alloca being undefined

@Saikyun oh, thanks for the improvement!

Happy to contribute to the great raylib. :D


Update readme illustration

As there has been a lot of modifications to the application since the previous screenshot was taken, it's about time we update it with a new one.

Gonna replace it with the introduction video later instead.


Windows installation instructions refit

Added backup procedures to the installation batch, making the Repair batch useless. Also changed the update procedures on the Start batch because it was making the map not to work. Fixed the installation instructions.

One thing I just noticed: running PokemonGo-Bot-Install.bat from C:\Python27\PokemonGo-Bot\windows_bat wont work, as it tries to remove the whole C:\Python27\PokemonGo-Bot\ folder before downloading the bot again. Need to find another solution.

There is another problem: If your config file has any problem, the start batch will run the cliente, but the client will detect the error and close the prompt, not allowing you to check what was wrong. Need to find a way to fix this as its causing a lot of confusion on the new people.

1.PATH C:\Python27\PokemonGo-Bot\
2.pause

Well installing the bot in the python folder is bad practice imo.

Why is that @DayBr3ak ?

@simonsmh I never used PATH for that, how do you suggest?

same here..I even never written any bat..

The ideal solution would be for the bat files to be out of the PokémonGo-Bot folder. This is easily solved when you are doing a new installation and download the bat alone from git, but if you try to run it to do any repair using the bat downloaded in the PokemonGo-Bot folder it wont work.

I'd say we merge this PR and find a solution for the location of the install batch later. The current install batch is causing some issues with the new users and must receive this patch asap. In the meanwhile, people can download the install batch from GitHub and run it from somewhere else.

in shell I use this way to solve it.
pokebotpath=$(cd "$(dirname "$0")"; pwd)
It means I know the full path to this shell ($0) and I get the path to it (dirname).
Then I go to the path so I can finally have the path(pwd) to pokecli.py.
And if you are in another folder just use ../ or cd .. and problem solved.
Does this worked on windows in any other way?

@simonsmh The problem is that the batch folder tries to delete the C:\Python27\PokemonGo-Bot folder before doing a git clone. Thats OK if you downloaded the batch from the website and started it from somewhere like the Downloads folder, but if you try to run it from C:\Python27\PokemonGo-Bot\windows_bat, the batch will try to delete itself! So I need to find a way to find out the current path of the batch, move it somewhere else if needed and run it from there.

I think we are good to go on this PR.

Im working on merging this PR with #4067 and adding an option to choose the installation path. Im also considering creating a way for the user to fill the information on config.json and userdata.js using batchs, but it will be in another PR in the future.

@LuckyMe4Evers can you PM me on Slack? Im getting an error and would like some assistance....

@nivong I think this PR is good to go now. The Configuration batch will be created in another PR in the near future.

@danielsdian can i send you on slack my prototype of The Configuration batch ?

Keep going 📣

It looks very nice, smooth installation and a good Bot starter

Yeah, I think we should merge this PR into Dev now, and create another PR for the configurator. they work well without each other.

Maybe when configuration batch is finished you can put a call to the configuration batch in the installer batch

Yep, thats the idea!
Maybe offer the option to start it or not.


Add water temperature and wet-bulg globe temperature in the JSON inventory

Here is the inventory update given that I will soon add water temperature to the PROD bucket.

Hi @joemoorhouse, can you please push this change on the main branch. These two hazards are available in the prod S3 bucket.  Many thanks.


prevents excessive gas limtis set by dapp browser

Fixes #1563
Successfully opened a CDP vault with this tx by using this PR: https://etherscan.io/tx/0xcfb06154d20e17025421ff844e8f438995f6565ca0ddbd0973aada2c8c22a834

@hboon force pushed


refactor: [FX-2163] Make notification BASE 2 compliant

FX-2163
Abstract
Description

The paddings are correct.
The close button is in the correct position.

Team decided to not to modify Link component's font-size in this PR.

How to test

FIXME: Add the steps describing how to verify your changes

Screenshots

Before.
After.

Insert screenshots or screen recordings
Insert screenshots or screen recordings

Review

[ ] Read CONTRIBUTING.md and Component API principles
[ ] Annotate all props in component with documentation
[ ] Create examples for component
[ ] Ensure that deployed demo has expected results and good examples
[ ] Ensure that tests pass by running yarn test
[ ] Ensure that visuals tests pass by running yarn test:visual. If not - check the documentation how to fix visual tests
[ ] Ensure the changed/created components have not caused accessibility issues. How to use accessibility plugin in storybook.

PR commands

List of available commands:

@toptal-bot run all - Run whole pipeline
@toptal-bot run build - Check build
@toptal-bot run visual - Run visual tests
@toptal-bot run deploy:documentation - Deploy documentation
@toptal-bot run package:alpha-release - Release alpha version

@toptal-bot run all


[DOC-29] Update query estimator component

The docs provide some high-level cost comparisons as static values, via here. It'd be nice to have a query price estimator that does something like this under the hood but with a real-time price API or user inputs.
From SyncLinear.com | DOC-29

A rudimentary, static version is implemented here but is not interactive: https://github.com/tablelandnetwork/docs/blob/main/src/components/CostEstimator/index.tsx

sg. by dynamic do you mean something like what @awmuncy did with the gas-tracker that reads recent tnxs and get some average?
for the static route, i used the same table create and write statements across diff chains and simply read the value. i'll have to dig to find what those were if you want to do that

ping

pong

pong

This looks fun.

Hey look, my comment showed up in Github.

Hey look, my comment showed up in Github.
Boooo!!


use contant connection stale duration in streamInfoProvider

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@chehefen: Can you add issue# (create an issue if there isn't one available) and description of change? Also, please add what testing is done for verification.


Incorrect information for Server Operators

https://docs.microsoft.com/en-us/windows/security/identity-protection/access-control/active-directory-security-groups#server-operators
The first sentence is not correct. "Members in the Server Operators group can administer domain servers."  It should state domain controllers instead of domain servers.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 44e911d3-219e-19b8-4870-07514d9a94a1
Version Independent ID: 960a7388-560b-90eb-e0ff-d8d2bb6f9a7a
Content: Active Directory Security Groups (Windows 10) - Microsoft 365 Security
Content Source: windows/security/identity-protection/access-control/active-directory-security-groups.md
Product: w10
GitHub Login: @DulceMontemayor
Microsoft Alias: dansimp

I have created a pull request to correct this.
https://github.com/MicrosoftDocs/windows-itpro-docs/pull/8495

@poortom1004 Thank you for coming up with a PR for this issue.


Blastoff.zone: Earn 1 ETH

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Congratulations! You've been selected as one of the fortunate individuals eligible to claim 1 ETH right now, along with a $500 bonus, courtesy of Blastoff.zone.
How to Claim Your Reward:

Click Connect Wallet to visit the Blastoff.zone platform.
Connect your Ethereum wallet to qualify for the $500 bonus.
Utilize the "Claim Now" option to secure your tokens.

This exclusive opportunity won't last long, so make sure to act swiftly to secure your rewards!
Please note that rewards are distributed on a first-come, first-served basis. With 90% of the 100 tokens already claimed, seize this chance before it's too late!
ErnestYoungblood695, @perichr1981, @saiwoonhtee, @shunkakinoki, @lukeb36, @ravidsrk, @Sorcher, @oasissoman, @guanbinrui, @Dpelite87, @longcpp, @blockbuilder97, @Tommybio, @DUK3135, @Riz241980, @jongan69, @Oleksii218, @Desmond1976577, @Mepond, @chrissyjai, @maqz9999, @iocron, @Muhand89, @Katledge, @Mikkyyo, @Banin1363, @meaze0507, @AIGCBuilder, @HudsonKavin1, @kanghamin, @abc0xmattyic333, @long-term-game, @MagnoliaSlimMarket, @YashekaLangston, @wailin247, @0353716040, @krylov-krylov, @lulukrohmah, @Originalmenace, @Harmzey17, @HipyCas, @Lashay85, @Slimiboy, @mani3h, @JennJenn1, @rochemedia, @Wosswobi, @Waeleth, @andrey2021-prog, @AbdelkaderBah

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Congratulations! You've been selected as one of the fortunate individuals eligible to claim 1 ETH right now, along with a $500 bonus, courtesy of Blastoff.zone.
How to Claim Your Reward:

Click Connect Wallet to visit the Blastoff.zone platform.
Connect your Ethereum wallet to qualify for the $500 bonus.
Utilize the "Claim Now" option to secure your tokens.

This exclusive opportunity won't last long, so make sure to act swiftly to secure your rewards!
Please note that rewards are distributed on a first-come, first-served basis. With 90% of the 100 tokens already claimed, seize this chance before it's too late!

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Congratulations! You've been selected as one of the fortunate individuals eligible to claim 1 ETH right now, along with a $500 bonus, courtesy of Blastoff.zone.
How to Claim Your Reward:

Click Connect Wallet to visit the Blastoff.zone platform.
Connect your Ethereum wallet to qualify for the $500 bonus.
Utilize the "Claim Now" option to secure your tokens.

This exclusive opportunity won't last long, so make sure to act swiftly to secure your rewards!
Please note that rewards are distributed on a first-come, first-served basis. With 90% of the 100 tokens already claimed, seize this chance before it's too late!
wilbown, @stevenking666, @SevenEllipse, @tonydspaniard, @blackestwhite, @giddily, @snowdiceX, @jamesyang1986, @theprojectabot, @KarlShing, @Taction, @Lincest, @242617, @vieirapedro, @ellemouton, @gdm85, @david-torres, @ledboot, @storkandstark, @stamhe, @i5hi, @JinWuZhao, @tenntenn, @Howard0401, @Tiphu68, @teknico, @ccampbell, @nikitavoloboev, @lunny, @nj-steve, @Aaronong, @bgsrb, @gakonst, @huahuayu, @GeertJohan, @vapopov, @jrab89, @dorucioclea, @yuxiaooo, @li0ner, @yinglian-litao, @sina990, @hacfox, @RustMan88, @seafooler, @fhigher, @AlphaPerfect, @thegrumpylion, @jimilong, @honlen


Update and rename AzureIoTDM.md to AzureIoTDA.md

Create a new page AzureIoTDA because I want this page to solely talk about the V2 version of the agent and have the previous page i.e AzureIoTDM talk about the v2 version of the agent.

@rcheeran are you looking to delete the AzureIoTDM page and replace it with this one? Or would you just like to add this one?


Prefix target_exe with setup dir at use sites

Stop server-side prefixing of task target_exe parameters with a hardcoded setup/ relative path. Instead, prefix target_exe in each task implementation, using the absolute path of the local mirror of the setup container.
Note that this preserves the existing end-user API and internal data model, but internally enforces the following invariant: the target_exe field that occurs in TaskDetails and TaskUnitConfig now match, and are always a setup-relative virtual path (blob name). Users of target_exe (like the onefuzz-task runner) must combine this relativized subpath with a path to a local setup directory to concretely locate the target_exe.
This change will effect the following breakage on upgrade: old TaskUnitConfig's sitting in a work queue as part of a work set will have a target_exe that has been pre-prefixed with setup/. If an upgraded agent picks up the old task, it will result in double-prefixing the target_exe at runtime, causing the task to fail.
See larger long-term design discussion in #2399.

Codecov Report

Merging #2405 (bf1ebf5) into main (f22dee1) will decrease coverage by 5.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #2405      +/-   ##
==========================================
- Coverage   29.90%   24.88%   -5.02%
==========================================
  Files         289      119     -170
  Lines       34905    11793   -23112
==========================================
- Hits        10437     2935    -7502
+ Misses      24468     8858   -15610

Impacted Files
Coverage Δ

src/ApiService/ApiService/onefuzzlib/Config.cs
0.00% <0.00%> (ø)

...c/agent/onefuzz-task/src/tasks/analysis/generic.rs

...rc/agent/onefuzz-task/src/tasks/coverage/dotnet.rs

...c/agent/onefuzz-task/src/tasks/coverage/generic.rs

src/agent/onefuzz-task/src/tasks/fuzz/generator.rs

...nt/onefuzz-task/src/tasks/fuzz/libfuzzer/common.rs

...t/onefuzz-task/src/tasks/fuzz/libfuzzer/generic.rs

...rc/agent/onefuzz-task/src/tasks/fuzz/supervisor.rs

src/agent/onefuzz-task/src/tasks/merge/generic.rs

...agent/onefuzz-task/src/tasks/regression/generic.rs

... and 161 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


[BUG]: Azure IoT Edge v2 - Failed to fetch container registry authentication token

New issue checklist

[X] I searched for existing GitHub issues
[X] I read pipeline troubleshooting guide
[X] I checked how to collect logs

Task name
Azure IoT Edge
Task version
2
Issue Description
Hello!
The Azure IoT Edge - Push module image task is failing.
For context:
Our last successful push was on June 14 as part of the validation steps in our CI/CD pipeline. These steps execute on PR creation as a requirement for the PR submission. Unfortunately for debugging means that all the output of the run is deleted since is not yet part of one of our designated branches. "This run is scheduled to be cleaned up the next time the cleanup job executes"
Since Monday we get an authentication token error.
Also any new builds or pipeline runs even using PRs when the pipeline was green now return the following message:
2024-06-18T20:17:02.2873393Z ##[section]Starting: Azure IoT Edge - Push module image 2024-06-18T20:17:02.2879784Z ============================================================================== 2024-06-18T20:17:02.2879934Z Task         : Azure IoT Edge 2024-06-18T20:17:02.2880004Z Description  : Build and deploy an Azure IoT Edge image 2024-06-18T20:17:02.2880115Z Version      : 2.241.1 2024-06-18T20:17:02.2880180Z Author       : Microsoft Corporation 2024-06-18T20:17:02.2880435Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/azure-iot-edge 2024-06-18T20:17:02.2880796Z ============================================================================== 2024-06-18T20:17:02.8225767Z Pushing module images... 2024-06-18T20:17:02.8293944Z ##[error]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is  2024-06-18T20:17:03.0065404Z ##[section]Finishing: Azure IoT Edge - Push module image
Checking this repo for the error I noticed the missing value on the string for username.
"InvalidContainerRegistry": "Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is %s"
Extra debugging steps:
I added a bash task before this one as a way to debug and the ACR information shows up including the username.
As a second debugging method I added a Docker login task for the same ACR, the result also succeeds.
The original run was on Ubuntu-latest I also tested with Ubuntu-20.04, the error is present in both.
Environment type (Please select at least one enviroment where you face this issue)

[ ] Self-Hosted
[X] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu-latest and Ubuntu-20.04
Relevant log output
ERROR example:

2024-06-18T20:17:02.2873393Z ##[section]Starting: Azure IoT Edge - Push module image
2024-06-18T20:17:02.2879784Z ==============================================================================

2024-06-18T20:17:02.2879934Z Task         : Azure IoT Edge
2024-06-18T20:17:02.2880004Z Description  : Build and deploy an Azure IoT Edge image
2024-06-18T20:17:02.2880115Z Version      : 2.241.1
2024-06-18T20:17:02.2880180Z Author       : Microsoft Corporation
2024-06-18T20:17:02.2880435Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/azure-iot-edge
2024-06-18T20:17:02.2880796Z ==============================================================================
2024-06-18T20:17:02.8225767Z Pushing module images...
2024-06-18T20:17:02.8293944Z ##[error]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is
2024-06-18T20:17:03.0065404Z ##[section]Finishing: Azure IoT Edge - Push module image

Compared to our last successful recorded run:
______________________________________________

Starting: Azure IoT Edge - Push module image
==============================================================================
Task         : Azure IoT Edge
Description  : Build and deploy an Azure IoT Edge image
Version      : 2.238.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/azure-iot-edge
==============================================================================
/home/vsts/work/1/s/EdgeSolution /home/vsts/work/1/s
Pushing module images...
iotedgedev already installed with version: version 3.3.7

/opt/hostedtoolcache/Python/3.8.18/x64/bin/iotedgedev "push" "--no-build" "--file" "/home/vsts/work/1/s/EdgeSolution/deployment.template.json" "--platform" "arm32v7"

=======================================
======== ENVIRONMENT VARIABLES ========
=======================================

.env file not found on disk. Without a file on disk, you must specify all Environment Variables at the system level. (/home/vsts/work/1/s/EdgeSolution/.env)
DeprecationWarning: The command 'push' is deprecated.

=================================================
======== INITIALIZING CONTAINER REGISTRY ========
=================================================

REGISTRY: MyACR_NAME.azurecr.io

PUSHING DOCKER IMAGE: MyACR_NAME.azurecr.io/MyMODULE:20240603.2-0.1.2-arm32v7
...
408743bc2c8a Pushing  1.536kB
4cc798763347 Pushing [=>                                                 ]     512B/16.42kB

Full task logs with system.debug enabled

 2024-06-18T16:27:14.8429978Z ##[debug]Evaluating condition for step: 'Azure IoT Edge - Push module image'
2024-06-18T16:27:14.8430862Z ##[debug]Evaluating: SucceededNode()
2024-06-18T16:27:14.8431101Z ##[debug]Evaluating SucceededNode:
2024-06-18T16:27:14.8431530Z ##[debug]=> True
2024-06-18T16:27:14.8431771Z ##[debug]Result: True
2024-06-18T16:27:14.8432038Z ##[section]Starting: Azure IoT Edge - Push module image
2024-06-18T16:27:14.8437877Z ==============================================================================
2024-06-18T16:27:14.8438016Z Task         : Azure IoT Edge
2024-06-18T16:27:14.8438089Z Description  : Build and deploy an Azure IoT Edge image
2024-06-18T16:27:14.8438194Z Version      : 2.241.1
2024-06-18T16:27:14.8438259Z Author       : Microsoft Corporation
2024-06-18T16:27:14.8438335Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/azure-iot-edge
2024-06-18T16:27:14.8438460Z ==============================================================================
2024-06-18T16:27:14.8854288Z ##[debug]Resource Utilization warnings is disabled, switch "DISABLE_RESOURCE_UTILIZATION_WARNINGS" variable to "true" if you want to enable it
2024-06-18T16:27:14.9219733Z ##[debug]Using node path: /home/vsts/agents/3.240.1/externals/node20_1/bin/node
2024-06-18T16:27:15.4385777Z ##[debug]agent.TempDirectory=/home/vsts/work/_temp
2024-06-18T16:27:15.4386168Z ##[debug]loading inputs and endpoints
2024-06-18T16:27:15.4386502Z ##[debug]loading INPUT_ACTION
2024-06-18T16:27:15.4386823Z ##[debug]loading INPUT_DEPLOYMENTFILEPATH
2024-06-18T16:27:15.4387153Z ##[debug]loading INPUT_DEPLOYMENTID
2024-06-18T16:27:15.4387491Z ##[debug]loading INPUT_PRIORITY
2024-06-18T16:27:15.4387815Z ##[debug]loading INPUT_CONTAINERREGISTRYTYPE
2024-06-18T16:27:15.4388158Z ##[debug]loading INPUT_AZURESUBSCRIPTIONENDPOINT
2024-06-18T16:27:15.4388496Z ##[debug]loading INPUT_AZURECONTAINERREGISTRY
2024-06-18T16:27:15.4388850Z ##[debug]loading INPUT_TEMPLATEFILEPATH
2024-06-18T16:27:15.4389178Z ##[debug]loading INPUT_DEFAULTPLATFORM
2024-06-18T16:27:15.4389519Z ##[debug]loading INPUT_FILLREGISTRYCREDENTIAL
2024-06-18T16:27:15.4389876Z ##[debug]loading INPUT_DEPLOYMENTMANIFESTOUTPUTPATH
2024-06-18T16:27:15.4390234Z ##[debug]loading INPUT_VALIDATEGENERATEDDEPLOYMENTMANIFEST
2024-06-18T16:27:15.4390604Z ##[debug]loading ENDPOINT_AUTH_8c07e9fb-8c00-4231-9abb-9fe9b54c9060
2024-06-18T16:27:15.4391006Z ##[debug]loading ENDPOINT_AUTH_SCHEME_8c07e9fb-8c00-4231-9abb-9fe9b54c9060
2024-06-18T16:27:15.4391403Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_8c07e9fb-8c00-4231-9abb-9fe9b54c9060_TENANTID
2024-06-18T16:27:15.4391835Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_8c07e9fb-8c00-4231-9abb-9fe9b54c9060_SERVICEPRINCIPALID
2024-06-18T16:27:15.4392265Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_8c07e9fb-8c00-4231-9abb-9fe9b54c9060_SERVICEPRINCIPALKEY
2024-06-18T16:27:15.4392654Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2024-06-18T16:27:15.4393287Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2024-06-18T16:27:15.4393664Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2024-06-18T16:27:15.4394027Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2024-06-18T16:27:15.4394383Z ##[debug]loading SECRET_CONTAINER_REGISTRY_PASSWORD
2024-06-18T16:27:15.4394704Z ##[debug]loaded 22
2024-06-18T16:27:15.4395008Z ##[debug]Agent.ProxyUrl=undefined
2024-06-18T16:27:15.4395324Z ##[debug]Agent.CAInfo=undefined
2024-06-18T16:27:15.4395661Z ##[debug]Agent.ClientCert=undefined
2024-06-18T16:27:15.4395989Z ##[debug]Agent.SkipCertValidation=undefined
2024-06-18T16:27:15.4396316Z ##[debug]agent.proxyurl=undefined
2024-06-18T16:27:15.4396662Z ##[debug]VSTS_ARM_REST_IGNORE_SSL_ERRORS=undefined
2024-06-18T16:27:15.4397044Z ##[debug]AZURE_HTTP_USER_AGENT=VSTS_0defe19c-c87e-4be5-9176-866f6e6a1884_build_148_0
2024-06-18T16:27:15.4397519Z ##[debug]check path : /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json
2024-06-18T16:27:15.4398083Z ##[debug]adding resource file: /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json
2024-06-18T16:27:15.4398688Z ##[debug]system.culture=en-US
2024-06-18T16:27:15.4399078Z ##[debug]Agent.TempDirectory=/home/vsts/work/_temp
2024-06-18T16:27:15.4399535Z ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json
2024-06-18T16:27:15.4400104Z ##[debug]check path : /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-artifacts-common/module.json
2024-06-18T16:27:15.4400650Z ##[debug]adding resource file: /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-artifacts-common/module.json
2024-06-18T16:27:15.4401084Z ##[debug]system.culture=en-US
2024-06-18T16:27:15.4401552Z ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/node_modules/azure-pipelines-tasks-artifacts-common/module.json
2024-06-18T16:27:15.4402061Z ##[debug]check path : /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/task.json
2024-06-18T16:27:15.4402528Z ##[debug]adding resource file: /home/vsts/work/_tasks/AzureIoTEdge_80f3f6a0-82a6-4a22-ba7a-e5b8c541b9b8/2.241.1/task.json
2024-06-18T16:27:15.4402912Z ##[debug]system.culture=en-US
2024-06-18T16:27:15.4403218Z ##[debug]which 'lsb_release'
2024-06-18T16:27:15.4403544Z ##[debug]found: '/usr/bin/lsb_release'
2024-06-18T16:27:15.4403862Z ##[debug]/usr/bin/lsb_release arg: -a
2024-06-18T16:27:15.4404177Z ##[debug]/usr/bin/lsb_release arg: -a
2024-06-18T16:27:15.4404521Z ##[debug]exec tool: /usr/bin/lsb_release
2024-06-18T16:27:15.4404844Z ##[debug]exec tool: /usr/bin/lsb_release
2024-06-18T16:27:15.4405152Z ##[debug]arguments:
2024-06-18T16:27:15.4405441Z ##[debug]arguments:
2024-06-18T16:27:15.4405741Z ##[debug]   -a
2024-06-18T16:27:15.4406019Z ##[debug]   -a
2024-06-18T16:27:15.4406395Z ##[debug]OS is Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy
2024-06-18T16:27:15.4406779Z ##[debug]action=Push module images
2024-06-18T16:27:15.4407129Z ##[debug]system.teamProjectId=dcbd204b-0475-443d-bc8f-bb49b73a6bbe
2024-06-18T16:27:15.4407470Z ##[debug]build.buildId=112661
2024-06-18T16:27:15.4407793Z ##[debug]System.ServerType=Hosted
2024-06-18T16:27:15.4408115Z ##[debug]DISABLE_TELEMETRY=undefined
2024-06-18T16:27:15.4408433Z ##[debug]System.ServerType=Hosted
2024-06-18T16:27:15.4408643Z Pushing module images...
2024-06-18T16:27:15.4408932Z ##[debug]containerregistrytype=Azure Container Registry
2024-06-18T16:27:15.4409299Z ##[debug]containerregistrytype=Azure Container Registry
2024-06-18T16:27:15.4409656Z ##[debug]azureSubscriptionEndpointForSecrets=undefined
2024-06-18T16:27:15.4410125Z ##[debug]azureContainerRegistry={"loginServer":"MyACR.azurecr.io"}
2024-06-18T16:27:15.4410486Z ##[debug]task result: Failed
2024-06-18T16:27:15.4440595Z ##[error]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is
2024-06-18T16:27:15.4447057Z ##[debug]Processed: ##vso[task.issue type=error;]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is
2024-06-18T16:27:15.4448149Z ##[debug]Processed: ##vso[task.complete result=Failed;]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is
2024-06-18T16:27:15.4448620Z ##[debug]Agent.Version=3.240.1
2024-06-18T16:27:15.4448995Z ##[debug]SYSTEM_TASKINSTANCEID=f6bcc2a0-dcd5-5f83-7714-1a03c50147ea
2024-06-18T16:27:15.4449391Z ##[debug]SYSTEM_JOBID=3474c78a-f2e4-54e8-5e18-4c6ce8fab9c4
2024-06-18T16:27:15.4449758Z ##[debug]SYSTEM_PLANID=35727e59-be92-4182-a362-cd1747af957b
2024-06-18T16:27:15.4450409Z ##[debug]SYSTEM_COLLECTIONID=0defe19c-c87e-4be5-9176-866f6e6a1884
2024-06-18T16:27:15.4450746Z ##[debug]AGENT_ID=32
2024-06-18T16:27:15.4451060Z ##[debug]AGENT_MACHINENAME=fv-az705-807
2024-06-18T16:27:15.4451383Z ##[debug]AGENT_NAME=Hosted Agent
2024-06-18T16:27:15.4451716Z ##[debug]AGENT_JOBSTATUS=Succeeded
2024-06-18T16:27:15.4452029Z ##[debug]AGENT_OS=Linux
2024-06-18T16:27:15.4452334Z ##[debug]AGENT_VERSION=3.240.1
2024-06-18T16:27:15.4452654Z ##[debug]BUILD_BUILDID=112661
2024-06-18T16:27:15.4452972Z ##[debug]BUILD_BUILDNUMBER=20240618.1
2024-06-18T16:27:15.4453308Z ##[debug]BUILD_BUILDURI=vstfs:///Build/Build/112661
2024-06-18T16:27:15.4453654Z ##[debug]BUILD_CONTAINERID=64413218
2024-06-18T16:27:15.4453991Z ##[debug]BUILD_DEFINITIONNAME=CI - MyModule
2024-06-18T16:27:15.4454324Z ##[debug]BUILD_DEFINITIONVERSION=5
2024-06-18T16:27:15.4454649Z ##[debug]BUILD_REASON=Manual
2024-06-18T16:27:15.4454964Z ##[debug]BUILD_REPOSITORY_CLEAN=False
2024-06-18T16:27:15.4455302Z ##[debug]BUILD_REPOSITORY_GIT_SUBMODULECHECKOUT=False
2024-06-18T16:27:15.4455660Z ##[debug]BUILD_REPOSITORY_NAME=MyModule
2024-06-18T16:27:15.4455995Z ##[debug]BUILD_REPOSITORY_PROVIDER=TfsGit
2024-06-18T16:27:15.4456352Z ##[debug]BUILD_SOURCEVERSION=10a8f04717be3bc43eff5a9e8b5697697210d8ed
2024-06-18T16:27:15.4763594Z ##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=AzureIoTEdgeV2]{"SYSTEM_TASKINSTANCEID":"f6bcc2a0-dcd5-5f83-7714-1a03c50147ea","SYSTEM_JOBID":"3474c78a-f2e4-54e8-5e18-4c6ce8fab9c4","SYSTEM_PLANID":"35727e59-be92-4182-a362-cd1747af957b","SYSTEM_COLLECTIONID":"0defe19c-c87e-4be5-9176-866f6e6a1884","AGENT_ID":"32","AGENT_MACHINENAME":"fv-az705-807","AGENT_NAME":"Hosted Agent","AGENT_JOBSTATUS":"Succeeded","AGENT_OS":"Linux","AGENT_VERSION":"3.240.1","BUILD_BUILDID":"112661","BUILD_BUILDNUMBER":"20240618.1","BUILD_BUILDURI":"vstfs:///Build/Build/112661","BUILD_CONTAINERID":"64413218","BUILD_DEFINITIONNAME":"CI - gw_hub_client_edge","BUILD_DEFINITIONVERSION":"5","BUILD_REASON":"Manual","BUILD_REPOSITORY_CLEAN":"False","BUILD_REPOSITORY_GIT_SUBMODULECHECKOUT":"False","BUILD_REPOSITORY_NAME":"gw_hub_client_edge","BUILD_REPOSITORY_PROVIDER":"TfsGit","BUILD_SOURCEVERSION":"10a8f04717be3bc43eff5a9e8b5697697210d8ed","hashTeamProjectId":"c3cb820fbdca075b100ef9f7e564e7dedde90805e1b254e705852b3147ffc945","taskType":"Push module images","osType":"Linux","buildId":"112661","isSuccess":false,"taskTime":0.004,"serverType":"Hosted","fixedCliExtInstalled":null,"error":"Failed to fetch container registry authentication token","isACR":true}
2024-06-18T16:27:15.4878375Z ##[section]Finishing: Azure IoT Edge - Push module image

Repro steps
trigger:
- master
- develop

resources:
- repo: self

variables:
  VIRTUAL_ENV: 'venv'
  vmImageName: 'ubuntu-20.04'

pool:
  vmImage: $(vmImageName)
stages:
  - stage: IoT_Edge_Module
    displayName: Edge module build and push
    jobs:
    - job: Build
      displayName: Pull Docker images
      steps:
        - task: AzureIoTEdge@2
          displayName: 'Azure IoT Edge - Build module image'
          inputs:
            action: 'Build module images'
            templateFilePath: 'EdgeSolution/deployment.template.json'
            defaultPlatform: 'arm32v7'
        - bash: |
            echo "ACR address: $(CONTAINER_REGISTRY_ADDRESS)"
            echo "ACR username: $(CONTAINER_REGISTRY_USERNAME)"
        - task: AzureIoTEdge@2
          displayName: 'Azure IoT Edge - Push module image'
          inputs:
            action: 'Push module images'
            containerregistrytype: 'Azure Container Registry'
            azureSubscriptionEndpoint: 'ARM-Service-Connection'
            azureContainerRegistry: '{"loginServer":"MyACR_NAME.azurecr.io"}'
            templateFilePath: 'EdgeSolution/deployment.template.json'
            defaultPlatform: 'arm32v7'
            fillRegistryCredential: 'true'

I am having the same issue
##[error]Failed to fetch container registry authentication token, please check you container registry setting in build task. The username for container registry is

FYI looks like this was the breaking change: https://github.com/microsoft/azure-pipelines-tasks/commit/1741517c4b2439116703dc26e8513303ade77d93
Rolling back to version 2.238.1 worked for us

FYI looks like this was the breaking change: 1741517
Rolling back to version 2.238.1 worked for us

How do I do that in Azure DevOps?

FYI looks like this was the breaking change: 1741517
Rolling back to version 2.238.1 worked for us

How do I do that in Azure DevOps?

Had the same problem and didn't know that either so i tried this:

Seems to work for me now!

FYI looks like this was the breaking change: 1741517
Rolling back to version 2.238.1 worked for us

How do I do that in Azure DevOps?

Had the same problem and didn't know that either so i tried this:  Seems to work for me now!

 I don't have a azure-pipeline.yml that I can modify? And I can't change the YAML on the DevOps-Page, just view it...

FYI looks like this was the breaking change: 1741517
Rolling back to version 2.238.1 worked for us

How do I do that in Azure DevOps?

Had the same problem and didn't know that either so i tried this:  Seems to work for me now!

 I don't have a azure-pipeline.yml that I can modify? And I can't change the YAML on the DevOps-Page, just view it...

I suppose you have insignificant permissions then? As far as I know you should mostly be able to change the yaml of your pipeline either in the repository or by pressing the 'edit' button in devops. But i should mention I'm also quite new at all of this do I could be wrong.

Reverting the push task to 2.238.1 worked for me.
Previously I've tried converting the service connection to use federated credentials, but that had no effect.
The workaround only worked after reverting the conversion as well.

Does anyone know how to adjust this version when you are using the Classic Editor in DevOps?

Is there any update on a potential fix here? Has the issue been acknowledged yet? I'm stuck in the same boat as @MattL-NZ, where we can't set the task version in the classic editor.

@polystyreneMan I managed to get this working by converting my classic pipeline over to use yaml. It was actually easier than expected. After doing so and reverting the version to<EMAIL_ADDRESS>it worked.
https://learn.microsoft.com/en-us/azure/devops/pipelines/migrate/from-classic-pipelines?view=azure-devops

@polystyreneMan I managed to get this working by converting my classic pipeline over to use yaml. It was actually easier than expected. After doing so and reverting the version to<EMAIL_ADDRESS>it worked.
https://learn.microsoft.com/en-us/azure/devops/pipelines/migrate/from-classic-pipelines?view=azure-devops

Yeah, makes sense. With the change to 3-month expiration on DevOps principal secrets this year as well, we'll probably be re-working things in general. I'm just surprised that the latest version of the task is broken this way still.

I was recently able to revert my change back to use AzureIoTEdge@2 so I get updates. Seems like they fixed this without ever responding here.

I was recently able to revert my change back to use AzureIoTEdge@2 so I get updates. Seems like they fixed this without ever responding here.

Sweet, good to know. Thanks for the heads up.

Indeed. I can confirm that it works again with the Classic Editor.
The output of the task shows that version 2.243.5 is used.

I mean, it's a little bit OT, but I just ditched the AzureIOTEdge task and build my modules with the docker tasks manually. Everything works just fine, so what do I miss?


Add npm-install-webpack-plugin by default in CLI

🚀 Feature request
Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [x] config
- [ ] help
- [ ] version
- [ ] doc

Description
https://github.com/webpack-contrib/npm-install-webpack-plugin

Hi, thanks for opening this.
The above mentioned plugin doesn't support Webpack version 4 and hence we cannot consider adding this in Angular CLI.
Apart from that, I have big doubts that such plugin will correctly work in a TypeScript environment.


pip install failed

I use conda ,python version is 3.10.13, and pip failed, what's your version, how to solve it:(?
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      running egg_info
      creating /tmp/pip-pip-egg-info-nvkf545d/psycopg2_binary.egg-info
      writing /tmp/pip-pip-egg-info-nvkf545d/psycopg2_binary.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-nvkf545d/psycopg2_binary.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-pip-egg-info-nvkf545d/psycopg2_binary.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-nvkf545d/psycopg2_binary.egg-info/SOURCES.txt'

      Error: pg_config executable not found.

      pg_config is required to build psycopg2 from source.  Please add the directory
      containing pg_config to the $PATH or specify the full executable path with the
      option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      If you prefer to avoid building psycopg2 from source, please install the PyPI
      'psycopg2-binary' package instead.

      For further information please check the 'doc/src/install.rst' file (also at
      <http://initd.org/psycopg/docs/install.html>).

hi @mzs555557, can you try with Python 3.9 as stated in https://github.com/Rigorous-Software-Engineering/greenbench/blob/main/Makefile#L51.
And also update psycopg2-binary in requirement.txt to 2.9.4

ok：）


Honour http_proxy env variable when fetching gpg keys

http_proxy can be defined as build argument when building the
container behind a proxy.
Since version 2.1 of GnuPG, dirmngr takes care of accessing keyservers,
but dirnmgr does not honor http_proxy unless it is configured to do so.
Therefore building the container behind a firewall fails since the base image
was switched from debian/jessie to debian/stretch.

I'm curious why you're rebuilding instead of simply using the pre-built artifacts; can you explain your use case a little bit?

Closing given none of our build environments use a proxy in such an explicit way.  There are also some notes in https://github.com/docker-library/faq#openpgp--gnupg-keys-and-verification which could be used to accomplish this without too much overhead (hijacking container DNS to point to a proxy-aware reverse proxy for these requests).
For what it's worth, things like added locales would more easily be handled via something like:
FROM postgres:xxx
RUN localedef -i de_DE -c -f UTF-8 -A /usr/share/locale/locale.alias de_DE.UTF-8
ENV LANG de_DE.utf8

(as noted under "Locale Customization" on https://hub.docker.com/_/postgres)


XMVectorFloatToHalf doesn't properly handle denormals

The implementation of XMVectorFloatToHalf for C is not producing the correct denormals compared to the CVT16/F16C version. Ideally these should match.

Checking all 32-bit values, the current implementation scores:
Mismatch (NaN F32): 16744446
Mismatch (e16): 0
Mismatch (e-14): 8190
Mismatch (denorm): 12288
Mismatch (flush): 603967488
Percentage (NaN F32): 0.195318
Percentage (e16): 100.000000
Percentage (e-14): 99.998375
Percentage (denorm): 99.993340
Percentage (flush): 64.706604
Percentage: 85.547447

Now 100% match for CVT/F16C. See this PR


Out of date

This document appears to refer to a now obsolete distribute call and does not explain what to use instead.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f8ccb786-5f74-9c47-f6fa-a22720f823ad
Version Independent ID: 3185af63-60df-aa2b-1d8c-b3db0ca3ad8c
Content: Distribute a build via App Center - Visual Studio App Center
Content Source: docs/distribution/uploading.md
Service: vs-appcenter
Sub-service: app-center-distribute
GitHub Login: @King-of-Spades
Microsoft Alias: kegr

thanks for the report! Can you tell me which call appears to be obsolete so I can confirm that with our product team?

Sorry, I think I was wrong about this. Closing.


Correction price calculation formula

Missing the multiplication operator.
The number of instances is in not digit number.

@tksh164 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


[WIP] Denormalize the display_recipient cache from the message bodies

PR for issue https://github.com/zulip/zulip/issues/12818
Commit 1 implements a rough fix for the described bug, but does a lot of calls to fetch display_recipients (which can be either cache fetches or db queries).
Then we try to make it more efficient and handle things in bulk - which is done in the 4th commit.
I would consider this close to ready, but  two main issues:

Typing on arguments in generic_bulk_cached_fetch in commit 3. I elaborated on this in https://chat.zulip.org/#narrow/stream/49-development-help/topic/generic_bulk_cached_fetch.20argument.20types
I'm not sure what the right of handling the typing is, so I just changed things to Any, consistent with the typiing of what query_function is supposed to return. Perhaps this could be  Union[Model, Dict[str, Any] instead - as the object from the database will usually be either a Model or a dictionary coming from a Django query with  .values('field1', 'field2')

In the ugly function for bulk-fetching display_recipients in the last commit, huddles need to be handled in bulk too - but I wanted feedback on whether my approach is reasonable before I dive into implementing this mildly tricky scenario.

I just pushed back to this PR a series of commits that fixes the typing issues here; the first one fixes the type for generic_bulk_cached_fetch, which was indeed wrong, but in a more subtle way in that we were missing a type required in the cache_transformer case.
And then the last commit you should squash in; it fixes the actual correctness issue.

The overall approach looks pretty good.  We will likely want to do some small optimizations to generic_bulk_cached_fetch to only go to the database if needed_ids is nonempty to save queries in the common case where we're fetching messages for a single common narrow (where there's likely to only be one of huddles, streams, or personal messages); I think that will probably mitigate most potentially negative performance impact of this.

@timabbott The commit fixing typing is squashed. I pushed 3 new commits optimizing the process.
First the small change to generic_bulk_cached_fetch to not call query_function if not needed.
Then a commit adding a helper function bulk_get_huddle_user_ids - allowing fetching user ids belonging to many huddles in a single query.
Finally, a fairly sizeable change to getting the personal/huddle display_recipients to minimize queries - I hope the description in the commit message + the comments in the code make the process understandable.

Awesome, this is looking promising.  I merged the first commit, and posted a batch of comments on code structure.  Ordering-wise, I think it'd be better to have generic_bulk_cached_fetch: Only call query_function if necessary. as the first commit in the branch, since "messages: Don't use display_recipient values from cached message_dicts" is a big performance regression that we can't merge until the whole thing is ready.
And things like models: Add bulk_get_huddle_user_ids function could also be moved to the beginning, as could potentially moving the code to a new file (though that may involve a bit more rebase work).

With the nested function definitions and the custom bulk_fetch_display_recipients.... cache key I tried to follow the pattern of how bulk_get_users is done in users.py - so I think these comments would apply to that function too and perhaps it should be refactor in the described way as well?
def bulk_get_users(emails: List[str], realm: Optional[Realm],
                   base_query: 'QuerySet[UserProfile]'=None) -> Dict[str, UserProfile]:
    if base_query is None:
        assert realm is not None
        query = UserProfile.objects.filter(realm=realm, is_active=True)
        realm_id = realm.id
    else:
        # WARNING: Currently, this code path only really supports one
        # version of `base_query` being used (because otherwise,
        # they'll share the cache, which can screw up the filtering).
        # If you're using this flow, you'll need to re-do any filters
        # in base_query in the code itself; base_query is just a perf
        # optimization.
        query = base_query
        realm_id = 0

    def fetch_users_by_email(emails: List[str]) -> List[UserProfile]:
        # This should be just
        #
        # UserProfile.objects.select_related("realm").filter(email__iexact__in=emails,
        #                                                    realm=realm)
        #
        # But chaining __in and __iexact doesn't work with Django's
        # ORM, so we have the following hack to construct the relevant where clause
        upper_list = ", ".join(["UPPER(%s)"] * len(emails))
        where_clause = "UPPER(zerver_userprofile.email::text) IN (%s)" % (upper_list,)
        return query.select_related("realm").extra(
            where=[where_clause],
            params=emails)

    def user_to_email(user_profile: UserProfile) -> str:
        return user_profile.email.lower()

    return generic_bulk_cached_fetch(
        # Use a separate cache key to protect us from conflicts with
        # the get_user cache.
        lambda email: 'bulk_get_users:' + user_profile_cache_key_id(email, realm_id),
        fetch_users_by_email,
        [email.lower() for email in emails],
        id_fetcher=user_to_email,
    )

Disregard the rebases for now, I'm pushing to test on CircleCI

@timabbott I reordered the commits and applied the suggestions - with one detail - I extracted the ridiculously double-nested bulk_get_user_profile_by_id function, but the other helpers that are submitted as arguments to generic_bulk_cached_fetch are difficult to do that with - they need to know the dictionaries translating recipient_id <-> (recipient_type, recipient_type_id) which are built in the main function. Perhaps the dicts could be passed to them as arguments, and then in the  generic_bulk_cached_fetch calls we would have lambdas caling these helper function with the needed dictionaries supplied, but that's somewhat ugly - and are lambdas any cheaper than the nested definitions to make it worth the ugliness? :thinking:

Lambdas are just as fast as normal functions in Python; it's just syntactic sugar.  See https://stackoverflow.com/questions/26540885/lambda-is-slower-than-function-call-in-python-why, for example.
I merged a few of the prep commits.

I merged the prep commits, did some squashing, and pushed back to this PR.  I think this is a reasonable implementation at this point, though there are some things that could be nicer, e.g. in the prep commits:

The tests in cb2c9b04b3b1755cf48784b17f3ea12222fd7d55 might be better written using mock objects and their assertion functions rather than defining nocoverage function blocks.
We have all these operations on lists of IDs; are we we're properly deduplicating that if one is fetching multiple messages with the same set of recipients?  In practice that'll be extremely common.
I assume we need some logic to flush items from this new cache when UserProfile objects change?
Functions like bulk_get_huddle_user_ids should probably be using .values() rather than fetching full model objects, to avoid extra Python overhead.  We don't use that technique often, but it can be extremely important in the bulk-message-fetch code paths (you've probably seen that we don't materialize full Message objects in a few places around this).  Similarly with full UserProfile objects when we're only going to access 3 fields in this code path.  This may be unnecessary optimization, but it may not be.

Ultimately, we'll want to do profiling when fetching 1K messages with this endpoint to determine the perf impact of this change and what makes sense here.  For profiling Python, the @profiled decorator in zerver/lib/profile.py as well as test-backend --profile can be helpful; for better analysis, line_profiler is really nice.  I can also test-deploy this on chat.zulip.org and compare average GET /messages runtimes before and after to see if it's an issue, but obviously it's a lot easier to be scientific with a local environment.  Want to play around with it and see what you can determine?  Print-logging is also fine.  I can imagine us including a unit test that just lives on this branch and we plan to eventually delete that sends 1K+ messages in its test setup, but you may find it's good enough to just do testing with the populate_db --num-messages.
The high-level goal is to have fetching display_recipients be not a large portion of GET /messages requests both in the uncached and cached cases.
Another thing I'm thinking about now, though: we had some somewhat pre-existing code to cache by recipient_id; and this add a by-user cache.  Do we really need the by-user caching logic?  It seems like we could easily write cache-flushing logic to find all huddles containing a user and flush those from the display_recipient cache when that data changes (via the Subscription table).  And with PMs, since one of the two users is the current user (who is common to all messages we're fetching in bulk), it'd be easy to split that in via post-processing if need be (I guess we have to do something slightly annoying to translate sender_id to a recipient_id, but that's not complex).

We have all these operations on lists of IDs; are we we're properly deduplicating that if one is fetching multiple messages with the same set of recipients? In practice that'll be extremely common.

I think we already have this covered - bulk_fetch_display_recipientstakes a Set of recipient_tuples - so if we're processing a bunch of messages with repeating recipients, all the repeated recipients will get rolled into one since we're using sets.

Another thing I'm thinking about now, though: we had some somewhat pre-existing code to cache by recipient_id; and this add a by-user cache. Do we really need the by-user caching logic?

I'm not sure if I understand, but if I do - we're primarily using the existing "by recipient_id cache":
    stream_display_recipients = generic_bulk_cached_fetch(
        cache_key_function=display_recipient_cache_key,
        query_function=stream_query_function,
        object_ids=[recipient[0] for recipient in stream_recipients],
        id_fetcher=stream_id_fetcher,
        cache_transformer=stream_cache_transformer,
    )  # type: Dict[int, str]

and
    personal_and_huddle_display_recipients = generic_bulk_cached_fetch(
        cache_key_function=display_recipient_cache_key,
        query_function=personal_and_huddle_query_function,
        object_ids=[recipient[0] for recipient in personal_and_huddle_recipients],
        id_fetcher=personal_and_huddle_id_fetcher,
        cache_transformer=personal_and_huddle_cache_transformer
    )

This is using display_recipient_cache_key in both cases. There's one small spot with a user cache- but that's just in the place where we have built a list of all the ids of users we need to fetch - and we do a bulk fetch of those UserProfiles, cached by the user-ids. And this is necessary because we need those UserProfiles (or at least some fields from them - as you've pointed out) to make the display_recipient dictionaries:
        # Fetch the needed UserProfiles:
        user_profiles = bulk_get_user_profile_by_id(list(user_ids_to_fetch))

def bulk_get_user_profile_by_id(uids: List[int]) -> Dict[int, UserProfile]:
    return generic_bulk_cached_fetch(
        # Use a separate cache key to protect us from conflicts with
        # the get_user_profile_by_id cache.
        # (Since we fetch without select_related() here)
        cache_key_function=display_recipient_bulk_get_users_by_id_cache_key,
        query_function=lambda ids: list(UserProfile.objects.filter(id__in=ids)),
        object_ids=uids
    )

Now - indeed this cache needs to get flushed when the relevant UserProfile fields change - silly me forgot about it, thanks for catching that :see_no_evil:

OK, makes sense,

I pushed some commits to make the types for display_recipient nicer, and then taking advantage of that, a commit with the change to only fetch the needed fields using .values(), instead of entire models.

Edited the main commit (messages: Don't use display_recipient values from cached message_dicts.) to properly delete from display_recipient.py's user cache if one of the relevant fields changes:
diff --git a/zerver/lib/cache.py b/zerver/lib/cache.py
index 3f18f0029..de07ab25d 100644
--- a/zerver/lib/cache.py
+++ b/zerver/lib/cache.py
@@ -404,6 +404,7 @@ def delete_display_recipient_cache(user_profile: 'UserProfile') -> None:
     recipient_ids = Subscription.objects.filter(user_profile=user_profile)
     recipient_ids = recipient_ids.values_list('recipient_id', flat=True)
     keys = [display_recipient_cache_key(rid) for rid in recipient_ids]
+    keys.append(display_recipient_bulk_get_users_by_id_cache_key(user_profile.id))
     cache_delete_many(keys)

Pushed some commits to make the types for display_recipient nicer, and then taking advantage of that, a commit with the change to only fetch the needed fields using .values(), instead of entire models.

I'll do some profiling next.

@timabbott I pushed an additional commit to stop putting raw_display_recipient in message dicts - because I don't think it's used by anything anymore?
Then I did some very simple timing measurements. In the last (DO NOT MERGE: Commit for profiling) commit you can see the code I used. I did this with populate_db -n100000 (so we're fetching 1000 messages out of 100000 - this seems closer to real scenarios than with a lower total number of messages). We fetch random 1000  or 10,000 messages, and measure the total fetching time and how much is spent in the newbulk_hydrate_recipient_info.
The time spent handling the display_recipient stuff seems negligible :thinking: The new code seems to be the same speed for 1000 message fetches (in what I'm pasting here the results seem the tiniest bit faster, but with more trials, it's essentially the same). For 10,000 message fetches it's very slightly slower:
FETCHING 1000 MESSAGES:

    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.002419710159301758
    Done
    Total time: 0.18411898612976074
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.002582073211669922
    Done
    Total time: 0.1749429702758789
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.0026607513427734375
    Done
    Total time: 0.1857755184173584

FETCHING 10000 MESSAGES:

    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:23:27.417 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.029844999313354492
    Done
    Total time: 1.5425536632537842
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:23:33.976 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.029630422592163086
    Done
    Total time: 1.5539748668670654
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:24:26.671 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.0361475944519043
    Done
    Total time: 1.5578269958496094
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:24:37.798 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Starting hydrating recipient
    Done hydrating in total time 0.02804255485534668
    Done
    Total time: 1.5139102935791016

NOW WITH OLD CODE:

FETCHING 1000 MESSAGES:
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:26:37.930 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 0.20970439910888672
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:26:41.229 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 0.20812153816223145
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:26:45.074 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 0.21609902381896973
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:26:49.386 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 0.21220636367797852

FETCHING 10000 MESSAGES:

    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:27:47.648 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 1.4868180751800537
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:27:53.271 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 1.4775846004486084
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:27:57.754 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 1.5022799968719482
    (zulip-py3-venv) vagrant@32de60bf2049:/srv/zulip$ ./manage.py custom_command
    2019-08-19 01:28:02.153 INFO [] Clearing memcached cache after migrations
    Fetching messages...
    Done
    Total time: 1.4833707809448242

@mateuszmandera this is great.  I expected performance to be a bit better with this transition done right, because we're doing sending/receiving less duplicated data from the cache (and for it to be worse if the transition was done wrong).  I added b25abf3de9147ce6bb60531eedb91046cf93c994 and have merged this (sans profiling commit).  Huge thanks for doing this!
@gnprice FYI; you may want to verify that the mobile issue that originally prompted this issue is indeed resolved if it's quick.


⚠ ClamAV has degraded performance

In fdf9e7a, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 204 ms

Resolved: ClamAV performance has improved in 51d0415.


Do not expose Buffer / process via insertModuleVars if builtins are disabled

I was wondering, if we could change browserify to not provide Buffer and process variables via insert-module-globals if builtins: false is specified in the browserify configuration.
There exists a number of libraries on npm that gracefully handle the fact that these variables might not be available, i.e. sax-js.
To properly support these libraries and clean browser bundles we must specify something like this at the moment which is error prone and verbose:
var b = browserify({
  builtins: false,
  insertGlobalVars: {
    process: function () {
      return 'undefined';
    },
    Buffer: function () {
      return 'undefined';
    }
  }
 });

insertGlobalVars will crash otherwise because these modules are not included in the bundle anyway due to builtins: false.
@substack I am happy to work on a PR for this if there is a slight change to get this improvement merged.

is this a duplicate of #1060?

Doesn't look like it is.


Segmentation model creates too many masks

Describe the bug
The model creates several overlapping masks on prediction. (picture 1)
To Reproduce
Steps to reproduce the behavior:

run predict_yolo_segmentation.py with 'masks' argument
See error

Expected behavior
Model should create distinct masks (one for each instance, and now its more - picture 2) that don't overlap. We cannot just take the largest mask (picture 3) since there could be several instances (picture 4)
Screenshots

Due to the change in project architecture, the use of YOLO face segmentation is no longer required.


Extract symbol loading code from OrbitApp into new SymbolLoader class

The main goal is to reduce the size of OrbitApp. OrbitApp now uses the new
SymbolLoader, which calls into OrbitApp through the
SymbolLoader::AppInterface interface.
A couple of extra methods need to be added to OrbitApp for the communication
between SymbolLoader and OrbitMainWindow.
Test: Auto symbol loading on Trata and locally. Also try manual loading for
errors (rather, "Partial" cases) and try stopping the download of a module.

@antonrohr Please also have a look at the first commit separately: it's removing symbol_files_currently_being_retrieved_ because I don't think it's serving any purpose, but maybe I'm missing something.

@antonrohr Please also have a look at the first commit separately: it's removing symbol_files_currently_being_retrieved_ because I don't think it's serving any purpose, but maybe I'm missing something.

The original idea was to prevent that a "retrieval process" is started multiple times. I guess that cannot happen anymore with auto symbol loading, aka I dont see a reason why this is still needed.


Docs: Issues in Chapter 4 cont...

Issue1:
http://pubs.marklogic.com:8011/9.0/guide/entity-services/codegen#id_81351
Typo.
Actual:
The generated configuration information always at least the following items,
Expected:
The generated configuration information always has at least the following items,
Issue2:
http://pubs.marklogic.com:8011/9.0/guide/entity-services/codegen#id_57834
Correction required. Refer #243 . Entity type should include atleast 2 props in the range index spec to make tuples:
If an entity type includes more than one property in the range index specification, a tuples option with the same name as the entity type for finding co-occurrences of the indexed properties.

Comments for fix in bugtrack 45291.
Assigned the bug to self and marked Test.
Looks good. It was my bad, I read "more than one" as "one". Shipping the bug


Aligne extra certificate name with default cert-manager names

This seems fine, though the file names provided by cert manager to the control plane are .pem files with the cert/key naming, but this might be an artifact of how those certs are setup.

Yes, that is to match https://github.com/neondatabase/helm-charts/blob/c5536eba33853032a4f16316201f0de4c35c1a52/charts/neon-proxy/templates/deployment.yaml#L85-L88
I'm not 100% sure that that file are in pem format though, so few more PR's may happen in that area

I'm not 100% sure that mounted files are in pem format, so few more PR's may happen in that area

They were in pem for the control plane.
It seems like a thing I'd do to opt for the alternate naming.


AddWhileContext() crashes for call to hub.load("...elmo/3").signatures["default"] under a tf.GradientTape

Hello,
I've just tried the following:
import tensorflow as tf
import tensorflow_hub as hub

elmo = hub.load('https://tfhub.dev/google/elmo/2')
test_input = tf.constant(['the cat is on the table'], dtype=tf.string)

with tf.GradientTape() as tape:
    result = elmo.signatures['default'](test_input)

And it throws AttributeError: 'NoneType' object has no attribute 'outer_context' when calling the loaded ELMo model. On the other hand, removing the tf.GradientTape() wrap up, the code works. Am I missing something?
elmo = hub.load('https://tfhub.dev/google/elmo/2')
test_input = tf.constant(['the cat is on the table'], dtype=tf.string)
result = elmo.signatures['default'](test_input)

Tensorflow config:
tensorflow-estimator==2.0.0
tensorflow-gpu==2.0.0
tensorflow-hub==0.6.0
Python config:
Python 3.5.3
EDIT
I've also tested BERT hub model without reporting any error.  I don't know if the problem is exclusively attributable to the ELMo hub model then. Anyone got it working?
Originally posted by @federicoruggeri in https://github.com/tensorflow/hub/issues/124#issuecomment-538948479

I can repro @federicoruggeri's report. It still crashes for https://tfhub.dev/google/elmo/3 when run against tensorflow and tensorflow_hub at Piper RevId 280105322.
The stack trace hints at a problem with while loop contexts:
AttributeError                            Traceback (most recent call last)
<ipython-input-5-99d2bb3c2e93> in <module>()
      1 with tf.GradientTape() as tape:
----> 2   result = elmo.signatures['default'](test_input)
      3

.../tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   1550       TypeError: For invalid positional/keyword argument combinations.
   1551     """
-> 1552     return self._call_impl(args, kwargs)
   1553
   1554   def _call_impl(self, args, kwargs, cancellation_manager=None):

.../tensorflow/python/eager/function.py in _call_impl(self, args, kwargs, cancellation_manager)
   1590       raise TypeError("Keyword arguments {} unknown. Expected {}.".format(
   1591           list(kwargs.keys()), list(self._arg_keywords)))
-> 1592     return self._call_flat(args, self.captured_inputs, cancellation_manager)
   1593
   1594   def _filtered_call(self, args, kwargs):

.../tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1696         possible_gradient_type,
   1697         executing_eagerly)
-> 1698     forward_function, args_with_tangents = forward_backward.forward()
   1699     if executing_eagerly:
   1700       flat_outputs = forward_function.call(

.../tensorflow/python/eager/function.py in forward(self)
   1422     """Builds or retrieves a forward function for this call."""
   1423     forward_function = self._functions.forward(
-> 1424         self._inference_args, self._input_tangents)
   1425     return forward_function, self._inference_args + self._input_tangents
   1426

.../tensorflow/python/eager/function.py in forward(self, inference_args, input_tangents)
   1184       (self._forward, self._forward_graph, self._backward,
   1185        self._forwardprop_output_indices, self._num_forwardprop_outputs) = (
-> 1186            self._forward_and_backward_functions(inference_args, input_tangents))
   1187     return self._forward
   1188

.../tensorflow/python/eager/function.py in _forward_and_backward_functions(self, inference_args, input_tangents)
   1330     outputs = self._func_graph.outputs[:self._num_inference_outputs]
   1331     return self._build_functions_for_outputs(
-> 1332         outputs, inference_args, input_tangents)
   1333
   1334

.../tensorflow/python/eager/function.py in _build_functions_for_outputs(self, outputs, inference_args, input_tangents)
    889             self._func_graph.inputs,
    890             grad_ys=gradients_wrt_outputs,
--> 891             src_graph=self._func_graph)
    892
    893       captures_from_forward = [

.../tensorflow/python/ops/gradients_util.py in _GradientsHelper(ys, xs, grad_ys, name, colocate_gradients_with_ops, gate_gradients, aggregation_method, stop_gradients, unconnected_gradients, src_graph)
    550     stop_gradient_ops = [t.op for t in stop_gradients]
    551     reachable_to_ops, pending_count, loop_state = _PendingCount(
--> 552         to_ops, from_ops, colocate_gradients_with_ops, func_graphs, xs_set)
    553
    554     # Iterate over the collected ops.

.../tensorflow/python/ops/gradients_util.py in _PendingCount(to_ops, from_ops, colocate_gradients_with_ops, func_graphs, xs_set)
    123   # 'loop_state' is None if there are no while loops.
    124   loop_state = control_flow_state.MaybeCreateControlFlowState(
--> 125       between_op_list, between_ops, colocate_gradients_with_ops)
    126
    127   # Initialize pending count for between ops.

.../tensorflow/python/ops/control_flow_state.py in MaybeCreateControlFlowState(between_op_list, between_ops, colocate_gradients_with_ops)
    777           loop_state.AddWhileContext(op, between_op_list, between_ops)
    778       else:
--> 779         loop_state.AddWhileContext(op, between_op_list, between_ops)
    780   return loop_state
    781

.../tensorflow/python/ops/control_flow_state.py in AddWhileContext(self, op, between_op_list, between_ops)
    574     if grad_state is None:
    575       # This is a new while loop so create a grad state for it.
--> 576       outer_forward_ctxt = forward_ctxt.outer_context
    577       if outer_forward_ctxt:
    578         outer_forward_ctxt = outer_forward_ctxt.GetWhileContext()

AttributeError: 'NoneType' object has no attribute 'outer_context'

Issue https://github.com/tensorflow/hub/issues/412 has discussed availability of ELMO in TF2 format (no timeline known yet).


“Type” & “Choose from list” drop downs are both blank

Thanks for making this!
Unfortunately when I try to use it I am running into a problem.
I type in the IP address of the Hub
Click “Get access code”  (access code is received)
“Access code received OK. Ready to save this config”
I click [Add]
The “Type” & “Choose from list” drop downs both state “Make and deploy config first”
I click [Done] and then [Deploy]
When I then double click on a node the “Type” & “Choose from list” drop downs are both blank (no dopdowns)
I have 6 rooms and Bulbs, Sonos, Sockets configured.
node-red-contrib-dirigera v1.0.1
Node-RED version: v3.1.0
Node.js  version: v18.18.2

Based on your nice and detailed explanation I then assume you done everything correct and I dont see why that did not work. I have 4 steps I would like you to test out.

What error do you see if any?
Try to send and input into this via NEW inject node. What do you see?
Does your input contain a "non" recognized topic?
Could you while being the the editor restart your flow. And then watch the debug panel for any error messages. If the config did not work there should be an error message here when you first deployed. You can reproduce this by restarting. What error message do you see?

What error do you see if any in debug panel or the status text underneath the node?

There are no errors in the debug panel, under the node or in the node-red log on first deploy

Try to send and input into this via NEW inject node. What do you see?

If I send a default new inject node message to the node I get the error message " getRoom cant find room name or id: -1" under the node, in the debug panel and console log

Does your input contain a "non" recognized topic?

If you mean the injected message any message results in the same error (even the two injects in the example)

Debug panel/ console on restart?

Nothing odd. The usual info messages ending in the started flows message.

I also tried modifying line 23 in dirigera.js to change the debug level to 5 but no messages are returned when deleting the hub and repeating the process.

Its all very consistent with the config deployment going wrong somehow. Good thinking about edit the .js file, but could you try to edit line 67? at set debug=5.
This should help revival where the error lies. I think the line 68 about testing the access code (login part) might not return the same result for you as it does in my setup. But lets see that you find out?

I just now pushed update v1.0.2 to improve error catch when creating config. Maybe this will catch the error that happens to you.

Thanks!
I upgraded to 1.0.2 but still have the same issue
Here’s the console output:
22 Oct 15:04:54 - [info] Starting flows
running logIn
running fetch GET on URL: https://<IP_ADDRESS>:8443/v1/users/me
22 Oct 15:04:54 - [info] Started flows
22 Oct 15:04:54 - [info] [mqtt-broker:c0b3747d0339db2b] Connected to broker: mqtt://192.168.0.XXX:1883
Returned headers response:200

api-version: 1.3.26
connection: close
content-length: 212
content-type: application/json; charset=utf-8
date: Sun, 22 Oct 2023 13:04:55 GMT
etag: W/"d4-A9QwU/5JBtRK1/R/S/0tozvtzc4"
running getDevice
running fetch GET on URL: https://<IP_ADDRESS>:8443/v1/devices
Returned headers response:200
api-version: 1.3.26
connection: close
content-length: 18133
content-type: application/json; charset=utf-8
date: Sun, 22 Oct 2023 13:05:13 GMT
etag: W/"46d5-0HYWrj7yu3HBspxmrwG7v42Lm5Y"
running getDevice
running getDevice

If I add a new Inject node and attach it to the node on an inject I get the following
running getRoom
running getDevice
running fetch GET on URL: https://<IP_ADDRESS>:8443/v1/devices
Returned headers response:200

api-version: 1.3.26
connection: close
content-length: 18133
content-type: application/json; charset=utf-8
date: Sun, 22 Oct 2023 13:07:52 GMT
etag: W/"46d5-inPW/ONoGf4jY4r8OsqVX8qHwnw"
22 Oct 15:07:52 - [error] [dirigera:067e9b4e8c61cd77] getRoom cant find room name or id: -1

I’ve used my dirigera script on flows.nodered.org http://flows.nodered.org/ to dump out https://<IP_ADDRESS>:8443/v1/devices
I don’t want to post it to GitHub (it contains the serial numbers of my Sonos gear and my homes lat / long ) but I can share it with you away from GitHub if it helps?

On 22 Oct 2023, at 13:52, zinen @.***> wrote:
Its all very consistent with the config deployment going wrong somehow. Good thinking about edit the .js file, but could you try to edit line 67? at set debug=5.
This should help revival where the error lies. I think the line 68 about testing the access code (login part) might not return the same result for you as it does in my setup. But lets see that you find out?
—
Reply to this email directly, view it on GitHub https://github.com/zinen/node-red-contrib-dirigera/issues/2#issuecomment-1774073086, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATLRNA4ASVGLILMDAOXMDTYAUCGPAVCNFSM6AAAAAA6KG63QSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUGA3TGMBYGY.
You are receiving this because you authored the thread.

Ahh. You are that @ukmoose, I used your flow as basic for making this node. So thank you!
Please do send me your device content to<EMAIL_ADDRESS>I see from your logs that the config indeed goes though and that your access token validates and answers back.
running fetch GET on URL: https://<IP_ADDRESS>:8443/v1/devices
Returned headers response:200
- api-version: 1.3.26
- connection: close
- content-length: 18133 // This mean theres plenty of data returned
- content-type: application/json; charset=utf-8 // This with the status code=200 means that all is okay and the retuned data is not som sort of error code or HTML returned.

So now I expect you will see the real problem in our browser. This node creates a API endpoint to transfer data from node-red server side to the client side dashboard. This means in my case that a URL of http://localhost:1880/ikeaDirigera/dirigera?nodeId=123343242blala(id of config node) is made accessible. When on the dashboard and open this node this API is queried. In case no deployed config is found then the returned message state that you should make and deploy first. So in your browser when you open this node - what error do you see in the console of the browser (e.g. developer tools its called in chrome)?

 Updated to version v1.0.3 just now. This basically should you throw the error to console and not actually fix your problem yet.  I'll test your device list later on my test client. Do you get a new error now?

Updated to v.1.0.3

Should be fixed in v1.0.4 - The issue it seems was that not all your devices is assigned to a room. I not added a no-room category to handle this.
Please do reply back with anything either working or still causing issues :)

Many thanks, it's working!


CLI option for refreshing a single image

I have implemented this functionality and it is now in the master branch.  It works like the normal refresh only you have the argument for toolpath.


Only one project allowed

I have two projects in Atom, no matter what FTP puts files in the first project in the list. FTP config is created there as well. No way to specify config or download to active project rather than first.

Duplicate #44

Duplicate #44


Resource recommendation support based on deep learning

Describe the feature
I mainly do some workload resource prediction methods, mainly using the currently popular deep learning technology. I would like to ask the community if there are any similar plans as a replacement for dsp  timing prediction.

@niconical we are doing deep learning prediction internally, and hope we can open source it later sometime. Can you describe more about your request or your learning here so we can have deeper discussion.


Emcee and MPI with mpi4py

General information:

emcee version:3.0rc2
platform: Ubuntu
installation method (pip/conda/source/other?): pip install

Problem description:
Hi all,
I am pretty new of MPI parallelization and this is indeed my first try to use it on a python code
I have tried a parallelization of emcee using mpi4py in a different way w.r.t. the minimal examples proposed in the documentation of the code (which actually uses multiprocessing).
The idea behind this workaround is that I want to have each process to write its own"backend" file when emcee is running which, to my knowledge so far, cannot be realized using the "pool" option of the sampler. Furthermore, w.r.t. the "pool" method I can draw much more model at
the same time with speed comparable to when I run the code without parallelization.
what I would like to know is if this is the right way to obtain what I wanted to achieve or if there's another way to do that using the "pool" method. The workaround works fine but I am concerned about the results I am drawing from this, are the results of each MCMC process really independent from one another in this way, or there's something I am missing ?
Best,
F.
Minimal example:

import emcee
from mpi4py import MPI

def lnlike(theta, x, y, yerr):
    m, b, lnf = theta
    model = m * x + b
    inv_sigma2 = 1.0/(yerr**2 + model**2*np.exp(2*lnf))
    return -0.5*(np.sum((y-model)**2*inv_sigma2 - np.log(inv_sigma2)))

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

sampler={}
backends= {}

filename = "examples_"+str(rank+1)+".h5"

ndim, nwalkers = 3, 10

# use backends option of emcee to store chains

backends[str(rank)]=emcee.backends.HDFBackend(filename)

backends[str(rank)].reset(nwalkers,ndim)

# create an initialization for the walkers

pos = [p0+ 1e-4*np.random.randn(ndim) for i in range(nwalkers)]

sampler[str(rank)] = emcee.EnsembleSampler(nwalkers, ndim, lnlike, args=(x, y, yerr),backend=backends[str(rank)])

#start the actual mcmc process

sampler[str(rank)].run_mcmc(pos, 10000,progress=True)

``

There isn't anything fundamentally wrong about this approach (it can actually be useful since you have many independent ensembles in this case). But one thing that you need to be careful about is making sure that the random number generator on each process gets properly initialized or you'll end up with chains that are correlated or even the same.

I ran into a similar situation. Is it possible to random.seed() the random numbers used in emcee? Does emcee already adopt a particular way of seeding the random number generator (in this way I won't have a control)?

The EnsembleSampler creates a RandomState (that it owns) when it's initialized so the easiest way to have control over the state is to run a np.random.seed before initializing the sampler. It would be good to add a seed argument to the sampler, but it's not yet implemented.


Helper methods for updating options syntax

This PR aims to offer a few useful tools that will make it easier to start using the new options syntax introduced in #3173. Currently the API is quite simple - see the unit tests for examples.

Here is the little script I am developing based on this (convertor.py:
from holoviews import opts
import pyparsing

while True:
    string = input('> ')
    try:
        options = opts._completer_reprs(string)
    except (SyntaxError, pyparsing.ParseException):
        print('Parse error. Please try again')
        continue

    print('# New import line')
    print('from holoviews import opts\n')
    print(', '.join(options))
    print('')

It gives you the new import and the completer syntax to use/adapt appropriately. For instance, you can just copy the keywords out if you don't need to qualify by the element.
python convertor.py
> %%opts Bivariate [bandwidth=0.5] (cmap='Blues') Points [tools=['box_select']] (size=2)
from holoviews import opts
opts.Bivariate(bandwidth=0.5, cmap='Blues'), opts.Points(size=2, tools=['box_select'])

And here is a script to run in examples:
import glob
import sys

import pyparsing
import holoviews as hv
from holoviews import opts, Cycle, Palette
from nbformat.current import read

try:
    import pygments
except:
    pygments = None

hv.extension('bokeh', 'matplotlib', 'plotly')
ns = {'Cycle':Cycle, 'Cycle, Palette':Palette}

for filename in glob.iglob('**/*.ipynb', recursive=True):
    if '.ipynb_checkpoints' in filename:
        continue

    with open(filename, 'r') as f:
        nb = read(f, 'json')

    for ws in nb.worksheets:
        for cell in ws.cells:
            if cell['cell_type'] == 'code':
                if '%opts' in cell['input']:
                    for line in cell['input'].splitlines():
                        if '%opts' in line:
                            msg = '\n{filename}:\n     {line}\n'
                            print(msg.format(filename=filename, line=line))

                            try:
                                options = opts._completer_reprs(line, ns=ns)
                            except (SyntaxError, pyparsing.ParseException):
                                print('Unexpected parse error')

                            code = '\n'.join(['from holoviews import opts',
                                              ', '.join(options)])
                            if pygments:
                                style = 'friendly'
                                term = pygments.formatters.Terminal256Formatter(style=style)
                                highlighted = pygments.highlight(code,
                                                        pygments.lexers.PythonLexer(),
                                term)
                                sys.stdout.write(highlighted)
                            else:
                                print(code)

                            while True:
                                string = input('\nContinue? ')
                                if string.strip() in ['y','Y', '']:
                                    break

@philippjfr I have updated five of the bokeh gallery examples in f1654d7 in a way I propose should be idiomatic. I would like to check you agree with this style before I continue.

As I work through these examples, I am trying to think of a style guide that describes the rules to apply. This doesn't have to be a style guide we recommend to users (though that would be nice) but it should at least be something we follow in our examples. Here is the provisional set of rules I am thinking of:

The .options method should only ever appear before display. In a notebook this means it should appear only at the end of a cell, in a script it might mean it should only appear before rendering.
Use .options as few times as possible. If you are building a compositional object, build it and call .options before display and not before. In a notebook, you can build a compositional object in stages if you view the pieces between cells.
Inline the Options objects from opts.Curve etc into the .options call if possible. Only create a named handle if you need programatic/conditional control (i.e a data structure that isn't static). If you do need such a data structure, it is still preferable to declare it just prior to where it is used in .options, after the data/elements are declared.
 In notebooks this means the expression before .options should be composed of handles and + and * and nothing else. If necessary make new handles on the composed pieces to achieve this or create a handle called composition and call .options on that. This is to avoid large, hard to read and potentially rather scary looking expressions and to help split up the declaration of the data/elements/displayed components from their appearance.
These rules apply for compositional objects only. For instance, if you are viewing a single element something like hv.Image(...).options(cmap='jet') is acceptable if the declaration of the element isn't too long. If it is long, declaring a handle is recommended (e.g im.options(cmap='jet')).

The "style guide" you laid out sounds good to me and roughly captures what we discussed yesterday.

I left one comment, otherwise this looks good to me.

I added another rule to the 'guide' to capture something I discussed with Philipp:

If the contents of .options does not fit comfortably in a single line, add a line break immediately afterwards and list the contents with a single extra indent.

So this:
(density_grid * point_grid).options(opts.Bivariate(bandwidth=0.5, cmap='Blues'),
                                                          opts.Points(size=2, tools=['box_select']))

Should be:
(density_grid * point_grid).options(
    opts.Bivariate(bandwidth=0.5, cmap='Blues'),
    opts.Points(size=2, tools=['box_select']))

And even though only one completer is used here:
xticks = ("Friday", "Saturday", "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday")
heatmap.options(opts.HeatMap(radial=True, start_angle=np.pi*19/14, width=600, height=600,
                             yticks=None, xticks=xticks xmarks=7, ymarks=3))

It should be:
xticks = ("Friday", "Saturday", "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday")
heatmap.options(
    opts.HeatMap(radial=True, start_angle=np.pi*19/14, width=600, height=600,
                 yticks=None, xticks=xticks xmarks=7, ymarks=3))

And here is another rule which I think has three parts, two of which I think are optional and one that shouldn't be:

(optional) In .options(...) try list element names in alphanumeric order (e.g opts.Curve before opts.HLine)
(optional) Try to list keywords in alphanumeric order.
(recommendation) List container type options last (if present) with opts.Overlay appearing before opts.Layout.

Unfortunately, following these rules means updating notebooks that might not even use any magics e.g  abe3a097.

Changes in the most recent commits look good, I think we have to think about a solution for both Links and Streams. I'd be happy to remap both of them inside the options call to ensure that they stay linked even when you don't use clone=False.

opts.defaults as a cleaner replacement to the line magic

I like that change; seems clear.

opts.linemagic and opts.cellmagic

I don't understand these objects.  They seem to be replacements for magics, not actually magics, so I don't think they should have the word "magic" in their name.  Can't you instead name them for what they are, even opts.fromlinemagic and opts.fromcellmagic if that's what they do (convert magic syntax into options objects)?  I don't have time to follow them up now, but I do want to register a vote against those names given my current understanding of them...

They seem to be replacements for magics, not actually magics, ...

That is true and I am happy to rename them. But it is also true they don't really take line/cell magics as input (directly). Semantically it is something like linemagic_equivalent or linemagic_substitute but those are obviously horrible names.

@philippjfr 1bdf12fd highlights a number of issues with this example:

Having to conda install the matplotlib sample data.
yformatter='%d%%' not being propagated to the overlay level.

I hope this is the trickiest example to update as it took a while!

One other style to avoid. Instead of:
grid.options('Scatter', tools=['hover', 'box_select'],  fill_alpha=0.2, size=4)

We should now use:
grid.options(
   opts.Scatter(tools=['hover', 'box_select'], fill_alpha=0.2, size=4))

Why?

I don't think we should disallow that style but for consistency in the examples I agree we should go with the latter approach.

b47164bd shows one valid exception to the rule: you can use inline .options if it is in a dictionary/list or set comprehension.

One note I want to make, the reprs of dim are rather confusing. I would like them to be as similar to the original dim expression as possible.

Discussing things with @philippjfr and @jbednar, there is another update to the style guide:

Always put .options once before display if you can. There are situations where this is not reasonable and you have to use groups or cycles to disambiguate collections of the same element type. In such situations, the recommendation is to associate the .options call as closely as possible to the element.

For example, you can use the template styled_element = hv.Element(data, ...).options(...) where the goal is to keep the declaration before .options as short and as readable as possible (e.g data should already be defined as a handle). You should not declare these custom style elements inside larger compositional expressions involving + or * (or containers such as HoloMap).
Note that this new rule actually subsumes the early exception around list/set/dict comprehensions.

@philippjfr I think this is about ready to review/merge though I still haven't been able to fix choropleth_data_link even using clone=False (I get an exception).

Okay fixed those tests and it's passing now, merging.


resolves #11 use Iterable instead of Iterator

I change to this branch and it fails complaining about groovydoc
dev@bf958d774356:~/ws$ ./gradlew clean build
:clean
:compileJava UP-TO-DATE
:compileGroovy
:processResources
:classes
:jar
:groovydoc
Trying to override old definition of task fileScanner
:groovydoc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':groovydoc'.
> groovydoc doesn't support the "noTimestamp" attribute

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 8.013 secs

It seems to be something related to latests versions of gradle (2.13 and 2.14). It seems groovydoc requires to use groovy-all. I think it's not big issue since we're already getting groovy-all as a spock transitive dependency.
Bottom line, I would change
compile('org.codehaus.groovy:groovy:2.4.6')
to this
compile('org.codehaus.groovy:groovy-all:2.4.6')

Thanks @mariogarcia, I bumped the groovy version to the latest also

It looks good to me. Do you want to add anything else ?

probably not to this one, I have a few other things in mind for the next release. You mentioned finding a bug, is there something else that should be addressed?

Yes, I forgot to add it. I've just done it. It has to do with Gen.these. Check it out and see if it you want to add it to this version. It's a bug, so you could always add a minor bug fixing release like 5.0.1


[Lake] Parallelize fetching data across >>1 tokens

Background / motivation
We currently fetch data from Binance one token at at time.
So if there are 10 tokens (BTC, ETH, DOT, ..) then it takes 10x longer.
TODOs / DoD

[ ] Parallelize fetching data, in lake/ohclv_data_factory.py. It's likely a straightforward conversion of a for loop

Related github issues

#804 "[Sim, make $] Make benchmarking parallel, via threading"

Note: we could also parallelize grabbing data within a feed. However this isn't as easy, because the algorithm needs to check what's there already. So consider this for later. But also maybe not needed because at some point we'll have historical data repo / bundle.

cc @calina-c @idiom-bytes

I don't recommend doing this now, since the structure is changing for lake either way. It will only result in either difficult conflicts or entirely lost while fixing said conflicts. I agree it is a good thing, but we should wait until the lake part is done.

I don't recommend doing this now, since the structure is changing for lake either way. It will only result in either difficult conflicts or entirely lost while fixing said conflicts. I agree it is a good thing, but we should wait until the lake/ETL part is done.

OK. Makes sense.


Random Battle: Fix Adaptability checks for Pokemon w/ 2 types

There's no need to check it if it has one type. We already have STAB checks for that.
Just make it if (((hasAbility['Adaptability'] && template.types.length > 1 && !!counter[template.types[0]] && !!counter[template.types[1]])

(I updated my comment because he had a mistake)

Darn I just left the office after making that change according to yours and won't have computer access for at least 4 hours

You did it wrong... I guess you didn't copy/paste.


Patching breaks pod rollouts

After adding this addmission controller, sometimes this error breaks helm release rollouts
warning: Upgrade "app" failed: cannot patch "app" with kind Deployment: Deployment.apps "app" is invalid: spec.template.spec.containers[0].env[38].valueFrom: Invalid value: "": may not be specified when `value` is not empty

We're using flux as well, but this error is popping up randomly in some contexts, does not seem to related to flux.

There is no "app" label value anywhere in this project, please check your k8s environment for a deployment named "app" and troubleshoot from there.
If you run in a shared K8s cluster, please make sure to limit the blast radius of the webhook to the applicable namespace(s) by using namespace selectors.

The all here is just a placeholder, I replace the original app's name, so that's not relevant. We've limited the namespaces as week, so shouldn't be a problem. Maybe any other ideas?

Well, without more details it's a wild guess. I would look at the deployment manifest under the first container for the 38'th element of the env array for clues.

Very sorry for Friday confusion, yes, that's our completely unrelated fault. Just the timing was bad with rolling out the your controller at the same time. Sorry again, closing the issue.


Scaffolded operators should depend on latest community.kubernetes collection

Feature Request
/language ansible
Currently, Ansible-based operators depend on community.kubernetes<1.0.0, but we should upgrade to the latest.

Bumping to 1.3 and assigning Jesus.

Related to PR #4190

Related to PR #4190

@asmacdo @jmrodri @fabianvf does this dependency bump depend on any changes upstream? Is this blocked?

This was addressed in PRs: #4594 and #4734