bosh did not cleanup VM resources on failure

Hi,
It looks like bosh did not cleanup the resources on create_vm failures.
create-vm failed on timeout + failed delete, and second attempt to deploy the same VM failed on "private IP address is duplicated" error.
CPI Release Version: v22
Please note I removed the password and keys from the logs/
First attempt:
{code}
Deploying:
Creating instance 'bosh/0':
Creating VM:

  Creating vm with stemcell cid 'm-uf6b6vtpujfo9zq2104i':

    CPI 'create_vm' method responded with error: CmdError{"type":"Bosh::Clouds::CloudError","message":"input=`{\"method\": \"create_vm\",\"arguments\": [\"dc87ab78-3ba7-4b92-5f30-77bf31aa7c85\",\"m-uf6b6vtpujfo9zq2104i\",{\"availability_zone\": \"cn-shanghai-e\",\"ephemeral_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"},\"instance_type\": \"ecs.c5.xlarge\",\"key_pair_name\": \"bosh-ssh-cf-canary2019050202420793370000000d\",\"system_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"}},{\"private\": {\"cloud_properties\": {\"security_group_ids\": [\"sg-uf6866edq67pmgnxkaz7\"],\"vswitch_id\": \"vsw-uf6jkcjaf400oeb5rgsx7\"},\"default\": [\"dns\",\"gateway\"],\"dns\": [\"<IP_ADDRESS>\",\"<IP_ADDRESS>\"],\"gateway\": \"<IP_ADDRESS>\",\"ip\": \"<IP_ADDRESS>\",\"netmask\": \"<IP_ADDRESS>\",\"type\": \"manual\"}},[],{\"bosh\": {\"mbus\": {\"cert\": {\"ca\": \"xxx\"}}],\"context\": {\"director_uuid\": \"305e8ff0-2651-488f-4c6c-a5fc711189b4\",\"vm\": {\"stemcell\": {\"api_version\": 2}}},\"api_version\": 1}` update i-uf61h0f16m81g4i1p7q9 failed delete failed [SDK.TimeoutError] The request timed out 6 times(6 for retry), perhaps we should have the threshold raised a little?\ncaused by:\nPost https://ecs-cn-hangzhou.aliyuncs.com/?AccessKeyId=Action=DescribeInstanceAttribute\u0026Format=JSON\u0026InstanceId=i-uf61h0f16m81g4i1p7q9\u0026RegionId=cn-shanghai\u0026SecurityToken=\u0026Signature=xxx!D(MISSING)\u0026SignatureMethod=HMAC-SHA1\u0026SignatureNonce=0c63d8aa681c45c18547ac9213b8cf0b\u0026SignatureType=\u0026SignatureVersion=1.0\u0026Timestamp=2019-05-03T09%!A(MISSING)46%!A(MISSING)38Z\u0026Version=2014-05-26: net/http: request canceled (Client.Timeout exceeded while awaiting headers): input=`{\"method\": \"create_vm\",\"arguments\": [\"dc87ab78-3ba7-4b92-5f30-77bf31aa7c85\",\"m-uf6b6vtpujfo9zq2104i\",{\"availability_zone\": \"cn-shanghai-e\",\"ephemeral_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"},\"instance_type\": \"ecs.c5.xlarge\",\"key_pair_name\": \"bosh-ssh-cf-canary2019050202420793370000000d\",\"system_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"}},{\"private\": {\"cloud_properties\": {\"security_group_ids\": [\"sg-uf6866edq67pmgnxkaz7\"],\"vswitch_id\": \"vsw-uf6jkcjaf400oeb5rgsx7\"},\"default\": [\"dns\",\"gateway\"],\"dns\": [\"<IP_ADDRESS>\",\"<IP_ADDRESS>\"],\"gateway\": \"<IP_ADDRESS>\",\"ip\": \"<IP_ADDRESS>\",\"netmask\": \"<IP_ADDRESS>\",\"type\": \"manual\"}},[],{\"bosh\": {\"mbus\": {\"cert\\"$xxx/\"}}],\"context\": {\"director_uuid\": \"305e8ff0-2651-488f-4c6c-a5fc711189b4\",\"vm\": {\"stemcell\": {\"api_version\": 2}}},\"api_version\": 1}` change i-uf61h0f16m81g4i1p7q9 to Running failed: [SDK.TimeoutError] The request timed out 6 times(6 for retry), perhaps we should have the threshold raised a little?\ncaused by:\nPost https://ecs-cn-hangzhou.aliyuncs.com/?AccessKeyId=\uAction=DescribeInstanceAttribute\u0026Format=JSON\u0026InstanceId=i-uf61h0f16m81g4i1p7q9\u0026RegionId=cn-shanghai\u0026SecurityToken=\u0026Signature=xxS6vbswq%!B(MISSING)0lQOvunCsyi2he2Go%!D(MISSING)\u0026SignatureMethod=HMAC-SHA1\u0026SignatureNonce=6ce25ba1710b45109e763deacd60695b\u0026SignatureType=\u0026SignatureVersion=1.0\u0026Timestamp=2019-05-03T09%!A(MISSING)40%!A(MISSING)38Z\u0026Version=2014-05-26: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)","ok_to_retry":false}

Exit code 1
{code}
Second attempt to deploy the VM:
{code}
Uploading stemcell 'bosh-alicloud-kvm-ubuntu-xenial-go_agent/250.23'... Skipped [Stemcell already uploaded] (00:00:00)
Started deploying
Creating VM for instance 'bosh/0' from stemcell 'm-uf6b6vtpujfo9zq2104i'... Failed (00:03:47)
Failed deploying (00:03:47)
Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)
Deploying:
Creating instance 'bosh/0':
Creating VM:

  Creating vm with stemcell cid 'm-uf6b6vtpujfo9zq2104i':

    CPI 'create_vm' method responded with error: CmdError{"type":"Bosh::Clouds::CloudError","message":"input=`{\"method\": \"create_vm\",\"arguments\": [\"23f2cace-a690-4a9c-7063-f710ed3fa596\",\"m-uf6b6vtpujfo9zq2104i\",{\"availability_zone\": \"cn-shanghai-e\",\"ephemeral_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"},\"instance_type\": \"ecs.c5.xlarge\",\"key_pair_name\": \"bosh-ssh-cf-canary2019050202420793370000000d\",\"system_disk\": {\"category\": \"cloud_efficiency\",\"encrypted\": true,\"size\": \"51_200\"}},{\"private\": {\"cloud_properties\": {\"security_group_ids\": [\"sg-uf6866edq67pmgnxkaz7\"],\"vswitch_id\": \"vsw-uf6jkcjaf400oeb5rgsx7\"},\"default\": [\"dns\",\"gateway\"],\"dns\": [\"<IP_ADDRESS>\",\"<IP_ADDRESS>\"],\"gateway\": \"<IP_ADDRESS>\",\"ip\": \"<IP_ADDRESS>\",\"netmask\": \"<IP_ADDRESS>\",\"type\": \"manual\"}},[],{\"bosh\": {\"mbus\": {\"cert\": {\"ca\": \"}}],\"context\": {\"director_uuid\": \"305e8ff0-2651-488f-4c6c-a5fc711189b4\",\"vm\": {\"stemcell\": {\"api_version\": 2}}},\"api_version\": 1}` create instance failed with input={\"Scheme\":\"HTTPS\",\"Method\":\"POST\",\"Domain\":\"ecs-cn-hangzhou.aliyuncs.com\",\"Port\":\"\",\"RegionId\":\"\",\"AcceptFormat\":\"JSON\",\"QueryParams\":{\"AccessKeyId\":\"\",\"Action\":\"CreateInstance\",\"ClientToken\":\"bosh-cpi-1556877860501762803-4e75e098-b524-41bc-b491-b32d1588e63\",\"DataDisk.1.Category\":\"cloud_efficiency\",\"DataDisk.1.DeleteWithInstance\":\"true\",\"DataDisk.1.Encrypted\":\"true\",\"DataDisk.1.Size\":\"50\",\"Format\":\"JSON\",\"ImageId\":\"m-uf6b6vtpujfo9zq2104i\",\"InstanceChargeType\":\"PostPaid\",\"InstanceType\":\"ecs.c5.xlarge\",\"IoOptimized\":\"optimized\",\"KeyPairName\":\"bosh-ssh-cf-canary2019050202420793370000000d\",\"PrivateIpAddress\":\"<IP_ADDRESS>\",\"RegionId\":\"cn-shanghai\",\"SecurityGroupId\":\"sg-uf6866edq67pmgnxkaz7\",\"SecurityToken\":\"\",\"Signature\":\"psCnr1vR1cxyZ/pLIY73196AZ+w=\",\"SignatureMethod\":\"HMAC-SHA1\",\"SignatureNonce\":\"4ee409dbcc8643b0948b1733382a4945\",\"SignatureType\":\"\",\"SignatureVersion\":\"1.0\",\"SpotPriceLimit\":\"0.000000\",\"SystemDisk.Category\":\"cloud_efficiency\",\"SystemDisk.Size\":\"50\",\"Timestamp\":\"2019-05-03T10:08:06Z\",\"UserData\":\"eyJyZWdpc3RyeSI6eyJlbmRwb2ludCI6Imh0dHA6Ly9yZWdpc3RyeTplZWRlNmZmZC1hMWM0LTQzMjMtNzZhOS04MDY5NDg2Yjk2YTFAMTI3LjAuMC4xOjY5MDEifX0=\",\"VSwitchId\":\"vsw-uf6jkcjaf400oeb5rgsx7\",\"Version\":\"2014-05-26\",\"ZoneId\":\"cn-shanghai-e\"},\"Headers\":{\"Accept-Encoding\":\"identity\",\"Content-Type\":\"application/x-www-form-urlencoded\",\"x-sdk-client\":\"golang/1.0.0\",\"x-sdk-core-version\":\"0.0.1\",\"x-sdk-invoke-type\":\"normal\"},\"FormParams\":{},\"Content\":\"\",\"ResourceOwnerId\":\"\",\"HpcClusterId\":\"\",\"SecurityEnhancementStrategy\":\"\",\"KeyPairName\":\"bosh-ssh-cf-canary2019050202420793370000000d\",\"SpotPriceLimit\":\"0.000000\",\"DeletionProtection\":\"\",\"ResourceGroupId\":\"\",\"HostName\":\"\",\"Password\":\"\",\"Tag\":null,\"AutoRenewPeriod\":\"\",\"NodeControllerId\":\"\",\"Period\":\"\",\"DryRun\":\"\",\"OwnerId\":\"\",\"VSwitchId\":\"vsw-uf6jkcjaf400oeb5rgsx7\",\"PrivateIpAddress\":\"<IP_ADDRESS>\",\"SpotStrategy\":\"\",\"PeriodUnit\":\"\",\"InstanceName\":\"\",\"AutoRenew\":\"\",\"InternetChargeType\":\"\",\"ZoneId\":\"cn-shanghai-e\",\"InternetMaxBandwidthIn\":\"\",\"UseAdditionalService\":\"\",\"ImageId\":\"m-uf6b6vtpujfo9zq2104i\",\"ClientToken\":\"bosh-cpi-1556877860501762803-4e75e098-b524-41bc-b491-b32d1588e63\",\"VlanId\":\"\",\"SpotInterruptionBehavior\":\"\",\"IoOptimized\":\"optimized\",\"SecurityGroupId\":\"sg-uf6866edq67pmgnxkaz7\",\"InternetMaxBandwidthOut\":\"\",\"Description\":\"\",\"SystemDiskCategory\":\"cloud_efficiency\",\"UserData\":\"eyJyZWdpc3RyeSI6eyJlbmRwb2ludCI6Imh0dHA6Ly9yZWdpc3RyeTplZWRlNmZmZC1hMWM0LTQzMjMtNzZhOS04MDY5NDg2Yjk2YTFAMTI3LjAuMC4xOjY5MDEifX0=\",\"PasswordInherit\":\"\",\"InstanceType\":\"ecs.c5.xlarge\",\"InstanceChargeType\":\"PostPaid\",\"DeploymentSetId\":\"\",\"InnerIpAddress\":\"\",\"ResourceOwnerAccount\":\"\",\"OwnerAccount\":\"\",\"SystemDiskDiskName\":\"\",\"RamRoleName\":\"\",\"DedicatedHostId\":\"\",\"ClusterId\":\"\",\"CreditSpecification\":\"\",\"DataDisk\":[{\"DiskName\":\"\",\"SnapshotId\":\"\",\"Size\":\"50\",\"Encrypted\":\"true\",\"Description\":\"\",\"Category\":\"cloud_efficiency\",\"Device\":\"\",\"DeleteWithInstance\":\"true\"}],\"SystemDiskSize\":\"50\",\"SystemDiskDescription\":\"\"} : over max retry: SDK.ServerError\nErrorCode: InvalidPrivateIpAddress.Duplicated\nRecommend: \nRequestId: 679E0D32-F871-45D8-9D9C-8470CA50751E\nMessage: Specified private IP address is duplicated.","ok_to_retry":false}

Exit code 1
{code}

HI  @OlegGerber This issue came from network connection failed leads deleting vm failed. I will push the product team to accelerate the specified request domain to avoid the network connection.
In addition, I will provide a parameter to remove the left vms when deploying a new bosh director. Does it can meet you?

Hi @xiaozhu36,
Can you share more insights on the connectivity issue that you see?
what is the "specified request domain" that need to be accelerated?
Thanks,
Alex

When you access ecs or other services, the CPI will get an endpoint based on the request region. When region is cn-shanghai, its endpoint is ecs-cn-hangzhou.aliyuncs.com, and this endpoint will be accelerated when your local host is not in China mainland to avoid some needless network issues.

Hi @xiaozhu36,
We are using hosts located in China main land to access this endpoint.
maybe the reason for the error is not network in this case?

The local host is not your computer, but the machine that sent the CPI request

yes i understand, we run the bosh director on a VM deployed in China main land.

Can you ping the endpoint ecs-cn-hangzhou.aliyuncs.com in your director? like #76

Hi @xiaozhu36,
See the
bosh/eff5:~$ nslookup  ecs-cn-hangzhou.aliyuncs.com
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>#53
ecs-cn-hangzhou.aliyuncs.com	canonical name = popunify-public.aliyuncs.com.
popunify-public.aliyuncs.com	canonical name = popunify-public.aliyuncs.com.gds.alibabadns.com.
Name:	popunify-public.aliyuncs.com.gds.alibabadns.com
Address: <IP_ADDRESS>
ping ecs-cn-hangzhou.aliyuncs.com
PING popunify-public.aliyuncs.com.gds.alibabadns.com (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=49 time=3.09 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=49 time=3.04 ms

Hi OlegGerber, Could you provide a test case so that we can reproduce the issue, please? It is difficult for us to reproduce the issue, appreciate your kindly help.

Hi @lemondlut,
This issue is not reproduced on every bosh deployment but we see it frequently, almost every day.
Please see the error to understand the flow:

The bosh instance update or deployment from scratch fails with timeout issue:

Task 1780

Task 1780 | 04:31:02 | Preparing deployment: Preparing deployment (00:00:01)
Task 1780 | 04:31:03 | Preparing deployment: Rendering templates (00:00:01)
Task 1780 | 04:31:04 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 1780 | 04:31:04 | Updating instance postgresql: postgresql/a372be1a-bf03-4b6f-958e-316d41d10ef5 (0) (canary) (00:11:44)
                    L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm","arguments": ["025ccff7-c09f-4258-99db-ef746bf23fc0","m-uf6216ctez1mji005iyz",{"availability_zone": "cn-shanghai-f","ephemeral_disk": {"encrypted": true,"size": 20480},"instance_type": "ecs.g5.4xlarge"},{"sf_bosh_services": {"type": "manual","ip": "<IP_ADDRESS>","netmask": "<IP_ADDRESS>","cloud_properties": {"security_group_ids": ["sg-uf62qvrx5zsy47bfccy3","sg-uf6e1i35512nblhamgvi"],"vswitch_id": "vsw-uf6rudkz8c8d03hfanm5a"},"default": ["dns","gateway"],"dns": ["<IP_ADDRESS>","<IP_ADDRESS>"],"gateway": "<IP_ADDRESS>"}},["d-uf6erxni5w5ce6wl9tsm"],{"bosh": {"blobstores": [{"options": {"endpoint": "https://<IP_ADDRESS>:25250","password": "","tls": {"cert": {"ca": "-----BEGIN CERTIFICATE-----
----END RSA PRIVATE KEY-----\n"}},"group": "bosh-sf-service-fabrik-1279-ccbfe480-9d34-4909-9c08-f8a3041d8b9d-postgresql","groups": ["bosh-sf","service-fabrik-1279-ccbfe480-9d34-4909-9c08-f8a3041d8b9d","postgresql","bosh-sf-service-fabrik-1279-ccbfe480-9d34-4909-9c08-f8a3041d8b9d","service-fabrik-1279-ccbfe480-9d34-4909-9c08-f8a3041d8b9d-postgresql","bosh-sf--service-fabrik-1279-ccbfe480-9d34-4909-9c08-f8a3041d8b9d-postgresql"]}}],"context": {"director_uuid": "6fc5522b-8dba-4684-986c-e2c2082f2b8c","request_id": "cpi-102736","vm": {"stemcell": {"api_version": 2}}},"api_version": 1}` change i-uf6e6nqcgk7ddziglw48 to Running failed: change instance i-uf6e6nqcgk7ddziglw48 to Running timeout' in 'create_vm' CPI method (CPI request ID: 'cpi-102736')
.....

Task 1780 Started  Mon Jul  1 04:31:02 UTC 2019
Task 1780 Finished Mon Jul  1 04:42:48 UTC 2019
Task 1780 Duration 00:11:46
Task 1780 error

Capturing task '1780' output:
  Expected task '1780' to succeed but state is 'error'

Exit code 1

The second attempt for the same bosh deployment fails with the "ErrorCode: InvalidPrivateIpAddress.Duplicated", which means the VM from the last deployment not deleted from the landscape, despite the fact bosh task failed.

Task 1813

Task 1813 | 05:03:21 | Preparing deployment: Preparing deployment (00:00:01)
Task 1813 | 05:03:22 | Preparing deployment: Rendering templates (00:00:01)
Task 1813 | 05:03:23 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 1813 | 05:03:23 | Creating missing vms: postgresql/a372be1a-bf03-4b6f-958e-316d41d10ef5 (0) (00:03:36)
                    L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm","arguments": ["16922489-4467-4fad-ab1a-dad39a49d9b0","m-uf6216ctez1mji005iyz",{"availability_zone": "cn-shanghai-f","ephemeral_disk": {"encrypted": true,"size": 20480},"instance_type": "ecs.g5.4xlarge"},{"sf_bosh_services": {"type": "manual","ip": "<IP_ADDRESS>","netmask": "<IP_ADDRESS>","cloud_properties": {"security_group_ids": ["sg-uf62qvrx5zsy47bfccy3","sg-uf6e1i35512nblhamgvi"],"vswitch_id": "vsw-uf6rudkz8c8d03hfanm5a"},"default": ["dns","gateway"],"dns": ["<IP_ADDRESS>","<IP_ADDRESS>"],"gateway": "<IP_ADDRESS>"}},["d-uf6erxni5w5ce6wl9tsm"],{"bosh": {"blobstores": [{"options": {"endpoint": "https://<IP_ADDRESS>:25250","password": "","
.....
"ResourceOwnerAccount":"","OwnerAccount":"","SystemDiskDiskName":"","RamRoleName":"","DedicatedHostId":"","ClusterId":"","CreditSpecification":"","DataDisk":[{"DiskName":"","SnapshotId":"","Size":"20","Encrypted":"true","Description":"","Category":"cloud_efficiency","KMSKeyId":"","Device":"","DeleteWithInstance":"true"}],"SystemDiskSize":"40","SystemDiskDescription":""} : over max retry: SDK.ServerError
ErrorCode: InvalidPrivateIpAddress.Duplicated
Recommend:
RequestId: DEF2F3D9-010C-4F8F-880C-5C1BA12A20CA
Message: Specified private IP address is duplicated.' in 'create_vm' CPI method (CPI request ID: 'cpi-289838')
...

Task 1813 Started  Mon Jul  1 05:03:21 UTC 2019
Task 1813 Finished Mon Jul  1 05:06:59 UTC 2019
Task 1813 Duration 00:03:38
Task 1813 error

Capturing task '1813' output:
  Expected task '1813' to succeed but state is 'error'

Exit code 1

HI @OlegGerber Although, the bosh-alicloud- cpi has added rollback when running create vm failed, it can not avoid connection timeout. Now, we have configured global acceleration service for the endpoint ecs-cn-hangzhou.aliyuncs.com and it will avoid the connection timeout error when you access China mainland region service. Besides, I enhanced  the rollback and it will retry 10 times when rollback failed. Now, it has been supported by the latest version 24.0.0. Please check it. Thanks.

HI @xiaozhu36 ,
It seems to be the issue with "InvalidIPAddress.AlreadyUsed" still reproducing on our landscapes during the component update.
However, the error message now is: "Message: The specified IPAddress is already used by other resource.' in 'create_vm' CPI method (CPI request ID: 'cpi-465242')"
Task 17061 | 19:40:56 | Compiling packages: elasticsearch/6afebaf8a3637d6e2aae6db61aafaa4abd7655bb28a1c9a25ec9118cb245562e (00:01:43)
Task 17061 | 19:41:53 | Compiling packages: python-3.6/976f472c218622aca411c953bc97d9d47665fd8f (00:02:49)
Task 17061 | 19:41:53 | Compiling packages: curator/29f3fa707032321547ccd95b58846d630eb3ff4bf80de879605796827d602f21 (00:00:10)
Task 17061 | 19:42:22 | Updating instance elasticsearch_master: elasticsearch_master/5133fda1-5ee1-4f65-a852-4d5eba9bbd0a (0) (canary) (00:03:00)
Task 17061 | 19:45:22 | Updating instance elasticsearch_master: elasticsearch_master/d1f98524-bfd1-4c87-894d-b2bd9e72f836 (1) (00:03:13)
Task 17061 | 19:48:35 | Updating instance elasticsearch_master: elasticsearch_master/bfb87525-35ab-451d-a248-b9e8b054eb00 (2) (00:03:26)
Task 17061 | 19:52:01 | Updating instance elasticsearch_data: elasticsearch_data/f4d83e89-6eb2-49f0-a689-54d7166cb5a0 (0) (canary) (00:04:31)
Task 17061 | 19:56:32 | Updating instance elasticsearch_data: elasticsearch_data/05c08b43-65fb-49a7-81b8-3b73e878e992 (1) (00:05:01)
Task 17061 | 20:01:34 | Updating instance zookeeper: zookeeper/3278e6cf-464f-44a3-a553-ef982116db4b (0) (canary) (00:02:45)
Task 17061 | 20:04:19 | Updating instance zookeeper: zookeeper/29eee6d1-f263-4296-adea-e4ebda51d32d (1) (00:04:03)
                     L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm","arguments": ["648af04d-48f0-4a0c-b4a3-51464463a962","m-uf6ff6n8gn1fqfg1i2fu",{"availability_zone": "cn-shanghai-f","ephemeral_disk": {"encrypted": true,"size": 20480},"instance_type": "ecs.c5.large"},{"logsearch": {"type": "manual","ip": "<IP_ADDRESS>","netmask": "<IP_ADDRESS>","cloud_properties": {"security_group_ids": ["sg-uf6076x9fu1dy4dzka87","sg-uf637q6lqzqr22b9tfw0","sg-uf6fuqggczshqf8tx4nq","sg-uf6aixmrq1fmw1ocjdsy","sg-uf65ns0580xqvcvk21jy","sg-uf6825diaozs5g81gqzm","sg-uf67m2mj8kzatnko0brf"],"vswitch_id": "vsw-uf6wf1cqwuseos3x1dpzd"},"default": ["dns","gateway"],"dns": ["<IP_ADDRESS>","<IP_ADDRESS>"],"gateway": "<IP_ADDRESS>"}},["d-uf60jpji0rlh06c6bvw8"],{"bosh": {"blobstores": [{"options": {"endpoint": "https://<IP_ADDRESS>:25250","password": \n"}},"group": "bosh-cf-cn40-staging-logsearch-zookeeper","groups": ["bosh-cf-cn40-staging","logsearch","zookeeper","bosh-cf-cn40-staging-logsearch","logsearch-zookeeper","bosh-cf-cn40-staging-logsearch-zookeeper"]}}],"context": {"director_uuid": "113d6236-e1e5-440d-88cc-f3b9824a934d","request_id": "cpi-465242","vm": {"stemcell": {"api_version": 2}}},"api_version": 1}` create instance failed with input={"Scheme":"HTTPS","Method":"POST","Domain":"ecs-cn-hangzhou.aliyuncs.com","Port":"","RegionId":"","ReadTimeout":0,"ConnectTimeout":0,"AcceptFormat":"JSON","QueryParams":{"AccessKeyId":"","Action":"CreateInstance","ClientToken":"Bosh-Cpi-CreateInstance-1564689889-284eb0e6-12c8-4b57-9c70-7e8b2","DataDisk.1.Category":"cloud_efficiency","DataDisk.1.DeleteWithInstance":"true","DataDisk.1.Encrypted":"true","DataDisk.1.Size":"20","Format":"JSON","ImageId":"m-uf6ff6n8gn1fqfg1i2fu","InstanceChargeType":"PostPaid","InstanceType":"ecs.c5.large","IoOptimized":"optimized","PrivateIpAddress":"<IP_ADDRESS>","RegionId":"cn-shanghai","SecurityGroupId":"sg-uf6076x9fu1dy4dzka87","SecurityToken":"","Signature":"=","SignatureMethod":"HMAC-SHA1","SignatureNonce":"945d320b331d47c19b68ff14a365c219","SignatureType":"","SignatureVersion":"1.0","SpotPriceLimit":"0.000000","SystemDisk.Category":"cloud_efficiency","SystemDisk.Size":"40","Timestamp":"2019-08-01T20:08:21Z","UserData":"=","VSwitchId":"vsw-uf6wf1cqwuseos3x1dpzd","Version":"2014-05-26","ZoneId":"cn-shanghai-f"},"Headers":{"Accept-Encoding":"identity","Content-Type":"application/x-www-form-urlencoded","x-sdk-client":"golang/1.0.0","x-sdk-core-version":"0.0.1","x-sdk-invoke-type":"normal"},"FormParams":{},"Content":"","ResourceOwnerId":"","HpcClusterId":"","SecurityEnhancementStrategy":"","KeyPairName":"","SpotPriceLimit":"0.000000","DeletionProtection":"","ResourceGroupId":"","HostName":"","Password":"","Tag":null,"AutoRenewPeriod":"","NodeControllerId":"","Period":"","DryRun":"","OwnerId":"","CapacityReservationPreference":"","VSwitchId":"vsw-uf6wf1cqwuseos3x1dpzd","PrivateIpAddress":"<IP_ADDRESS>","SpotStrategy":"","PeriodUnit":"","InstanceName":"","AutoRenew":"","InternetChargeType":"","ZoneId":"cn-shanghai-f","InternetMaxBandwidthIn":"","UseAdditionalService":"","ImageId":"m-uf6ff6n8gn1fqfg1i2fu","ClientToken":"Bosh-Cpi-CreateInstance-1564689889-284eb0e6-12c8-4b57-9c70-7e8b2","VlanId":"","SpotInterruptionBehavior":"","IoOptimized":"optimized","SecurityGroupId":"sg-uf6076x9fu1dy4dzka87","InternetMaxBandwidthOut":"","Description":"","SystemDiskCategory":"cloud_efficiency","CapacityReservationId":"","UserData":"=","":"","InstanceType":"ecs.c5.large","InstanceChargeType":"PostPaid","DeploymentSetId":"","InnerIpAddress":"","ResourceOwnerAccount":"","OwnerAccount":"","SystemDiskDiskName":"","RamRoleName":"","DedicatedHostId":"","ClusterId":"","CreditSpecification":"","DataDisk":[{"DiskName":"","SnapshotId":"","Size":"20","Encrypted":"true","Description":"","Category":"cloud_efficiency","KMSKeyId":"","Device":"","DeleteWithInstance":"true"}],"SystemDiskSize":"40","SystemDiskDescription":""} : over max retry: SDK.ServerError
ErrorCode: InvalidIPAddress.AlreadyUsed
Recommend:
RequestId: 9B9AA73D-EFDC-49FF-B42C-2732F6C1CA25
Message: The specified IPAddress is already used by other resource.' in 'create_vm' CPI method (CPI request ID: 'cpi-465242')

HI @OlegGerber Your private Ip <IP_ADDRESS> has been used by the instance i-uf64zjiazflkqpzqtvtl which created in 2019-08-02 14:46:42. Can you check it or your manifest?

Hi @xiaozhu36 ,
instance ID:i-uf64zjiazflkqpzqtvtl belong to the bosh instance zookeeper.29eee6d1-f263-4296-adea-e4ebda51d32d, which is failed on the bosh update process.

HI @OlegGerber Why did the bosh update process failed? What is your update command? Did the failure come from bosh cpi?

Hi @xiaozhu36 ,
Bosh update process failed on duplicate private IP (which is in use by VM itself)
The update is a regular bosh update procedure, which runs as a result of any manifest changes or stemcell update.
During the update, old VM may be deleted and a new one created, it seems to be that once VM deleted, it takes time to release the private IP. Few retries fix the issue.
Can you please check if it may be the reason for this kind of failures?

HI @OlegGerber You are right and I agree with you. But, actually, the CPI has added the retry strategy (times: 20, interval 10s) for this error: https://github.com/cloudfoundry-incubator/bosh-alicloud-cpi-release/blob/master/src/bosh-alicloud-cpi/alicloud/instance_manager.go#L100. Did you use the latest bosh cpi?

Hi @OlegGerber Is there any other error before the error InvalidIPAddress.AlreadyUsed? There is one case: connection timeout or request timeout results in the specified instance has been created, but bosh return a timeout error. When you retry bosh command, the IP has been used by the last VM.

HI @OlegGerber The latest cpi release 25.0.0 has been published and it has enlarged the retry times for the error InvalidIPAddress.AlreadyUsed. Please check it.

HI @OlegGerber The CPI release 26.0.0 has been published and it has the following improvements:

enlarge the timeout to 10 minutes when waiting for instance status, like running, deleted and so on
In some extreme case, the vm needs too long time to be running, so waiting too more time can avoid some internal error. If there is also timeout, the cpi will rollback and delete the vm. In the rollback, too more time can ensure the vm can be deleted successfully. It depends on backend system will kill the vm forcibly after 5 minutes and remove it.

add one method to cleanup the remaining network interface which occupies the vm's private ip
In some extreme case, if one vm is deleted in a non-normal ways, the vm's primarily network interface may be left and it will occupies the vm 's private ip. It also will throw an error InvalidIPAddress.AlreadyUsed when recreating the new vm. So, after deleting the instance, checkand cleanup the remaining network interface is necessary.

add extra error message when an error happened and it can help us to find the error reason more quickly.

Please use the release 26.0.0 to have a try.

Hi @xiaozhu36 ,
Unfortunately, this issue reproduced again with version: "26.0.0"

  stemcells:
+ - alias: default
+   os: ubuntu-xenial
+   version: '456.16'
- - alias: default
-   os: ubuntu-xenial
-   version: '456.12'
Task 3596

Task 3596 | 02:05:35 | Preparing deployment: Preparing deployment (00:00:16)
Task 3596 | 02:05:52 | Preparing deployment: Rendering templates (00:00:24)
Task 3596 | 02:06:21 | Preparing package compilation: Finding packages to compile (00:00:01)
Task 3596 | 02:06:24 | Updating instance adapter: adapter/5d37d36c-0e80-410d-9d03-01c81118b4cd (1) (canary) (00:03:56)
Task 3596 | 02:10:20 | Updating instance diego-api: diego-api/ea07b6fe-43d2-45a1-938b-51f00e407b9f (0) (canary) (00:03:19)
Task 3596 | 02:13:39 | Updating instance diego-api: diego-api/7215fb64-8992-4c54-a9fd-d30b95980321 (1) (00:03:17)
Task 3596 | 02:16:56 | Updating instance uaa: uaa/4906ce75-ddbf-4d7c-82cd-79dad44ddec0 (0) (canary) (00:04:10)
Task 3596 | 02:21:06 | Updating instance uaa: uaa/1605c073-986e-4434-a6ea-9e6e0128bb14 (1) (00:04:46)
Task 3596 | 02:25:52 | Updating instance bits: bits/9efbba78-bf85-47e8-bcbe-7ddaaa69e7a9 (0) (canary) (00:03:10)
Task 3596 | 02:29:02 | Updating instance bits: bits/0e294baf-b909-45d1-bdfa-4d772f446a88 (1) (00:03:04)
Task 3596 | 02:32:06 | Updating instance cc-worker: cc-worker/cc4d3f03-7918-4d05-b486-04176d1444e7 (0) (canary)
Task 3596 | 02:32:06 | Updating instance api: api/b1d2f599-f29b-4c4f-9c3f-5c85fc2c89f1 (0) (canary)
Task 3596 | 02:35:41 | Updating instance cc-worker: cc-worker/cc4d3f03-7918-4d05-b486-04176d1444e7 (0) (canary) (00:03:35)
Task 3596 | 02:35:41 | Updating instance cc-worker: cc-worker/01a626a2-0efd-4607-ac78-b148230d95ec (1)
Task 3596 | 02:36:44 | Updating instance api: api/b1d2f599-f29b-4c4f-9c3f-5c85fc2c89f1 (0) (canary) (00:04:38)
Task 3596 | 02:36:44 | Updating instance api: api/e5e8416c-6afd-45a1-80cf-2538966f2130 (2)
Task 3596 | 02:38:56 | Updating instance cc-worker: cc-worker/01a626a2-0efd-4607-ac78-b148230d95ec (1) (00:03:15)
Task 3596 | 02:40:50 | Updating instance api: api/e5e8416c-6afd-45a1-80cf-2538966f2130 (2) (00:04:06)
Task 3596 | 02:40:50 | Updating instance api: api/14f3a6b5-ebcb-47a4-b74c-07261d602ade (3) (00:04:05)
Task 3596 | 02:44:55 | Updating instance api: api/7353dd97-8e3d-4123-8791-ac3653594767 (1) (00:04:28)
Task 3596 | 02:49:23 | Updating instance scheduler: scheduler/ab98aec9-af7f-4c8d-ac2a-0e36a0d4846c (0) (canary) (00:06:02)
                     L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm","arguments": ["3f225874-fc19-456a-8e79-5c003aaaedac","m-uf639zkz5hy5p22rz231",{"availability_zone": "cn-shanghai-e","ephemeral_disk": {"encrypted": true,"size": 20480},"instance_type": "ecs.c5.xlarge"},{"cf": {"type": "manual","ip": "<IP_ADDRESS>","netmask": "<IP_ADDRESS>","cloud_properties": {"security_group_ids": ["sg-uf68jc3nwxdjnixbvrw7","sg-uf66oinxyjtague1d1ox","sg-uf65ns0580xqvcvk21jy","sg-uf66oinxyjtague1d1ot","sg-uf6fuqggczshqh7v16ng","sg-uf6825diaozs5g81gqzm","sg-uf67m2mj8kzatnko0brf"],"vswitch_id": "vsw-uf6didcjh497z4wa8bpkc"},"default": ["dns","gateway"],"dns": ["<IP_ADDRESS>","<IP_ADDRESS>"],"gateway": "<IP_ADDRESS>"}},[],{"bosh": {"blobstores": [{"options": {"endpoint": "https://<IP_ADDRESS>:25250",","group": "bosh-cf-cn40-staging-cf-scheduler","groups": ["bosh-cf-cn40-staging","cf","scheduler","bosh-cf-cn40-staging-cf","cf-scheduler","bosh-cf-cn40-staging-cf-scheduler"]}}],"context": {"director_uuid": "e9a039b7-0a89-4d52-a3fd-71a8dc125170","request_id": "cpi-198350","vm": {"stemcell": {"api_version": 2}}},"api_version": 1}` create instance failed with input={"Scheme":"HTTPS","Method":"POST","Domain":"ecs-cn-hangzhou.aliyuncs.com","Port":"","RegionId":"","ReadTimeout":0,"ConnectTimeout":0,"AcceptFormat":"JSON","QueryParams":{"AccessKeyId":"LTAIPdGJTly9Iyo3","Action":"CreateInstance","ClientToken":"Bosh-Cpi-CreateInstance-1567478989-6177f87c-0084-4128-969f-85523","DataDisk.1.Category":"cloud_efficiency","DataDisk.1.DeleteWithInstance":"true","DataDisk.1.Encrypted":"true","DataDisk.1.Size":"20","Format":"JSON","ImageId":"m-uf639zkz5hy5p22rz231","InstanceChargeType":"PostPaid","InstanceType":"ecs.c5.xlarge","IoOptimized":"optimized","PrivateIpAddress":"<IP_ADDRESS>","RegionId":"cn-shanghai","SecurityGroupId":"sg-uf68jc3nwxdjnixbvrw7","SecurityToken":"","=","SignatureMethod":"HMAC-SHA1","SignatureNonce":"","SignatureType":"","SignatureVersion":"1.0","SpotPriceLimit":"0.000000","SystemDisk.Category":"cloud_efficiency","SystemDisk.Size":"40","Timestamp":"2019-09-03T02:55:25Z","UserData":""VSwitchId":"vsw-uf6didcjh497z4wa8bpkc","Version":"2014-05-26","ZoneId":"cn-shanghai-e"},"Headers":{"Accept-Encoding":"identity","Content-Type":"application/x-www-form-urlencoded","x-sdk-client":"golang/1.0.0","x-sdk-core-version":"0.0.1","x-sdk-invoke-type":"normal"},"FormParams":{},"Content":"","ResourceOwnerId":"","HpcClusterId":"","SecurityEnhancementStrategy":"","KeyPairName":"","SpotPriceLimit":"0.000000","DeletionProtection":"","ResourceGroupId":"","HostName":"","":"","StorageSetPartitionNumber":"","Tag":null,"AutoRenewPeriod":"","NodeControllerId":"","Period":"","DryRun":"","OwnerId":"","CapacityReservationPreference":"","VSwitchId":"vsw-uf6didcjh497z4wa8bpkc","PrivateIpAddress":"<IP_ADDRESS>","SpotStrategy":"","PeriodUnit":"","InstanceName":"","AutoRenew":"","InternetChargeType":"","ZoneId":"cn-shanghai-e","InternetMaxBandwidthIn":"","UseAdditionalService":"","Affinity":"","ImageId":"m-","ClientToken":"Bosh-Cpi-CreateInstance-1567478989-6177f87c-0084-4128-969f-85523","VlanId":"","SpotInterruptionBehavior":"","IoOptimized":"optimized","SecurityGroupId":"sg-uf68jc3nwxdjnixbvrw7","InternetMaxBandwidthOut":"","Description":"","SystemDiskCategory":"cloud_efficiency","CapacityReservationId":"","SystemDiskPerformanceLevel":"","UserData":"":"","InstanceType":"ecs.c5.xlarge","Arn":null,"InstanceChargeType":"PostPaid","DeploymentSetId":"","InnerIpAddress":"","ResourceOwnerAccount":"","OwnerAccount":"","Tenancy":"","SystemDiskDiskName":"","RamRoleName":"","DedicatedHostId":"","ClusterId":"","CreditSpecification":"","DataDisk":[{"DiskName":"","SnapshotId":"","Size":"20","Encrypted":"true","PerformanceLevel":"","Description":"","Category":"cloud_efficiency","KMSKeyId":"","Device":"","DeleteWithInstance":"true"}],"StorageSetId":"","SystemDiskSize":"40","SystemDiskDescription":""} : over max retry: SDK.ServerError

ErrorCode: InvalidIPAddress.AlreadyUsed
Recommend:

As discussed in the call right now, @xiaozhu36 will have a deep-diving here to solve it. He may bother you later @OlegGerber .

Do we have any progress, we are using version 27, still has this error.

Fixed by version v32.0.0. Please check it.


chore(checks): Rename repo to trivy-checks

Fixes: https://github.com/aquasecurity/trivy/issues/5609

@nikpivkin could you take a look? The PR itself is minor but please feel to point if if we've missed any other part of the repo (or elsewhere) that needs to be renamed.

I've verified the generation of the checks bundle here https://github.com/aquasecurity/trivy-checks/pkgs/container/trivy-checks

@simar7 Trivy has successfully downloaded the new bundle. There is still some use of the term policies in code, but mostly in tests. Should this be changed too, or is it not critical?

@simar7 Trivy has successfully downloaded the new bundle. There is still some use of the term policies in code, but mostly in tests. Should this be changed too, or is it not critical?

Yeah I thought to just leave it for now.


zurawiki.gptcommit version 0.5.7

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.
Supercedes #101701

Microsoft Reviewers: Open in CodeFlow

This package has a runtime dependency on Microsoft.VCRedist.2015+.x64

/AzurePipelines run


iOS simulator crashes upon start [ERROR: "Thread 1: signal SIGABRT"]

So, after few hours of debugging and trying to figure out what causes an app to crash I came to conclusion that this plugin causes it. Now, I've tried re adding platform and plugin, but it seems to not fix the error. Also I tried to change ads, nothing... App just crashes while the plugin is present and works as soon as I delete it.
Screenshot:

I solved it like this: https://stackoverflow.com/questions/55781439/cordova-admob-free-in-ios-exception-when-app-running


Emoji support

This site gives some info: http://caniemoji.com
But I can't figure out how trustworthy it is or when it has last been updated.

I saw there's already an issue for this: https://github.com/Fyrd/caniuse/issues/1522
(which was created by myself.. oops..)


X must be a 2-D matrix.

it also gives an error at end
Error using flipud (line 19)
X must be a 2-D matrix.
can you please guide me about this error
thanks

I think you must be using an old version of matlab that does not support flipud being applied to 3D arrays (i.e. colour images). You could replace the line in render_face_FV.m that says:
im = flipud(im);
with:
im = im(end:1,:,:);


Why is Pageable just CursorPageable? [VOTE]

Description
Looking at this Pageable
compared to Micronaut
and Spring Data
There is no point in separating Pageable and KeySetPageable, because the "abstract" class/interface does no abstraction at all, it just deals with KeySetPageable and its Cursor anyway.
All this API supports is cursor-based pagination
While Spring Data and Micronaut (without 3rd party extensions) both rely on
offset-based pagination.
@graemerocher, @odrotbohm Was the offset-based pagination so far practiced in Micronaut or Spring Data a mistake and you would like to force its current users to switch all apps to cursor now, or why is there no offset Pageable here yet?
If both can be supported I would have imagined an abstract base Pageable with just the page number, size and (unless #54 gets them separated) sorts, but keeping all the KeySet stuff in KeySetPageable.
Then another (frankly the names would be a bit confusing, hence CursorPageable would sound clearer?) OffsetPageable class or interface (or record ;-)
That's how Collection works, it is inherited by specialized sub-interfaces like List, Queue or Set, but any factory methods like of() are in the subsequent subtypes.
Why was this concept broken here?

offset based pagination is just more commonly used and understood and more easily made into a generic API, we looked at keyset pagination but were unsure how to make it generic so left it up to users to implement

Thanks for bringing up this topic.  Originally, when we thought of KeysetPageable its purpose was to be the way in which the user indicates that keyset cursor-based pagination is wanted for a repository method.  Later on, we came up with a different way of indicating this: the KeysetAwarePage/Slice return type of the repository method.  Given that KeysetPageable is no longer used for that purpose, and the only difference between Pageable and KeysetPageable is that the latter adds on a cursor with a direction, it would be possible to simplify things by not having KeysetPageable at all, and just use a single Pageable interface to cover both offset and cursor pagination.  The default mode of a Pageable (no keyset cursor) would be OFFSET, and when there is a cursor, then the mode would be CURSOR_NEXT or CURSOR_PREVIOUS.  This would make the usage patterns for both forms of pagination be more similar for the end user, and it would also reflect the possibility of using the two in combination.  A user can request the first page with offset pagination and then use keyset pagination from there to get the next or previous page reducing the risk of misses and duplicates when traversing pages.

Once we have two PRs merged, I'll close this one.


beautify code

remove unnecessary else{}.

What kind of change does this PR introduce? (check at least one)

[ ] Bugfix
[ ] Feature
[x] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[ ] It's submitted to the dev branch for v2.x (or to a previous version branch), not the master branch
[ ] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing: https://github.com/vuejs/vue/blob/dev/.github/CONTRIBUTING.md#development-setup
[ ] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

IMO the previous version is much clear to read, understand and maintain.

Thanks for the proposal but the usage of if/else even with return statements is indeed intended

ok, thank you for your patience


ChunkExtractor loads more chunks than necessary

💬 Questions and Help
Hi, I am experiencing an issue with code-splitting and server-side rendering where ChunkExtractor loads a lot of extra chunks upon page load rather than just the chunk the view requires.
(Almost all of these chunks are for other unrelated views):

This issue occurs on both development and production and the behaviour is similar whether we use ChunkExtractorManager or collectChunks. This is how we do it at the moment:
import { ChunkExtractor } from '@loadable/server'

...

const statsFile = './home/deploy/razzle/build/loadable-stats.json'
const extractor = new ChunkExtractor({ statsFile, entrypoints: ['client'] })

...

const server = express()
server.
    ...
    .get('/*', async (req, res) => {
        ...
        const app = (
            <CookiesProvider cookies={req.universalCookies}>
                <ApolloProvider client={client}>
                     <StaticRouter context={context} location={req.url}>
                         <App />
                     </StaticRouter>
                </ApolloProvider>
             </CookiesProvider>
         )

        getDataFromTree(app)
            .then(() => {

                ...
                html = renderToString(extractor.collectChunks(app))
                ...

Hello @kronosfere, did you use "webpackPrefetch" or something like that? Can you tell me if you see scripts in the HTML generated from the server?

Thanks for the quick reply @neoziro ! No, we are not using any prefetching. There are script tags generated from the server and they correspond to each of the chunks loaded,
<script async data-chunk="<chunkname>" src="<CDN link>"></script>

I got it, you have to create a new ChunkExtractor for every request. Then it will collect only chunks for current rendering.

Sorry for taking so long to reply, finally managed to get back to that part of the project, and yes that was the issue. Thanks!

Good to know!


Gzip response is not being decompressed automatically

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[x] I commit to help with one of those options 👆

Example Code
from fastapi.middleware.gzip import GZipMiddleware
from fastapi import FastAPI
app = FastAPI(
    title="xyz",
    description="",
    version="1.0.0",
)

app.add_middleware(GZipMiddleware)

@app.get("/status", status_code=200, tags=["Health Check"])
async def health_check(request: Request):
    logger.info(f"health check was requested from {request.client.host}")
    return

Description
Hi,
I am trying to compress the data given from our API, as it sometimes include very large JSON data sets.
i was looking into the built-in GZIP middleware .
I am not sure if what i encounter is an issue with FastAPI or something that i'm doing wrong, because when using some other endpoints that have GZIP response (like https://postman-echo.com/gzip) the response is being decompressed automatically both from postman and the browser itself.
Also, when trying to look for it online I could not find anything, which makes me believe something i do is wrong.
when trying to access the health check endpoint, i get the following response (should be stringified null):

Same result is being received in the browser itself.
Am i missing something?
Thanks in advance.
Operating System
Linux, macOS
Operating System Details
No response
FastAPI Version
0.63.0
Python Version
3.8.0
Additional Context
No response

This is weird, it should be working.
What exactly server of yours sends in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding Content-Encoding header in RESPONSE? It looks for me that for some reason you overwrite this header?
It should be equal to Content-Encoding: gzip and then with any luck it is properly handled by postman, browser etc.

hey @rafsaf, thank you for the reply.
I did not overwrite this header, and it does being sent as gzip.
Any more idea i can try?
thanks.

Update:
the problem seems to be with the proxy-headers flag of uvicorn.
when i try to start my API like so:
uvicorn.run("main:app", host="<IP_ADDRESS>", port=8999, log_level="info", reload=True, proxy_headers=True, lifespan="on")

it does not work, but doing it like that:
uvicorn.run("main:app", host="<IP_ADDRESS>", port=8999, log_level="info", reload=True, lifespan="on")
does work.
We need to have proxy headers enabled for logging purposes behind a reverse proxy.
Any idea on what we can do?

Interesting stuff, what about other custom headers?
for example add Trick-Or-Treat: Treat
Does specifically proxy headers make a difference here or any non-standard headers at all, that is the only question comes to my mind.

I am also facing this issue, if I enable gzip middleware then the response fails to render in insomnia and chrome as well.
Any idea how to fix?

As i remember this issue, the problem is with headers your server send. Author had proxy headers from uvicorn enabled and that is something that broke postman, chrome preview etc.
Please share exact headers you get from server using curl
For example for "working" example of g zipped response https://postman-echo.com/gzip
$ rafsaf@SZCZERY:~$ curl --head https://postman-echo.com/gzip

Results with
HTTP/2 200
date: Sat, 21 May 2022 05:27:05 GMT
content-type: application/json; charset=utf-8
content-length: 207
content-encoding: gzip
etag: W/"cf-Tazgf9i7aAH61KGJVvSKgCv7nPQ"
vary: Accept-Encoding
set-cookie: sails.sid=s%3AEywKA41RcdiDkaAO9RkI51Cfan2rcIxm.DvvFZFUF6jUGbSf0H%2FD2U4YoknLKx7MjqIG4kAgSeE8; Path=/; HttpOnly

Then you can play around trying to change your response disabling or enabling some headers.

You can enhanced APIRoute by get_route_handler.
class GzipRequest(Request):
    async def body(self) -> bytes:
        if not hasattr(self, "_body"):
            body = await super().body()
            if "gzip" in self.headers.getlist("Content-Encoding"):
                body = gzip.decompress(body)
            self._body = body
        return self._body

class GzipRoute(APIRoute):
    def get_route_handler(self) -> Callable:
        original_route_handler = super().get_route_handler()

        async def custom_route_handler(request: Request) -> Response:
            request = GzipRequest(request.scope, request.receive)
            return await original_route_handler(request)

        return custom_route_handler

And then bind to the FastAPI app's route_class：
app = FastAPI(**self.base_settings)
app.router.route_class = GzipRoute

ref: https://github.com/bali-framework/bali/blob/main/bali/application.py

@rafsaf This is the response headers I am receiving:
I am starting the app like this  uvicorn app.main:app --reload
do i need to pass any other parameter?

You can enhanced APIRoute by get_route_handler.
class GzipRequest(Request):
    async def body(self) -> bytes:
        if not hasattr(self, "_body"):
            body = await super().body()
            if "gzip" in self.headers.getlist("Content-Encoding"):
                body = gzip.decompress(body)
            self._body = body
        return self._body

class GzipRoute(APIRoute):
    def get_route_handler(self) -> Callable:
        original_route_handler = super().get_route_handler()

        async def custom_route_handler(request: Request) -> Response:
            request = GzipRequest(request.scope, request.receive)
            return await original_route_handler(request)

        return custom_route_handler

And then bind to the FastAPI app's route_class：
app = FastAPI(**self.base_settings)
app.router.route_class = GzipRoute

ref: https://github.com/bali-framework/bali/blob/main/bali/application.py

It looks like body is being decompressed before sending the response, shouldn't client handle the decompression?
If my server is sending the decompressed response then why to compress it at first place?
Could you elaborate? I might be wrong here.

Are you sure you're using GZipMiddleware as stated in docs?
from fastapi.middleware.gzip import GZipMiddleware
https://fastapi.tiangolo.com/advanced/middleware/#gzipmiddleware
See its code in https://github.com/encode/starlette/blob/master/starlette/middleware/gzip.py#L66, FastAPI imports it directly from Starlette, in line 66 it set response header content-encoding to "gzip", and you have "br".
And by the way, it uses gzip only for bigger responses, and i don't see any reason to use custom gzip middleware if we have one smart, extensible and out of the box @JoshYuJump.

@rafsaf no, I was using a broti-compression compression package, was experimenting with brotli and gzip
but it is same chrome unable to render with the 3rd party library as well as the provided GZip middleware
Below is the screenshot if using the provided GZip Middleware

I've pasted "working" example of response headers few posts above, the only difference is with vary and content-type, first one has doubled headers but it doesn't matter but in content type header you don't send charset, https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type, maybe its something chrome could not stand? But it should be by default utf-8, I'm not expert in this topic.
Can you spend some time investigating it? Just try modifying headers through fastapi or proxy or whatever and find exactly where (in which step) there is a problem? It could be with uvicorn or middleware itself or fastapi or your code.

Are you sure you're using GZipMiddleware as stated in docs? from fastapi.middleware.gzip import GZipMiddleware https://fastapi.tiangolo.com/advanced/middleware/#gzipmiddleware
See its code in https://github.com/encode/starlette/blob/master/starlette/middleware/gzip.py#L66, FastAPI imports it directly from Starlette, in line 66 it set response header content-encoding to "gzip", and you have "br".
And by the way, it uses gzip only for bigger responses, and i don't see any reason to use custom gzip middleware if we have one smart, extensible and out of the box @JoshYuJump.

@rafsaf I tried Starlette's GZipMiddleware before, but It seems can not decompress the gzip requests. It works well in gzip responses.

@rafsaf ok I will try to investigate, although I did try to modify the response object to add content-type with "application/json, charset=utf-8" using this example https://fastapi.tiangolo.com/advanced/response-headers/
It added another header for content-type but still chrome couldn't decompress :(
My other plan is to use nginx for compression once I deploy, can't figure out why fast api isn't working.
I had worked with django in the past with gzip compression and it was perfect.

Yup, I see.
@JoshYuJump has fair point that if it doesn't work and "something is no yes" with bundled Gzip Middleware or compression at all in FastAPI without clear reason (at least for three of us), then using custom based on build-in can be a nice alternative.
But it would be nice if we know that reason and create issue in this project or related one or document it if it is kind of expected and it actually works.
Have a nice day then! :P

@JoshYuJump your solution is for Requests whose body is gzip compressed, right?
But for me chrome is unable to handle gzip Response sent from uvicorn

Ah finally found the issue, I was using app.mount to mount the same app at the '/api' endpoint.
Like this app.mount('/api', app)
removing this mount point fixed the issue, thanks!


ExtHostDocumentSaveParticipant integration test Failure

https://dev.azure.com/vscode/VSCode/_build/results?buildId=48602&view=results
On Mac
  1) ExtHostDocumentSaveParticipant event delivery, waitUntil will timeout:

      AssertionError [ERR_ASSERTION]: true == false
      + expected - actual

      -true
      +false

      at file:///Users/runner/work/1/s/out/vs/workbench/test/browser/api/extHostDocumentSaveParticipant.test.js:178:24

Haven't see this again recently


Authenticated Client Not Retrieving Messages

Hello,
When I attempt to connect an authenticated client, none of the authenticated methods seem to be working. All of them are raising the below raw_decode error, which I believe is saying no messages are returning from the initial request. However, all public client methods are working. I've looked along the thread and seen that this was a similar problem in Issue #86. I've attempted with multiple API keys, on both the sandbox and normal coinbase pro.  Could you help direct me to a solution?
Code:
#Create Authenticated Client for sandbox
auth_client = cbpro.AuthenticatedClient(api_key, api_secret, pass_phrase, api_url)
try:
auth_client.get_time()
except:
auth_client.get_time()

Error:
line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None

Have you given the correct permissions to your api key?

Yes, I have both view and trade permissions, no transfer permission though.

Does it create the exact same error without a URL parameter?

Thanks, that seemed to fix the issue for connecting to regular coinbase pro, however, do you know how I could connect to the sandbox?

Did you create a unique api key for the sandbox first? Just checking the basics. But I've also heard the sandbox support can be unreliable and I have not personally used it in a while.

I did create a unique api key for the sandbox, but what you say makes sense.


BiMultiHeadAttention

mmdetection implement BiMultiHeadAttention, used in grounding dino.
Why does this reimplement the whole multiheadatt ?
Is this different that doing
A
B
cross attention AB
cross attention BA
as illustrated in the figure from the grounding dino paper?

@Chop1 We rely on the code as the reference. You can go and verify the code for confirmation. Thank you.


Added test for .save() on populated models

Add tests for balderdashy/waterline#937 and a couple more:

Update nested association using .save();
Update parent association using .update();
Update parent association using .save();

@dmarcelino you the man!

Merging with @particlebanana's blessing, https://github.com/balderdashy/waterline/issues/937#issuecomment-90979473:

#938 / balderdashy/waterline-adapter-tests#46 - looks great


Logos

The following Logos should appear on the site.
Very Visible
Utah Highway Patrol
UDOT
UDPS
Much Smaller
AGRC

Do you have the images?


How do I allow a tenant's service account to create clusterresources?

I am trying to setup a multi-tenant environment, where I can separate responsibilities in teams. However, when I am trying to install some helm charts, I get the following errors:

2024-09-12T16:44:39.048Z error HelmRelease/nfs-subdir-external-provisioner.platform - Reconciler error Unable to continue with install: could not get information about the resource StorageClass "nfs-client" in namespace "": storageclasses.storage.k8s.io "nfs-client" is forbidden: User "system:serviceaccount:platform:platform-team" cannot get resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
2024-09-12T16:44:42.386Z error HelmRelease/cert-manager.platform - Reconciler error Unable to continue with install: could not get information about the resource CustomResourceDefinition "certificaterequests.cert-manager.io" in namespace "": customresourcedefinitions.apiextensions.k8s.io "certificaterequests.cert-manager.io" is forbidden: User "system:serviceaccount:platform:platform-team" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
2024-09-12T16:45:03.108Z error HelmRelease/metrics-server.platform - Reconciler error Unable to continue with install: could not get information about the resource ClusterRole "system:metrics-server-aggregated-reader" in namespace "": clusterroles.rbac.authorization.k8s.io "system:metrics-server-aggregated-reader" is forbidden: User "system:serviceaccount:platform:platform-team" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
2024-09-12T16:45:03.144Z error HelmRelease/purelb.platform - Reconciler error failed to apply CustomResourceDefinitions: failed to create CustomResourceDefinition lbnodeagents.purelb.io: 1 error occurred:
* customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:platform:platform-team" cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope

As you can see, I have issues retrieving storageclasses, customresourcedefinitions and clusterroles and creating customresourcedefinitions.
I have created the following file:
# rbac.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform-team
  namespace: platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: platform-permissions
  namespace: platform
rules:
  - apiGroups: ["apiextensions.k8s.io"]
    resources: ["CustomResourceDefinitions", "ClusterRoles"]
    verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform-team-reconciler
  namespace: platform
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: gotk:platform:reconciler
- kind: ServiceAccount
  name: platform-team
  namespace: platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform-team-crds
  namespace: platform
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: platform-permissions
subjects:
  - kind: ServiceAccount
    name: platform-team
    namespace: platform

But this doesn't seem to work... Can anyone point me in the right direction?

IMO the platform-team should have cluster admin, if it needs to create cluster roles, then a custom RBAC is useless. So I would just do:
---
apiVersion: v1
kind: Namespace
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform-team
  namespace: platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    toolkit.fluxcd.io/tenant: platform-team
  name: platform-team
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: platform-team
    namespace: platform

Make sure that the Flux Kustomizations and HelmReleases in the platform namespace have serviceAccountName: platform-team.

So the Kustomizations can apply but the HelmRelease fails? Can you post here the binding with kubectl get clusterrolebinding platform-team -oyaml --show-managed-fields


Disable preview for text files

fixes #623

@guergana
If user can't open anything except for table then it will be easier just to remove File and Text controllers completely


Unable to load preload script:

Versions + Platform

[ ] SDK version -<EMAIL_ADDRESS>[ ] Electron version - 5.0.8
[ ] Platform - Windows

Description
When I use this version, I find that I can't import. But I can't upgrade the node version yet. I want to ask if there is a solution. Const {init} = require ('@ sentry / electron'); if used in this way, there will be a warning could not connect to sentry main process. Did you call init?

may be you understand chinese;
当你在preload中集成sentry时，有两点需要注意：

不能使用es6 module的语法，原因时electron中的preload脚本只能支持commonjs规范的模块语法，所以你需要写成如下的形式：

const { remote } = require('electron');
const Sentry = require('@sentry/electron');

sentry需要init两次，一次在main进程中，一次在preload脚本中

从你上述描述看，应该是main进程中没有调用init方法

The latest version of @sentry/electron is 2.4.1 so I'm going to close this as stale and probably fixed but feel free open a new issue if the problem persists.


Improved assertions on warnings

Description
Improved assertions on warnings to prevent warnings to leak from one testcase to another
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[ ] Attached issue to pull request
[x] Changelog entry
[ ] Type annotations are present
[x] Code is clear and sufficiently documented
[ ] No (preventable) type errors (check using make mypy or make mypy-diff)
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[x] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )

@Hugo-Inmanta it could, but I don't think that is the case here.

Processing this pull request

Processing #4787.

Processing #4786.


Document the accounts that the Executable Books project uses, and share access with team members

There are a few accounts that are generic to the Executable Books project. In some cases these are controlled by a single person, in some cases we have a generic EBP username, etc. We should share access and control of these accounts, and make them person-agnostic as much as possible so that we don't create bottlenecks and low bus factors.
While we do not want to casually give out access to services that can potentially do destructive actions, we should generally follow a practice of de-bottlenecking and increasing our bus factor for critical actions like publishing. For individuals that wish to have access, and are already trusted members of the team, we should just give it out.
We should document the services that have restricted access, who has access to it, and share access with other team members that want it.
Our goal is to have at least two team members with admin access to every account here.
Organization and administration

[ ]<EMAIL_ADDRESS>(@chrisjsewell, see: https://github.com/executablebooks/meta/discussions/412)
[ ] The executablebooks.org and myst.tools domain names (@choldgraf)
[x] The Google Drive folder (@mmcky owner, all team members have edit access)

[ ] @mmcky is the owner of the Google Drive folder, but this should be a generic folder instead.

[ ] Admin access for the EBP Slack (@mmcky (owner), @choldgraf)
[ ] Admin access for the EBP calendar (https://calendar.google.com/calendar/embed?src=s425j1984fmcs7u4cdmdmpau9o%40group.calendar.google.com&ctz=America%2FLos_Angeles)

Google Groups

executablebooks-conduct: Used for reporting Code of Conduct violations and requests. Anybody in the internet can send messages to this group, but it is only visible to group members. Currently, the group members are the Project PIs (myself, @gregcaporaso and @jstac )
[executablebooks-team](https://groups.go ogle.com/g/executablebooks-team): Used for private communication between team members. This should only be used to discuss topics that must be private. Anybody can post to this group, but only group members can see posts. Currently this is everybody listed on the team page here as well as @gregcaporaso .

Development

[x] GitHub repository access I think is fine, because we are all added on the executablebooks org

Bot accounts

[ ] https://github.com/executablebooks/team-compass/issues/10
[ ] The ebp-bot GitHub account (https://github.com/ebp-bot)

Access across many repositories

[ ] ReadTheDocs access across our various repositories (several people, mostly @chrisjsewell and @choldgraf )
[ ] Our PyPI accounts (mostly a combination of @chrisjsewell and @choldgraf )
[ ] Our npm accounts (@chrisjsewell and @rowanc1)

Social media

[ ] @executablebooks -> @choldgraf has manager access
[ ] @myst_tools -> @mikemorrison has manager access

Web domains

[ ] executablebooks.org -> NameCheap, @choldgraf has ownership + @rowanc1 has manager access
[ ] jupyterbook.org -> NameCheap, @choldgraf has ownership
[ ] myst-tools.org -> google domains, @rowanc1 and @choldgraf has access.

Could we use ebp-bot for centralizing access?
It occurs to me that one approach we could take here is to use ebp-bot for as much as possible as the "minimal accounts with access" requirement for all of our accounts. Then as long as the steering council always has admin access to the ebp-bot, we do not need to do a huge update every time there is churn. For example, we could use ebp-bot as the primary pusher to PyPI and the main account on all ReadTheDocs sites.
Just writing this thought down so I don't lose it. Curious what others think.

Added domain access to top comment
I added the accounts for our various web domains to the top comment, since that's another shared resource that we use. I've noted that @rowanc1 has manager access to the namecheap account as well, so that he can set up the e-mail for our new google workspace account.


util.h: Add log2ceil function

Using __builtin_clz() and friends.
This is a function that I have found useful here and there, but has been missing from Zephyr AFAICT, so this is my proposal. Parameter and return types can be discussed. I can add tests if desired, just tell me where.

@cfriedt Ok, so you're proposing adding a LOG2CEIL in the style of that LOG2? Probably makes sense to add it in #53405, or what do you think?

@cfriedt Ok, so you're proposing adding a LOG2CEIL in the style of that LOG2? Probably makes sense to add it in #53405, or what do you think?
Funny timing indeed :)

I could add it to #53405 if you like, but I would much rather you update your PR. We're a couple of days away from feature freeze from what I understand, and it's unlikely that anything other than v3.3 features and bugfixes will get merged soon, so we could revisit this after the release. Does that work for you?

@cfriedt Ok, so you're proposing adding a LOG2CEIL in the style of that LOG2? Probably makes sense to add it in #53405, or what do you think?
Funny timing indeed :)

Sure - I can add it to #53405 - I'll credit you with it in the commit message and add some testing too.

Will be added in #53405


Speed limit for access point

Is there a way to limit the network speed of the access point ?

You can use tc for this. You can find a tutorial here: https://wiki.archlinux.org/index.php/Advanced_traffic_control
I don't have plans to implement it yet. I will think about it.

Thanks. I'll try that link.

Please consider doing it It's a really rich feature for your project !!!


Ability to set the discord token in an .env file

I use klipper backup and would love to be able to not potentially leak my discord tokens.

hey there, you can use the include feature in the config
[include test.cfg]


⚡ Optimize upper bit cleaning with mask

Description
Implements the use of an address mask to save a couple runtime gas where deployment gas is also reduced.
This can also be implemented in ERC20.sol and ERC721.sol, but I chose not to include since this method increases deployment size on those contracts. See below gas diffs of ERC20 & ERC721 for decision on inclusion, will add if desired:

ERC20: https://www.diffchecker.com/cpuEi1Xs/
ERC721: https://www.diffchecker.com/cpuEi1Xs/

Checklist
Ensure you completed all of the steps below before submitting your pull request:

[x] Ran forge fmt?
[x] Ran forge snapshot?
[x] Ran forge test?

Solc optimizer not doing this? Intresting.

@kadenzipfel Thanks for the PR. solc optimizer already doing this. checked forge snapshot --diff with main branch there i can't see any gas different adding this changes. So for better readability we keep old version.
@Vectorized tagging you for double check.

@kadenzipfel Thanks for the PR. solc optimizer already doing this. checked forge snapshot --diff with main branch there i can't see any gas different adding this changes. So for better readability we keep old version.
@Vectorized tagging you for double check.

Hmm, I'm getting different results on forge test, should be using the optimizer still, right? Gas savings are pretty minimal and only on applied methods, but here's an example diff with Ownable: https://www.diffchecker.com/R81lABu8/

Hmm... i prefer the aesthetics of shr(96, shl(96, a)).

Hmm... i prefer the aesthetics of shr(96, shl(96, a)).
Don't need one more constant kekw, which adds to vertical scroll space.

We can always inline it, e.g.:
newOwner := and(newOwner, 0x00ffffffffffffffffffffffffffffffffffffffff)

but i feel u


Add initial version of node-buffer-blob

First version of node-buffer-blob

The node constructor uses a union type, accepting a list of blobs, a list of arraybuffers, a list of strings, ....
I have created constructor functions fromArrays, fromBlobs for the moment. Not sure if there is a better way to do it without depending on something like untagged-union..

Apart from this, I think this initial pr is ready for review.

🏓 @JordanMartinez

Also, I wonder if the singular versions of Blob.from (e.g. fromString) should be dropped. It's not too hard to go from fromString "foo" to fromStrings <<< Array.singleton or just fromStrings ["foo"]. Then there's less FFI to write.
But if calling blob.from([]) is problematic, then perhaps it should be NonEmptyArray a where a is one of those 5 types that Blob.from accepts.

Also, I wonder if the singular versions of Blob.from (e.g. fromString) should be dropped. It's not too hard to go from fromString "foo" to fromStrings <<< Array.singleton or just fromStrings ["foo"]. Then there's less FFI to write.
But if calling blob.from([]) is problematic, then perhaps it should be NonEmptyArray a where a is one of those 5 types that Blob.from accepts.

Yeah that makes sense. I changed everything to NonEmptyArrays.

Could you clarify why you didn't implement Blob.from for the following types?
NonEmptyArray TypedArray
NonEmptyArray DataView
NonEmptyArray Blob

Could you clarify why you didn't implement Blob.from for the following types?

Oh I just missed these, will add them

Ok I added fromBlobs and fromDataView.  I haven't yet figured out how the TypedArray works and at this point I would prefer to have that addressed in a subsequent PR if someone requires it.

Ok I added fromBlobs and fromDataView. I haven't yet figured out how the TypedArray works and at this point I would prefer to have that addressed in a subsequent PR if someone requires it.

I'm good with that.


fix(coc.nvim): add highlight for MenuSel

A recent commit in coc.nvim changed the highlight of the selected item in the completion popup menu, which looks awful with this theme.
This PR changes the definition back to how it was before: a link to CursorLine.

Thank you!


dev/model/parts: Changes to Code should reflect in diagram

Changing the pin definitions should update the diagram.

This is fixed - edits to Properties and Pins reflect on commit, and Delete works again.


npm test does not run on Windows

The single quotes in the package.json definition of the test script breaks it on Windows.

Fixed in #12.


additional tenant attributes

I would like to have the following additional properties on a Tenant please:

description
logo/image
date fields: created, activated, deactivated

About the date fields - I propose to have two date fields instead: dateCreated, dateActiveChanged. The second one will hold the date when the active status changed for the last time. Is this OK?

Sounds good to me.


Disable setappimage if winebincode != AppImage

This is a partial fix for #5. It adds conditional logic to the CLI, TUI, and GUI that either removes or disables a function if the config file states that the Logos install was created with something other than an AppImage.
It also creates a new GUI tooltip function. This is applied to the set appimage button, so that when the user hovers over it with his mouse, it will tell him why the button is disabled.

The general logic looks good.
I just started testing with the Tk dialog and ran into this:
Exception in Tkinter callback
Traceback (most recent call last):
  File "/opt/lib/python3.12/tkinter/__init__.py", line 1962, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "/home/nate/g/LogosLinuxInstaller/gui_app.py", line 228, in on_wine_check_released
    self.gui.wine_dropdown['values'] = utils.get_wine_options(utils.find_appimage_files(), utils.find_wine_binary_files())
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nate/g/LogosLinuxInstaller/utils.py", line 757, in get_wine_options
    return sorted(wine_binary_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'PosixPath' and 'str'

It looks like the list wine_binary_options is a mix between strings and pathlib.Paths. It looks like utils.find_appimage_files() returns pathlib.Paths, while utils.find_wine_binary_files() returns strings. Both of those get passed into utils.get_wine_options(), so we either need to change what type one of them returns, or we need to covert it in utils.get_wine_options() to be the same as the other one.

It looks like the list wine_binary_options is a mix between strings and pathlib.Paths. It looks like utils.find_appimage_files() returns pathlib.Paths, while utils.find_wine_binary_files() returns strings. Both of those get passed into utils.get_wine_options(), so we either need to change what type one of them returns, or we need to covert it in utils.get_wine_options() to be the same as the other one.

Okay. That should be an easy fix; I'll mod find_appimage_files. If there is a need to have both returned, I'll add a flag for the function(s).

I was running my system's default Python version, 3.11, and I hit an error in code. I was about to code around this until I checked docs and saw it was a 3.12 feature added. While I don't plan to do a full-blown version check for every feature, I added this for clarity's sake since the CLI reported an unexpected keyword error. Thus I added this commit: 229bcb8.

Exception in thread Thread-2 (get_logos_releases):
Traceback (most recent call last):
  File "/opt/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/opt/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/nate/g/LogosLinuxInstaller/utils.py", line 702, in get_logos_releases
    filtered_releases = filter_versions(releases, 30, 1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nate/g/LogosLinuxInstaller/utils.py", line 671, in filter_versions
    return [version for version in versions if check_logos_release_version(version, threshold, check_version_part)]
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nate/g/LogosLinuxInstaller/utils.py", line 667, in check_logos_release_version
    return version_parts[check_version_part - 1] < threshold
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'map' object is not subscriptable

So it apparently doesn't work to pluck an item by index from the map function object without first converting that into a list, in which case, I'd say just make it a list from the beginning. So L666 could be something like:
version_parts = [int(p) for p in version.split('.')]

or:
version_parts = list(map(int, version.split('.')))

I tested the speed of both of these, just out of curiosity. Here's what I found:
>>> import timeit
>>> timeit.timeit('list(map(int, "30.2.001".split(".")))', number=10000)
0.014728516049217433
>>> timeit.timeit('[int(p) for p in "30.2.001".split(".")]', number=10000)
0.011726382013875991

So the list comprehension option seems to be faster, in case you ever want to do this on a larger number of items.

Ah, I had simplified that code from if/elif series/case statement to that. Sounds good. Will test a change and push.


SSO Reset Secondary Password (Old 58)

In gitlab by @A-Lawrence on Aug 31, 2015, 01:09
Add the ability for members to reset their own secondary password

In gitlab by @A-Lawrence on Aug 31, 2015, 01:15
Added the ability for users to request a password reset on SSO. Resolves issue #488552 (58).
ÔåÆ <>
ALawrence66 - 2013-11-10T02:37:56.635768+00:00

In gitlab by @A-Lawrence on Aug 31, 2015, 01:15
Removing milestone: DEVELOPMENT - NO SET RELEASE (automated comment)
ALawrence66 - 2013-11-02T20:56:14.122116+00:00

In gitlab by @A-Lawrence on Aug 31, 2015, 01:15
Linked to issue #488551 (59) - waiting on mailing system before this can be properly done.
ALawrence66 - 2013-11-02T17:39:12.358694+00:00


Create new automerge process

In addition to the existing AI-Pure automerge process and the DOI-based automerge process, we want to create another process that matches when the publications are very similar but one or both don't have DOIs. Like the DOI-based automerger, it should review (and, if appropriate, merge) each pair of publications in a duplicate group.
The starting point for the criteria should be what's currently in the DOI-merge process (prior to the changes in #529 ), but with the following updates:

Title and subtitle: Add requirement that the combined title and subtitle of the two publications fuzzy match at a very high percent -- perhaps 90 or 95%.
DOI: If the DOIs conflict, do not merge. OK to merge if neither pub has a DOI or if only one pub does.

Background discussion is in #523

@anaelizabethenriquez We also need to determine how we want to merge these.  We could use the merging policy that we use for the DOI automerger, so we'd rename PublicationMergeOnDoiPolicy to PublicationMergePolicy and use that.  I think it does a decent job of picking the best data.  Or have some other way to determine the better record.

@ajkiessl Using the policy that's currently used for the DOI automerger sounds good to me.


ZeroMQ PubSink & SubSource

Why ZeroMQ?
ZeroMQ is by default in GNU Radio Companion.
ZMQSubSource & ZMQPubSink as blocks
I would like to have blocks to publish stream & receive stream from other systems.
Underlying ZMQ rust lib?
I have done an experiment with zmq library which is blocking one.
Should we consider using async-zmq (non blocking) or the more idiomatic libzmq (but unmaintained).
Additional context
Having those would enable tests where only part of a chain is migrated to FutureSDR. For instance, it could enable to split a chain between GNU Radio and FutureSDR until FutureSDR  had all the mandatory blocks of the overall chain.

I'll have a close look at the implementation, as soon as I have time. But just because I forgot to mention it in the intro video: One can specify through the BlockMeta that a block uses blocking calls in work, which will spawn it in a separate thread, using its own executor. See
https://docs.rs/futuresdr/0.0.4/futuresdr/runtime/struct.BlockMetaBuilder.html#method.blocking
https://github.com/FutureSDR/FutureSDR/blob/master/src/blocks/soapy_src.rs#L27

@bastibl Thanks for the tip. I will add this.
Once I have properly add ZeroMQ as a feature (by default?), I will create a pull request.


Yet Another Random Memory Comment :3

If you're feeling extra safe, you can check if the memory allocation on line 126 was successful. Aaaand you can avoid checking in the reallocs that follow by simply allocating to the greatest possible size which is sizeof(char)*7 :3

Will do! Thx!


E2E chat history in db for the selected book

This pull request includes several changes to enhance the chat service functionality and improve session management. The most important changes involve adding a new method to update chat history in the database, improving session handling, and updating the frontend to support saving chat history.
Enhancements to chat service functionality:

Models/ChatService.cs: Added a new method UpdateBooksChat to update the chat history in the database. Modified the AIResponse method to update the chat history and session with user and assistant messages. [1] [2] [3]
Program.cs: Changed the registration of ChatService from singleton to scoped to ensure a new instance is created per request. Added a new endpoint /api/updateChat to handle chat updates. [1] [2]

Improvements to session handling:

Middleware/JsonToHtmlMiddleware.cs: Removed the ChatHistory from the session when serializing book details back to JSON.

Frontend updates:

wwwroot/index.html: Added a new button to save chat history and implemented a JavaScript function to call the new /api/updateChat endpoint. [1] [2]

Minor changes:

Models/ChatService.cs: Added a new dependency LibraryDbContext to the ChatService constructor. [1] [2]

This pull request introduces several changes to enhance the chat functionality and session management in the application. The most important changes include adding a new method to update book chat details, modifying session handling, and updating the front-end to support saving chat history.
Backend improvements:

Models/ChatService.cs: Added a new method UpdateBooksChat to update book chat details in the database.
Models/ChatService.cs: Modified the AIResponse method to include user and assistant messages in the chat history and update the session. [1] [2]
Middleware/JsonToHtmlMiddleware.cs: Removed the ChatHistory from the session after serializing the book details.
Program.cs: Changed the registration of ChatService from singleton to scoped and added a new endpoint /api/updateChat for updating the chat. [1] [2]

Front-end updates:

wwwroot/index.html: Added a new button to save the chat and implemented the saveBooksChat function to call the new endpoint. [1] [2]


Updates to genbank_accession lines (1540 new, seqs downloaded 2020-12-04)

Another bi-weekly update of genbank accessions matched by 1. sequences being identical (possibly except for runs of N's at beginning and/or end) and 2. similar country/isolate/year names.
This time, in addition to adding new mappings, I've also done a little cleanup of some cases where the same sequence was apparently submitted multiple times to GISAID and/or GenBank:

In some cases, the same GenBank accession was assigned to more than one EPI ID with identical sequence and similar name (apparent duplicate submissions to GISAID).  I removed 9 duplicate mappings to make the mapping 1-to-1.

In other cases, there are multiple GenBank accessions that could be assigned to one EPI ID (apparent duplicate submissions to GenBank).  Different versions of my pipeline have arbitrarily chosen different GenBank accessions over time.  I changed 219 GenBank accessions to match the latest output of my pipeline.

There's also one case of duplicate submission to both GenBank and GISAID, and I straightened that out to make a 1-to-1 mapping.

2 apparent duplicate submissions to GISAID had their newer EPI IDs retracted (the epicov.org search for those accessions says "EPI_ISL_... is currently not available at the Submitter's request"), I removed those as well.

Thanks!   :)

I also just found that 4 sequences have been removed from GenBank by the submitter's request, so I removed the corresponding 4 genbank_accession lines.

Thank you once again Angie for this great work and for the detailed explanations! It looks great and I'll merge now! Thanks for keeping up with this, I think it's so incredibly useful!

Thank you Emma, all the tools that you and the rest of the Nextstrain team are providing are absolutely vital to SARS-CoV-2 genomic research, so I'm thrilled I can help in this small way.  :)


25/7/2023: Adding the ancestral allele rules to the pipeline - what is the input?

The rules for inferring the ancestral allele should run if the file is not already present.
However, if the file is present, it's path is specified in the config['ancestralAllele'], which would be somewhere in the Data directory, BUT if the file is created by the pipeline, it's placed in the Projects/AncestralAllele, which is a directory created by the pipeline (both Projects/ and AncestralAllele/ are created by the pipeline). We might not want to create this directory beforehand, but how do we the specify the input if it could be coming from two different places on the disk?
Do we want to put the final file with ancestral alleles back onto the disk, but that is not in concordance with where we put the rest of the results.

Can't we say that in the config file, if ancestralAllele = None it runs the est-sfs pipeline else it mv/cp from the given path to Projects/AncestralAllele? Then we always have it starting from the same location.

I don’t think would be too bad a problem, it is still a two col file

Ok - if ancestral inference is required, the config file should say null for ancestralAllele.


Fix #308: Removed Events Page

Removed Events Page
Updated Sidebar

@Ash-2k3 PTAL! Thanks!

Assigning @seanlip for final review, Thanks!


Is Sanic running with all workers?

By default runs with 1, we can set as many cores we have
app.run(host="<IP_ADDRESS>", port=port, workers=os.cpu_count())

Any tunings are welcome!! I'll try this, thanks!

I made the fix https://github.com/tbrand/which_is_the_fastest/pull/45

Thanks!!!


Embedded tag in reply not working

RSTS replies.yml#embedded_tags failed:
java.lang.AssertionError: Did not get expected reply for input: My name is Bob.
Expected: is "I thought your name was Alice?"
     but: was ">I thought your name was <get name?"

Fixed with 54ccd54


Add methods for computing cycle times

Compute the start of a cycle and its duration.
To do:

[ ] Add documentation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

15 of 15 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 83.459%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
772

Relevant Lines:
925

💛  - Coveralls

I talked with @victorventuri offline, and we agreed that picking the duration to be between the starts times of cycles is OK. That said, we both hope the difference between this definition and measuring the span between first/last points of a cycle is minimal.
I also added a warning if there is only one point per cycle.


Frontend Project "QrComponent" Improved Styling

Title and Issue number
Frontend Project "QrComponent" Improved Styling
Close #3703
Output
Before

After

Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md

Hi @MohitGupta121
Kindly review this pull request.


feat: add wasm bindings

Description

This PR is based on #132 and adds wasm bindings.
Tasks

[ ]

References

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Appropriate labels applied
[ ] Targeted PR against correct branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the module structure standards.
[ ] Wrote unit and integration
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

I will rework this on the other PR.


python3Packages.bpemb: init at 0.3.2

Description of changes
Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE)
https://github.com/bheinzerling/bpemb

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[x] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/1016/107


detects wired but not Bluetooth on Android

Why? What can be the reason?

Update: Bluetooth works but very much delayed.

Update: Bluetooth works but very much delayed.

Well, I think that it depends on device not this package.


RtImagesITCase.java:69-72: To have multiple controlled...

The puzzle 187-99b12eee from #187 has to be resolved:
https://github.com/amihaiemil/docker-java-api/blob/adec112b22042776d6dfd36ade12db8694fc0b17/src/test/java/com/amihaiemil/docker/RtImagesITCase.java#L69-L72
The puzzle was created by @amihaiemil on 17-Dec-19.
Estimate: 30 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@amihaiemil please pay attention to this new issue.

@amihaiemil/z please, pay attention to this issue


Fix API to use author for commits instead of committer

There is a bug in modules/convert.ToCommit where the committer is used in place of the author.
This PR fixes this.

Codecov Report

Merging #16276 (f017071) into main (9a0cd3a) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main   #16276      +/-   ##
==========================================
+ Coverage   45.40%   45.42%   +0.01%
==========================================
  Files         709      709
  Lines       83484    83484
==========================================
+ Hits        37910    37919       +9
+ Misses      39495    39481      -14
- Partials     6079     6084       +5

Impacted Files
Coverage Δ

modules/convert/git_commit.go
87.70% <100.00%> (ø)

modules/indexer/stats/db.go
42.85% <0.00%> (-7.15%)
:arrow_down:

models/repo_list.go
77.04% <0.00%> (-0.78%)
:arrow_down:

modules/notification/mail/mail.go
38.23% <0.00%> (+1.96%)
:arrow_up:

modules/queue/workerpool.go
56.48% <0.00%> (+2.67%)
:arrow_up:

models/unit.go
43.83% <0.00%> (+2.73%)
:arrow_up:

services/mailer/mail_comment.go
85.18% <0.00%> (+7.40%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9a0cd3a...f017071. Read the comment docs.

Please send backport.


New package: Thales.SafeNetAuthenticationClient version 10.8.2154.0

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Verify:

This is a tray-based application.

Hi @ye4241,
Could you add a PackageUrl?

@stephengillie Done.

@stephengillie Done.

Thanks.
I'm having trouble finding the InstallerUrl - is it linked from another page?

@stephengillie
https://knowledge.digicert.com/general-information/how-to-download-safenet-authentication-client

Verify:

please note that this is an older version and is affected by multiple CVEs:
CVE-2023-7016
A flaw in Thales SafeNet Authentication Client prior to 10.8 R10 on Windows allows an attacker to execute code at a SYSTEM level via local access.
CVE-2023-5993
A flaw in the Windows Installer in Thales SafeNet Authentication Client prior to 10.8 R10 on Windows allows an attacker to escalate their privilege level via local access.
There is a newer version available for download at DigiCert's page, version 10.9  (10.9.3283.0) ... so i opened an issue for the new version:   https://github.com/microsoft/winget-pkgs/issues/173741


(SERVER-885) Style fixups for #849

This PR includes 2 commits to address feedback from PR #849 .

:+1:


Events list: inconsistent ordering of event cards

Problem to be solved

The event cards are displayed in random order. For example, 'Body image' on 8 Nov event is being displayed after 'Metals' on 21 Nov on 'What's on' and 'All events'
Event cards are being displayed in the wrong months. For example, there are 3 November events in 'December'.

Homepage

What's on

All events

Expected behaviour
What's on: display events in chronological order starting with most recent upcoming event
All events: display events in chronological order starting with most recent upcoming event

TODO
/past/events
Only display here when all events are past
/whats-on and /events
Display by next upcoming event
Multi-month events
Leave as is: display in first month, not in second month; then display in second month when first month has passed

@jennpb Just to clarify, did we decide to remove the past events on /events and add a link to /events/past at the bottom of the list?
Also, we didn't get to discuss the ordering of the mixed list (This week on Homepage and Today/This weekend on What's on). Should we leave it as is or follow the same grouping/ordering we discussed for Exhibitions and Events?

https://wellcomecollection.org/events/past
The ordering in this past view has changed and I can't tell what order they are in anymore. Also, the event cards in the 1st and 2nd pages are identical.

To follow up on the conversation from @gestchild #3531 Events ordering, here are the usability reasons for removing Past events in /events

Users often miss 'past' status and think all events in the list are available to attend
It's likely people will miss the 'Past events' grouping in a long list
Past events are only relevant when trying to find about specific event program i.e Packed lunch

Having said that, it wouldn't hurt to add 'Past events' link at the bottom of /events to allow for people looking for past events they've attended. (note: 265 clicks for Past events from What's on past 3 months.)

Exhibition and Events ordering on /exhibitions, /events and /whats-on
There is a disconnect between how we are able to query for a list of events or exhibitions from Prismic and how we want to display those lists to our users.
The following attempts to capture what is currently happening and to highlight some of the difficulties and potential problems.
/exhibitions ordering
Request from Prismic

Each exhibition has a single date range with a start date and an end date.
The request from Prismic retrieves all exhibitions in descending order based on their end date.
Prismic doesn't deliver the results in one single response, but divides them across several pages.
We retrieve 20 results per page (Prismic allows a maximum of 100 results per page).

How we display

For a page of results, we use the date range of each exhibition or it's 'isPermanent' property to place it in a time group (Permanent, Current, Upcoming and Past)
We then display those groups in the following order: Current > Permanent > Upcoming > Past
Within each group they still display in descending order as per the original request from Prismic.

Potential problems

This works fine until any of the first 3 groups span more than one result page, because we can only do the grouping and re ordering for each page of results we receive. If, for example, the first page from Prismic contained exhibitions that belonged to the Permanent, Upcoming and Current groups and the second page contained exhibitions that belonged to the Current and Past groups, our reordering would mean the user would see Current, then Permanent, then Upcoming on page 1 (as desired), but would then see the next set of Current, followed by Past on page 2.

In reality, due to the limited number of Current and Upcoming exhibitions we have, this won't be a problem.

Having Past displayed in descending order makes sense, not sure it does for the Current and Upcoming groups?

/events ordering

For Current and Upcoming events, we want to display them in ascending order, i.e. starting closest to today and getting further into the future.
For Past events we want to display them in descending order, i.e. starting closest to today and getting further into the past.
We also want to display the events in the following groupings and order Current > Upcoming > Past

If we were to request the events from Prismic in descending order and then regroup them for display (as we do for exhibitions), we are far more likely to hit the pagination problem described above, as there are far more current and upcoming events.

We therefore only request current and upcoming events from Prismic in ascending order and then can display them exactly as they arrive, without the need to Group and reorder.
There were also UX reasons for wanting to remove the past events

An alternative solution would be to trust we never have more current and upcoming events, than the number that are shown on an individual results page (currently 100 - see below). We'd also then need to group the returned results and reorder the current and upcoming ones so that they displayed in ascending order.
Almost there

Events can have multiple date ranges; rdering of events lists, returned from Prismic, is done based on the end date of the last date range that they have
Ee only display the next upcoming date of the event, so they can appear to be out of order
We therfore reorder the page based on the next display date of an event

Potential problems

The fact we reorder each results page based on the next display date of event, could lead to events appearing out of order
This would happen if the last date in an events date range, puts it on a later page, than it's next upcoming date
The likelyhood of this occuring has been mitigated by returning the maximum 100 results per page.

/events/past

An event can have multiple start and end dates
When we request past events from Prismic, it uses the first end date it comes across to determine if the event is past
As a result, events that have one or more past dates but still have upcoming dates are included in past dates
We therefore have to filter these from the list before displaying

/whats-on events ordering

Event lists on the /whats-on page work in the same way as /events, but undergo an additional steps
We group events into months
Where events have multiple dates, they are duplicated for each month in which they occur (as long as that date has not already occured).

:clap: Thank you for documenting the complexity @gestchild


test: send friend request

write a test to verify a user's ability to send friend request

closing see #20