BlobClient returns ClientAuthenticationError when Azure CDN is enabled

Package Name: azure-storage-blob
Package Version: 12.10.0
Operating System: Ubuntu 20.04.4 LTS
Python Version: 3.9.5

Describe the bug
Calling the BlobClient.exists method raises a ClientAuthenticationError when an Azure CDN account URL is used.
To Reproduce
Steps to reproduce the behavior:
from azure.storage.blob import BlobServiceClient

ACCOUNT_NAME = "myaccount"
ACCOUNT_KEY = "myaccount-key"
CREDENTIAL = {"account_name": ACCOUNT_NAME, "account_key": ACCOUNT_KEY}

ACCOUNT_URL = f"https://{ACCOUNT_NAME}.blob.core.windows.net"
ACCOUNT_URL_CDN = f"https://{ACCOUNT_NAME}.azureedge.net"
ACCOUNT_URL_CDN_CUSTOM = f"https://static.{ACCOUNT_NAME}.com"

CONTAINER_NAME = "mycontainer"
BLOB_NAME = "path/to/image.png"

# The "standard" blob client endpoint
blob_client = BlobServiceClient(
    account_url=ACCOUNT_URL,
    credential=CREDENTIAL,
).get_blob_client(
    container=CONTAINER_NAME,
    blob=BLOB_NAME,
)

# This works! It returns True
blob_client.exists()

# Now, using the Azure CDN blob endpoint
blob_client_cdn = BlobServiceClient(
    account_url=ACCOUNT_URL_CDN,
    credential=CREDENTIAL,
).get_blob_client(
    container=CONTAINER_NAME,
    blob=BLOB_NAME,
)

# This does not work, it raises a `ClientAuthenticationError` error
blob_client_cdn.exists()

# Using a custom CDN (which is fully verified)
blob_client_cdn_custom = BlobServiceClient(
    account_url=ACCOUNT_URL_CDN_CUSTOM,
    credential=CREDENTIAL,
).get_blob_client(
    container=CONTAINER_NAME,
    blob=BLOB_NAME,
)

# This does not work, it raises a `ClientAuthenticationError` error
blob_client_cdn_custom.exists()

Expected behavior
I would expect the client to authenticate correctly when using an Azure CDN or a custom CDN when the client authenticates correctly when using the standard blob endpoint.

Hi @kierandarcy , thanks for the feedback. We'll take a look at this and get back to you asap.

Hi @kierandarcy Kieran, I was unable to reproduce the ClientAuthenticationError you are facing on my local machine. However, I will note that there is a difference between my local test environment and your environment (I have tested it on Windows 10, while you are running Ubuntu), so I will spin up a Ubuntu VM to cross-verify my results.
Has the issue since resolved? We have had issues with CDN in the past related to permissions simply not propagating quickly enough, so I've attached a document here outlining that point:
https://docs.microsoft.com/en-us/azure/cdn/cdn-create-new-endpoint
Relevant excerpt:
'Select a Standard Akamai option from the drop-down list. (Deployment time for the Akamai tier is about one minute. The Microsoft tier takes about 10 minutes and the Verizon tiers take about 30 minutes.)'
Thanks!

Thank you for looking into this @vincenttran-msft!
It's been almost 24 hours since the CDN was created and I still have the error. I tested it locally on Ubuntu and Windows 10, and remotely on an Azure VM and a Google Cloud VM.
The CDN seems to be working fine through a browser, it is just the BlobClient that seems to continue to have an issue. I have been able to upload files and retrieve them from all three endpoints (origin, edge and custom). The custom SSL certificate looks correct (from Microsoft, with the correct domains).
I will wait to see if the issues resolves itself. I will update you if it does.

Hopefully this is useful (probably not, as it looks like it's just an HttpResponseError). This is from an Azure VM running Ubuntu.
In [2]: blob_client_cdn_custom.exists()
---------------------------------------------------------------------------
ClientAuthenticationError                 Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 blob_client_cdn_custom.exists()

File ~/staging/.venv/lib/python3.9/site-packages/azure/core/tracing/decorator.py:73, in distributed_trace.<locals>.decorator.<locals>.wrapper_use_tracer(*args, **kwargs)
     71 span_impl_type = settings.tracing_implementation()
     72 if span_impl_type is None:
---> 73     return func(*args, **kwargs)
     75 # Merge span is parameter is set, but only if no explicit parent are passed
     76 if merge_span and not passed_in_parent:

File ~/staging/.venv/lib/python3.9/site-packages/azure/storage/blob/_blob_client.py:1158, in BlobClient.exists(self, **kwargs)
   1156 except HttpResponseError as error:
   1157     try:
-> 1158         process_storage_error(error)
   1159     except ResourceNotFoundError:
   1160         return False

File ~/staging/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py:181, in process_storage_error(storage_error)
    178 error.args = (error.message,)
    179 try:
    180     # `from None` prevents us from double printing the exception (suppresses generated layer error context)
--> 181     exec("raise error from None")   # pylint: disable=exec-used # nosec
    182 except SyntaxError:
    183     raise error

File <string>:1, in <module>

ClientAuthenticationError: Operation returned an invalid status 'Forbidden'
ErrorCode:AuthenticationFailed

@vincenttran-msft Just to update you. It's been 4 days, and the error still exists.

Hi @kierandarcy Kieran, thanks for the follow-up! We were able to reproduce your error on an Ubuntu test VM and are currently working to further investigate the issue. I am sorry for any inconvenience this may cause, and we will get back to you when we have any updates or findings!
Thanks!

Hi @kierandarcy Kieran, as it turns out I was not able to reproduce your error as I had a mistake when copying my Account key into my test file. I would recommend you check your Account key and see if that resolves your issue.
If that does not resolve your issue, can you pull a recent requestID (within the last day) from a failing request? The requestID is x-ms-request-id header in the response. You may need to use Fiddler to look at the response or debug through the code (please let me know if you need help locating this). This will allow us to debug the issue from the service :)

Thank you @vincenttran-msft
One of the first things I tried was to recycle my keys, I have also tried using SAS tokens - all give the same response, which is correct operation when using the https://myaccount.blob.core.windows.net, but authentication errors when using an Azure CDN endpoint.

I'm also encountering this issue after upgrading azure-storage-blob from version 2.1 -> 12.10.0. As @kierandarcy noted, this is only an issue when a CDN URL is used. It is definitely not an account key issue, as the same action (we're collecting static assets to store on Blob) does not produce an error before upgrading, nor after upgrading if the environment is not using a CDN URL.
@vincenttran-msft Any update on your testing? This is actually blocking the deployment of a feature to our Production environment, so any advice or work-arounds would be greatly appreciated.

Hi @kierandarcy Kieran and @Sbacon017 Steph, sorry that both of you are running into this issue. Our team has been actively looking into the issue; however, I believe we are approaching the limit of what we can do with the currently provided information (as we are unable to reproduce the issue locally).
If either (or both of you) could provide a recent (as they are auto-cleared on a rolling basis) RequestID, then we would be able to further investigate. The RequestID is the x-ms-request-id header in the response. The easiest approach to retrieve this would be to us Fiddler to examine the response, and another possible alternative is to run your code in a debugger and retrieve the RequestID that way.
Thanks!

@vincenttran-msft I sent a request last night with this x-ms-request-id: 308a9386-d01e-0051-7628-498db8000000
How quickly do they auto-cleared? I can generate one of these fairly quickly.

@vincenttran-msft I have generated a few more RequestIDs for you (just in case you didn't manage to get to see the others - I guess were a few timezone apart).
# A successful response (without a CDN endpoint)
'x-ms-request-id': '369adfbf-801e-00b5-3d9c-4a2208000000'

# A failed response using the azuredge.net CDN endpoint
'x-ms-request-id': 'dc3b93de-e01e-0035-639d-4add0e000000'

# A failed response using a the CDN with a custom domain
'x-ms-request-id': '44f78dc3-701e-00b1-5d9d-4aaf0f000000'

These were generated using the example code in my original Steps to reproduce the behaviour (with account names and credentials updated).
This issue is preventing us from going live with our project. I would be willing to send you access tokens and keys (which I can recycle, if we do eventually get to go live) so you can reproduce the error yourself.
I have pasted the full output from running my Steps to reproduce the behaviour script if it helps.
In [2]: blob_client.exists()
Request URL: 'https://gamesmap.blob.core.windows.net/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Thu, 07 Apr 2022 16:31:26 GMT'
    'x-ms-client-request-id': '2ba19090-b690-11ec-b637-3d6a688c8c41'
    'Authorization': '*****'
Request body:
None
Response status: 200
Response headers:
    'Content-Length': '195963'
    'Content-Type': 'image/png'
    'Content-MD5': 'CHWOotVKq8/w7uG6kzMrbA=='
    'Last-Modified': 'Thu, 24 Mar 2022 09:54:18 GMT'
    'Accept-Ranges': 'bytes'
    'ETag': '"0x8DA0D7C436A3DDB"'
    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
    'x-ms-request-id': '369adfbf-801e-00b5-3d9c-4a2208000000'
    'x-ms-client-request-id': '2ba19090-b690-11ec-b637-3d6a688c8c41'
    'x-ms-version': '2020-10-02'
    'x-ms-creation-time': 'Thu, 24 Mar 2022 09:54:18 GMT'
    'x-ms-lease-status': 'unlocked'
    'x-ms-lease-state': 'available'
    'x-ms-blob-type': 'BlockBlob'
    'Content-Disposition': ''
    'x-ms-server-encrypted': 'true'
    'x-ms-access-tier': 'Cool'
    'x-ms-access-tier-inferred': 'true'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-client-request-id,Server,x-ms-version,Content-Type,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,Content-Disposition,x-ms-server-encrypted,x-ms-access-tier,x-ms-access-tier-inferred,Accept-Ranges,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'Date': 'Thu, 07 Apr 2022 16:31:26 GMT'
Response content:
Body contains image data.
Out[2]: True

In [3]: blob_client_cdn.exists()
Request URL: 'https://gamesmap.azureedge.net/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Thu, 07 Apr 2022 16:32:12 GMT'
    'x-ms-client-request-id': '46beedbe-b690-11ec-b637-3d6a688c8c41'
    'Authorization': '*****'
Request body:
None
Response status: 403
Response headers:
    'Content-Length': '748'
    'Content-Type': 'application/xml'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'X-Cache': 'TCP_MISS'
    'x-ms-request-id': 'dc3b93de-e01e-0035-639d-4add0e000000'
    'x-ms-error-code': 'AuthenticationFailed'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-error-code,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'X-Azure-Ref-OriginShield': '0DBJPYgAAAABhQgjqImZdQY9Gfc3OysxBTE9OMjFFREdFMDExMwA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'X-Azure-Ref': '0DBJPYgAAAACuOb0FYH0IT4JNCm3+NIjtTUFOMzBFREdFMDcxMwA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'Date': 'Thu, 07 Apr 2022 16:32:12 GMT'
Response content:

---------------------------------------------------------------------------
ClientAuthenticationError                 Traceback (most recent call last)
<ipython-input-3-aade6ad209e1> in <module>
----> 1 blob_client_cdn.exists()

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/core/tracing/decorator.py in wrapper_use_tracer(*args, **kwargs)
     71             span_impl_type = settings.tracing_implementation()
     72             if span_impl_type is None:
---> 73                 return func(*args, **kwargs)
     74
     75             # Merge span is parameter is set, but only if no explicit parent are passed

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_blob_client.py in exists(self, **kwargs)
   1153         except HttpResponseError as error:
   1154             try:
-> 1155                 process_storage_error(error)
   1156             except ResourceNotFoundError:
   1157                 return False

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    175     try:
    176         # `from None` prevents us from double printing the exception (suppresses generated layer error context)
--> 177         exec("raise error from None")   # pylint: disable=exec-used # nosec
    178     except SyntaxError:
    179         raise error

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in <module>

ClientAuthenticationError: Operation returned an invalid status 'Forbidden'
ErrorCode:AuthenticationFailed

In [4]: blob_client_cdn_custom.exists()
Request URL: 'https://static.gamesmap.uk/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Thu, 07 Apr 2022 16:33:52 GMT'
    'x-ms-client-request-id': '82ea3e9c-b690-11ec-b637-3d6a688c8c41'
    'Authorization': '*****'
Request body:
None
Response status: 403
Response headers:
    'Content-Length': '748'
    'Content-Type': 'application/xml'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'X-Cache': 'TCP_MISS'
    'x-ms-request-id': '44f78dc3-701e-00b1-5d9d-4aaf0f000000'
    'x-ms-error-code': 'AuthenticationFailed'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-error-code,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'X-Azure-Ref-OriginShield': '0cRJPYgAAAACpFIw1TVNgRLsvkhpuRcTkTE9OMjFFREdFMDEyMgA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'X-Azure-Ref': '0cRJPYgAAAABqi3bBiEA9RroHEa+DJoIcTUFOMzBFREdFMDcwOQA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'Date': 'Thu, 07 Apr 2022 16:33:53 GMT'
Response content:

---------------------------------------------------------------------------
ClientAuthenticationError                 Traceback (most recent call last)
<ipython-input-4-6305c8075a85> in <module>
----> 1 blob_client_cdn_custom.exists()

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/core/tracing/decorator.py in wrapper_use_tracer(*args, **kwargs)
     71             span_impl_type = settings.tracing_implementation()
     72             if span_impl_type is None:
---> 73                 return func(*args, **kwargs)
     74
     75             # Merge span is parameter is set, but only if no explicit parent are passed

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_blob_client.py in exists(self, **kwargs)
   1153         except HttpResponseError as error:
   1154             try:
-> 1155                 process_storage_error(error)
   1156             except ResourceNotFoundError:
   1157                 return False

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    175     try:
    176         # `from None` prevents us from double printing the exception (suppresses generated layer error context)
--> 177         exec("raise error from None")   # pylint: disable=exec-used # nosec
    178     except SyntaxError:
    179         raise error

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in <module>

ClientAuthenticationError: Operation returned an invalid status 'Forbidden'
ErrorCode:AuthenticationFailed

In [5]:

Hi @kierandarcy Kieran and @Sbacon017 Steph, thank you for your patience as we are trying our best to resolve this issue, and I really appreciate the follow ups from both of you!
We have looked into all the RequestIDs provided and unfortunately, they have not been as helpful as we were anticipating. However, we do have another thing we would like to check but would require you to follow the following procedure. If one (or both) of you could please follow this procedure and provide us with the information required, hopefully we will be able to identify the root cause!
Procedure:

Retrieve the value of string_to_sign (an easy way could be to set a breakpoint and give us the value that pops out): https://github.com/Azure/azure-sdk-for-python/blob/6580b278e0ff108e93b4ada80421f9895f526a04/sdk/storage/azure-storage-blob/azure/storage/blob/_shared/authentication.py#L128
From the same server request that you pulled the string_to_sign from, please also pull the RequestID just as you have both done previously

Thanks!

Hi @vincenttran-msft. I uncommented this line to produce these results: https://github.com/Azure/azure-sdk-for-python/blob/6580b278e0ff108e93b4ada80421f9895f526a04/sdk/storage/azure-storage-blob/azure/storage/blob/_shared/authentication.py#L143
The full output is below, but the highlights are:
A Successful response
# A successful response  (without a CDN endpoint)
 'x-ms-request-id': 'b6c32e11-e01e-0025-2919-4b1866000000'

# The output from `string_to_sign`
String_to_sign=HEAD

x-ms-client-request-id:218be790-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:25:56 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png

Failed responses
# A failed response using the azuredge.net CDN endpoint
'x-ms-request-id': '4eeaf368-001e-00ab-2819-4bced0000000'

# The output from `string_to_sign`
String_to_sign=HEAD

x-ms-client-request-id:25931ce6-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:26:03 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png

# A failed response using a the CDN with a custom domain
'x-ms-request-id': '4eeaf368-001e-00ab-2819-4bced0000000'

# The output from `string_to_sign`
String_to_sign=HEAD

x-ms-client-request-id:25931ce6-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:26:03 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png

And now, the verbose output from all three...
In [2]: blob_client.exists()
String_to_sign=HEAD

x-ms-client-request-id:218be790-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:25:56 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png
Request URL: 'https://gamesmap.blob.core.windows.net/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Fri, 08 Apr 2022 07:25:56 GMT'
    'x-ms-client-request-id': '218be790-b70d-11ec-baee-dd1cad881d9a'
    'Authorization': '*****'
Request body:
None
Response status: 200
Response headers:
    'Content-Length': '195963'
    'Content-Type': 'image/png'
    'Content-MD5': 'CHWOotVKq8/w7uG6kzMrbA=='
    'Last-Modified': 'Thu, 24 Mar 2022 09:54:18 GMT'
    'Accept-Ranges': 'bytes'
    'ETag': '"0x8DA0D7C436A3DDB"'
    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
    'x-ms-request-id': 'b6c32e11-e01e-0025-2919-4b1866000000'
    'x-ms-client-request-id': '218be790-b70d-11ec-baee-dd1cad881d9a'
    'x-ms-version': '2020-10-02'
    'x-ms-creation-time': 'Thu, 24 Mar 2022 09:54:18 GMT'
    'x-ms-lease-status': 'unlocked'
    'x-ms-lease-state': 'available'
    'x-ms-blob-type': 'BlockBlob'
    'Content-Disposition': ''
    'x-ms-server-encrypted': 'true'
    'x-ms-access-tier': 'Cool'
    'x-ms-access-tier-inferred': 'true'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-client-request-id,Server,x-ms-version,Content-Type,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,Content-Disposition,x-ms-server-encrypted,x-ms-access-tier,x-ms-access-tier-inferred,Accept-Ranges,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'Date': 'Fri, 08 Apr 2022 07:25:56 GMT'
Response content:
Body contains image data.
Out[2]: True

In [3]: blob_client_cdn.exists()
String_to_sign=HEAD

x-ms-client-request-id:25931ce6-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:26:03 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png
Request URL: 'https://gamesmap.azureedge.net/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Fri, 08 Apr 2022 07:26:03 GMT'
    'x-ms-client-request-id': '25931ce6-b70d-11ec-baee-dd1cad881d9a'
    'Authorization': '*****'
Request body:
None
Response status: 403
Response headers:
    'Content-Length': '748'
    'Content-Type': 'application/xml'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'X-Cache': 'TCP_MISS'
    'x-ms-request-id': '4eeaf368-001e-00ab-2819-4bced0000000'
    'x-ms-error-code': 'AuthenticationFailed'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-error-code,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'X-Azure-Ref-OriginShield': '0i+NPYgAAAAA97o8TBF+nQIAVmanuNuuBTE9OMjFFREdFMDIxNwA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'X-Azure-Ref': '0i+NPYgAAAADImNP5zb/iS6vH7OSbl7tTTUFOMzBFREdFMDcyMQA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'Date': 'Fri, 08 Apr 2022 07:26:03 GMT'
Response content:

---------------------------------------------------------------------------
ClientAuthenticationError                 Traceback (most recent call last)
<ipython-input-3-aade6ad209e1> in <module>
----> 1 blob_client_cdn.exists()

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/core/tracing/decorator.py in wrapper_use_tracer(*args, **kwargs)
     71             span_impl_type = settings.tracing_implementation()
     72             if span_impl_type is None:
---> 73                 return func(*args, **kwargs)
     74
     75             # Merge span is parameter is set, but only if no explicit parent are passed

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_blob_client.py in exists(self, **kwargs)
   1153         except HttpResponseError as error:
   1154             try:
-> 1155                 process_storage_error(error)
   1156             except ResourceNotFoundError:
   1157                 return False

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    175     try:
    176         # `from None` prevents us from double printing the exception (suppresses generated layer error context)
--> 177         exec("raise error from None")   # pylint: disable=exec-used # nosec
    178     except SyntaxError:
    179         raise error

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in <module>

ClientAuthenticationError: Operation returned an invalid status 'Forbidden'
ErrorCode:AuthenticationFailed

In [4]: blob_client_cdn_custom.exists()
String_to_sign=HEAD

x-ms-client-request-id:2a5e3346-b70d-11ec-baee-dd1cad881d9a
x-ms-date:Fri, 08 Apr 2022 07:26:11 GMT
x-ms-version:2020-10-02
/gamesmap/media/images/system/photo-2454235000000215156.png
Request URL: 'https://static.gamesmap.uk/media/images/system/photo-2454235000000215156.png'
Request method: 'HEAD'
Request headers:
    'x-ms-version': '2020-10-02'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.9.0 Python/3.9.5 (Linux-5.13.0-39-generic-x86_64-with-glibc2.31)'
    'x-ms-date': 'Fri, 08 Apr 2022 07:26:11 GMT'
    'x-ms-client-request-id': '2a5e3346-b70d-11ec-baee-dd1cad881d9a'
    'Authorization': '*****'
Request body:
None
Response status: 403
Response headers:
    'Content-Length': '748'
    'Content-Type': 'application/xml'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'X-Cache': 'TCP_MISS'
    'x-ms-request-id': '2b423b58-401e-0085-0a19-4b9cc7000000'
    'x-ms-error-code': 'AuthenticationFailed'
    'Access-Control-Expose-Headers': 'x-ms-request-id,x-ms-error-code,Content-Length,Date,Transfer-Encoding'
    'Access-Control-Allow-Origin': '*'
    'X-Azure-Ref-OriginShield': '0k+NPYgAAAACix8l/0R91R4UN/IXwb8V6TE9OMjFFREdFMDExNAA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'X-Azure-Ref': '0k+NPYgAAAACjA9mnmfRZTL3XFtowlo3XTUFOMzBFREdFMDcxNwA2ZmJlNzM3NC03YWIzLTQzNTItODRiNC0xMGIwNjRhYjA4Zjc='
    'Date': 'Fri, 08 Apr 2022 07:26:11 GMT'
Response content:

---------------------------------------------------------------------------
ClientAuthenticationError                 Traceback (most recent call last)
<ipython-input-4-6305c8075a85> in <module>
----> 1 blob_client_cdn_custom.exists()

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/core/tracing/decorator.py in wrapper_use_tracer(*args, **kwargs)
     71             span_impl_type = settings.tracing_implementation()
     72             if span_impl_type is None:
---> 73                 return func(*args, **kwargs)
     74
     75             # Merge span is parameter is set, but only if no explicit parent are passed

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_blob_client.py in exists(self, **kwargs)
   1153         except HttpResponseError as error:
   1154             try:
-> 1155                 process_storage_error(error)
   1156             except ResourceNotFoundError:
   1157                 return False

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    175     try:
    176         # `from None` prevents us from double printing the exception (suppresses generated layer error context)
--> 177         exec("raise error from None")   # pylint: disable=exec-used # nosec
    178     except SyntaxError:
    179         raise error

~/projects/games-map/.venv/lib/python3.9/site-packages/azure/storage/blob/_shared/response_handlers.py in <module>

ClientAuthenticationError: Operation returned an invalid status 'Forbidden'
ErrorCode:AuthenticationFailed

In [5]:

Hi @kierandarcy Kieran, I have come back with (potentially) good news! From our testing, we have found that the CDN server seems to be changing the HTTP verb on HEAD requests to GET (exists() runs as a HEAD request, and so fails).
Unfortunately, this is the at the limit of what the client team can investigate. With that being said, we have already started an internal conversation with the Service team.
The first step would be to open a New Support Request ticket. You can find this at the bottom of your Storage Account page in Azure Portal. Please include your debug logs and the requestIDs just as you have done here so that they may be able to further assist you.
Note: It would be a good idea to regenerate the requests as close as possible to opening your ticket in order to prevent them from being autocleared from the logs.

In the meantime, here are some possible workarounds that may be able to unblock your workflow while you await a response from the Service team.

Since we have only observed (from our limited testing) that the CDN server is changing the verb from HEAD to GET, it is possible that any operations that doesn't send a HEAD (i.e. exists(), get_blob_properties()) may still work properly. Thus, if your workflow does not use these, it may be possible to move forward.
We have only found this bug to be reproducible on the Microsoft CDN (classic) pricing tier. Therefore, if you can migrate to a different pricing tier (in our testing we found Standard Verizon and Standard Akamai to be working), this may also unblock your workflow.

Thank you for the many follow-ups and your patience, we really appreciate it and hopefully the Service team or one of these workarounds will help to unblock your workflow.
CC: @Sbacon017 Steph

@vincenttran-msft and @Sbacon017
I had a long support chat with the CDN Service Team. Azure CDN is not enabled for "Free Trial" accounts, and that the issue will go away when I upgrade my account. I upgraded my account today, and I still have the issue. Hopefully it is resolved soon.
I was directed me to these two pages by way of explanation:

https://intellipaat.com/community/44245/is-cdn-is-not-under-free-trail
https://docs.microsoft.com/en-us/azure/media-services/latest/stream-scale-streaming-cdn-concept#enable-azure-cdn-integration

Hi @kierandarcy Kieran, I understand that you have upgraded your account from "Free Trial", but if your CDN endpoint is still hosted under the Pricing tier Microsoft CDN (classic), I believe you will still run into the error as we reproduced the error locally on a non-trial account. If possible, could you try one of the workarounds as provided in a previous response above?
Thanks!

Thank you @vincenttran-msft.
The option to change the Pricing Tier is not available for some reason (the CDN service team thought it might be because I was on a free trial. but that seems not to be related). I guess my only option is to fully delete the CDN and create another with a different pricing tier - I will try that today.
I also tried other methods (e.g. get_blob_properties() as you suggested). The issue exists regardless of whether it is a GET or a HEAD request

@kierandarcy does the issue still happen with your Custom CDN?
I don't use Azure CDN, I use CloudFlare instead, and I'm having the same issue.

@vincenttran-msft what's the latest on that conversation with the CDN service team? This issue is still affecting Microsoft CDNs (classic). I've opened up an Azure support ticket recently about this as well.


Rename from InWorldz to Halcyon

The Ubuntu build is a known failure.  The PR CI build under Windows succeeded.


how to run if we have multiple input in x and one output.

getting following error
ValueError: The number of columns of X must be the same as the length of exog_order.
my X has 7 column and Y has one column. how I run this.

You need to set exog_order as a list with 7 elements in it.

You need to set exog_order as a list with 7 elements in it.


Upgrade 1.20.3

Upgrade to backstage version 1.20.3
I used the yarn backstage-cli versions:bump to upgrade from version 1.16.0 to 1.20.3. This bumped the versions of packages. As part of this it bumped the versions in our two plugins. It also added the "sideEffects": false entry to the package.json file for each plugin. Reference: https://github.com/backstage/backstage/releases/tag/v1.18.0
I then used https://backstage.github.io/upgrade-helper/?from=1.16.0&to=1.20.3 to check if anything else needed to be updated. I made the updates as noted on that page.
I did NOT migrate us to use the new backend system as part of this upgrade. I set the env variable to  LEGACY_BACKEND_START=true instead, as the default is to use the new backend system. Reference: https://github.com/backstage/backstage/releases/tag/v1.19.0

@sheaphillips - thank you for giving it a thorough look over!


Block-network by default

@hchauvin @damienmg wdyt about adding block-network by default to bazel_java_integration_test and allowing users to opt-out? It's a bit more code but I think that it pushes the users to do the right thing out of the box

block-network is a big slowdown on Linux and has no effect on other platform, I definitely think this should be the default behavior but the perf hit is just too high.
@philwo knows more about that performance issue.

Recent Linux kernels supposedly have a fix for the performance issue that we observed. I also agree that this "should" be the default behavior. I won't be able to work towards it before my vacation for Bazel in general though, so maybe give it a try here?
Here's some context: https://github.com/bazelbuild/bazel/issues/3453#issuecomment-318021140

Maybe it was the 4.12 kernel that did it, but I know the 4.19 kernel I have been running has been awesome.  I'm finally seeing 100% CPU usage reliably on a 20 core/40 thread box.  I've been running with either --experimental_sandbox_default_allow_network=false or the patch which flipped the default to allow-network reverted for years.

Thanks for updating!
On Fri, 6 Sep 2019 at 21:17 Austin Schuh<EMAIL_ADDRESS>wrote:

Maybe it was the 4.12 kernel that did it, but I know the 4.19 kernel I
have been running has been awesome. I'm finally seeing 100% CPU usage
reliably on a 20 core/40 thread box. I've been running with either
--experimental_sandbox_default_allow_network=false or the patch which
flipped the default to allow-network reverted for years.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel-integration-testing/issues/76?email_source=notifications&email_token=AAKQQF666CZ4GJJG6ALKAHLQIKND5A5CNFSM4E5PX7AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6DUXKY#issuecomment-528960427,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKQQF2QMLAHDDY2BIFPNGTQIKND5ANCNFSM4E5PX7AA
.

--
Ittai Zeidman
Cell:<PHONE_NUMBER>
40 Hanamal street, Tel Aviv, Israel
http://www.wix.com


Define default value on CRUD service properties

It would be very useful to set a default value on not required CRUD properties.
This default value would be applied just on insert operations.

To be addressed when working on #4


gcc: Place target system libraries in a separate output

Describe the bug
Cross-compiled gcc 9, used in cross-compiling stdenv in master, puts target libraries in its out output, forcing programs that link against the GCC runtime to depend on the GCC closure.
For armv7l, this results in outputs built with the cross-compiling stdenv depending on the gcc out output, which comes to 748 MB not counting its own closure. The actual set of target-system libraries, in ``armv7l-unknown-linux-gnueabihf/lib`, comes to 35MB.
To Reproduce
Steps to reproduce the behavior:

Build a cross-compiling gcc: nix-build -A pkgsCross.armv7l-hf-multiplatform.buildPackages.gcc-unwrapped
Investigate result/<system>/lib and note libraries for target system are present in that path under the out output.
Obtain the lib output nix-build -A pkgsCross.armv7l-hf-multiplatform.buildPackages.gcc-unwrapped.lib
Test .so files with the file utility, note that they are only for the compiler's host platform, not its target platform.
Build e.g. nix cross-compiled: nix-build -A pkgsCross.armv7lhf-multiplatform.nix
View the runtime dependency closure sorted by size: nix-store -qR result | xargs du -d0 -h -c | sort -h

Expected behavior
Cross-compiling GCC should place target system libraries in a separate output, so programs which it dynamically links with those libraries do not pull all of GCC into their runtime closure.
Additional context
This is a continuation of the investigation in https://github.com/NixOS/nixpkgs/issues/81153, as to why cross-compiled nix has build-system dependencies in its runtime closure.
Metadata
 - system: `"x86_64-linux"`
 - host os: `Linux 5.5.2, NixOS, 20.03pre-git (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.2`
 - channels(root): `"nixos-19.09.1965.274e095f761"`
 - channels(edward): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
- pkgsCross.armv7l-hf-multiplatform.buildPackages.gcc-unwrapped
# a list of nixos modules affected by the problem
module:

@flokli I'm pretty sure this is the reason build-system paths are still making it into the closure for cross-compiled nix, just not sure how to fix it.

ccing @Synthetica9 as the gcc 9 maintainer

https://github.com/NixOS/nixpkgs/pull/58606 should fix this

@lopsided98 looks like it, though some libraries for GCC's host platform (i.e. the build platform for things built with that gcc) would still be in the lib output, no? I'll try cross-compiling Nix against that PR and see if there is still build-system pollution in the closure.

This seems to be a dup of #58501 but that was closed...not sure why.

This still should be desirable. @matthewbauer, was https://github.com/NixOS/nixpkgs/pull/59787 probably not enough?

Right - that was not enough, but https://github.com/NixOS/nixpkgs/commit/e1831ebea3bcb415800a74fe458f802e63549e1c#diff-c9fb175e04caebec67031e96e3dc5f20 should be.

Yes, I can confirm that; I recently built a minimal cross-compiled NixOS system with no build arch references.


Feature: option to configure renderIndentGuides

VSCode Version: Version 1.10.2 (1.10.2)
OS Version: Mac OS 10.4
configure options to renderIndentGuides

set colors based on the depth hierarchy
highlight options on matching braces
different line options - dashed, solid etc..

Please don't add more than one request per issue. I

Having indentGuides enabled is very very useful, I can't imagine working without them, however, seeing them active all over the file is a little annoying. It would be great it the work like sublime text's. Here's a screenshot of sublime text:

(1) They only show up relative to the cursor position.
(2) They don't show up here, which is good.
(3) Also, the brackets of the block are highlighted because the cursor is inside it, that's very useful. VSCode only highlight them when the cursor is exactly next to the bracket. The brackets are highlighted with a subtle underline easy on the eyes :sweat_smile:, but I guess that's another subject.

how is this going?

This would be mighty helpful - any update?

I want to second this, specially the one recommended by @cheoAlejo as it is exactly what I was going to request, showing indent guides only relative to the cursor position.  Thank you.

Thanks @aderchox , I'll try that.

Listo.
https://git.delfineco.com/db-design/modelo-rcd
https://git.delfineco.com/db-design/modelo-crb
On Wed, Sep 22, 2021 at 10:10 AM aderchox @.***> wrote:

I want to second this, specially the one recommended by @cheoAlejo
https://github.com/cheoAlejo as it is exactly what I was going to
request, showing indent guides only relative to the cursor position.
Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/23609#issuecomment-925023623,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQICF3LL6JULJOTIKNTXVDUDHWWFANCNFSM4DFVWZ3A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--

For 1. see #122193
For 2. see #85775


Update readyplayer3.tsx

updating workshops date to reflect accurate dates!

📝 Changed routes:

/hackathon/readyplayer3

(view full size)

Commit e22e03db936d16c6bd08540310000439c33f223e (https://thirdweb-o2vdqlhth.thirdweb-preview.com).


Fixed randomops tests

@sheeley @bgphipps Please review

LGTM :+1:

LGTM :+1:

Looks goodish to me, but I'm prolly not the best judge.


Locales generation

Hello,
I don't know if I am doing something wrong, but *.js locale files are not generated properly. In our project we currently have two locales (en, sk), so our i18n-js.yml config contains following:
translations:
- file: "app/assets/javascripts/i18n/en.js"
  only: "en.*"
- file: "app/assets/javascripts/i18n/sk.js"
  only: "sk.*"

The en.js is generated properly, however sk.js fills in all locales with Slovak translations. Therefore, even if I18n.currentLocale() returns correct locale set in Rails (4.2.5), every I18n.t() call returns Slovak translation of specified key.
Thanks for any tips.

You said:
sk.js fills in all locales with Slovak translations

Can you explain more why this is a problem?
I parse the above statement as:
// sk.js
I18n.locales = {"sk": {"slovak": "translations"}}
// Not exact code, just try to present the expected content here :P

en.js
I18n.translations || (I18n.translations = {});
I18n.translations["en"] = I18n.extend((I18n.translations["en"] || {}), {"key1" : {"subkey1": "value_en"}});
I18n.translations["ru"] = I18n.extend((I18n.translations["ru"] || {}), {});
I18n.translations["en-GB"] = I18n.extend((I18n.translations["en-GB"] || {}), {});
...

sk.js
I18n.translations || (I18n.translations = {});
I18n.translations["sk"] = I18n.extend((I18n.translations["sk"] || {}), {"key1" : {"subkey1": "value_sk"}});
I18n.translations["en"] = I18n.extend((I18n.translations["en"] || {}), {"key1" : {"subkey1": "value_sk"}});
I18n.translations["ru"] = I18n.extend((I18n.translations["ru"] || {}), {"key1" : {"subkey1": "value_sk"}});
I18n.translations["en-GB"] = I18n.extend((I18n.translations["en-GB"] || {}), {"key1" : {"subkey1": "value_sk"}});
...

Have you tried changing the value in only node for sk.js?
Try to change it to en.* or ru.* to see if it generates translations as expected

If I change it to en.* or ru.*, everything works as expected.

Can you paste a portion of your translations for sk and en?
Maybe there is something special that triggers this issue?
I can't really determine without the translation right now :)

Sure, here is a part of "comments" translations:
en:
  comments:
    errors:
      edit_banned:      "You can't edit banned comment."
      reply_banned:     "You can't reply to banned comment."
    labels:
      add_comment:      "Add comment"
      author:           "Author"
      comments:         "Comments"
      reply:            "Reply"
    text:
      banned:           "Comment has been deleted due to inappropriate content."
      remove_success:   "Comment has been successfully removed."

sk:
  comments:
    errors:
      edit_banned:      "Nemôžete editovať zakazáný komentár."
      reply_banned:     "Nemôžete odpovedať na zakázaný komentár."
    labels:
      add_comment:      "Pridať komentár"
      author:           "Zadávateľ"
      comments:         "Komentáre"
      reply:            "Odpovedať"
    text:
      banned:           "Komentár bol zmazaný pre nevhodný obsah."
      remove_success:   "Komentár bol úspešne odstránený."

If you have tested all other locales and only sk got problem
Maybe there is something wrong in sk translations?
To confirm this you can remove your sk translation files and
copy files for a working locale (like en) into sk and see if that happens again.

After debugging I have finally found what was the problem - I forgot to set the fallbacks: false in config/i18n-js.yml. Everything works now :+1:
Sorry for bothering you and thanks for your help.


Designated Upload Container

Hi there - I am looking at the read me document and it says for testing, upload an EICAR file to the designated upload container. I deployed this project using the existing ARM template that is given. Within the host storage account, I do not see any containers to upload to except for "azure-webjobs-hosts" and "azure-webjobs-secrets". After peeking at the given ARM template, there seems to be "new-files", "quarantined-files", "clean-files", and "source-code" storage containers created within the storage account but they are not created for me. I feel like I am missing something. Any input is helpful thanks!

Hi,
This system was designed to protect existing container in an existing storage account. The storage account you are referring to is a storage account created during deployment to serve as a host for the Azure function.
The ARM template expects the details of the existing storage account to protect through the following parameters:
SubscriptionIDOfTheTargetStorageAccount
NameOfTheResourceGroupTheTargetStorageAccountBelongsTo
NameOfTargetStorageAccount
NameOfTargetContainer (Optional, defaults to "new-files")
After the deployment you should go to the 'NameOfTargetStorageAccount' storage account and upload a the EICAR file to the container 'NameOfTargetContainer', if the container doesn't exist, you can create it and the system will work without re-deployment.

Closing for inactivity.
Feel free to reopen this issue.


Enable UT test in nightly with On-demand test and On-demand test for public pytorch without torch-xpu-ops update

Stop the preci since there is a conflict to resolve.

close it because the function has enabled by other PRs


[vcpkg] remove flushall

@atkawa7,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot


GitLab Merge request remain not covered after docu-signed

Steps :

Go to gated Easycla PR
Click on Not covered
Proceed as an Individual Contributor
Click on Sing CLA
Fill docusign pdf form

Actual results : GitLab Merge request remain not covered after docu-signed
Refer video clip :
https://images.zenhubusercontent.com/194341141/464fc3e1-b6a5-432d-8939-b673edf2d172/notcovered.mp4

Tested issue on dev site & found that it is reproducible as reported, So making issue status QA-Failed
Refer video clip :
https://user-images.githubusercontent.com/75417507/220549895-683583ca-bb3a-4952-93e8-56cfbe8f12b8.mp4

MR : https://gitlab.com/marsgitlabgroup/repo01/-/merge_requests/32

Tested issue with multiple user & found that it is working fine except veerendrat user

Added separate tickets for failed user : https://app.zenhub.com/workspaces/easycla-5ed619832c71a855a76ef404/issues/gh/communitybridge/easycla/3816

Refer screenshot for more details :

@thakurveerendras confirmed its working as expected. Hence closing


Proposal: Template inheritance

At Twitter we run into a very common case where we want to have one template where we maintain the "bones" of the page and each real page on the site merely replaces sections of that page with appropriate content. We would like to avoid putting any business logic in the bones of the page for including the right content, nor do we want to write custom backing code for each of those pages.
I give up trying to put my code in the issue, here is a gist: https://gist.github.com/1854699

I'm pretty uninterested in the logic-less template stuff, but this strikes me as something that could be really useful to have, and so I'd like to see something like this.
But what strikes me about it is that sub.mustache is not really a mustache template but more of a list of substitutions (which are mustache templates). Is there any way an inheriting template isn't just a list of substitutions? Because if not, why couldn't it just be defined that if an implementation wants to create a mechanism for creating derived templates, it can do that by letting you define (however you like; json? yaml?) a mapping of names to mustache-template strings or mustache files and replacing regular interpolation tags with the indicated mustache code when the derived template is loaded? It seems to me (albeit after only like 4 minutes thought) that this would potentially be even more convenient while simultaneously not requiring major changes to the parsers of existing mustaches. And also, it would prevent us from using up two precious special characters to implement this proposal. But, I'm probably missing something in that logic.

Two cents:
As @davidsantiago, I regret that sub.mustache is "not really a mustache template", and, more specifically, that we don't know what happens when sub.mustache contains content outside of the substitutions blocks. That alone makes the gist weird, even though the need is pretty clear.
I know about a solution for this problem: Ruby on Rails' one : one template is rendered in one layout.
This gist show hows a Rails-like two-pass rendering could look in Mustache (the syntax mimics Rail's one) : https://gist.github.com/1855111
The main differences with @spullara's gist:

sub.mustache has no reference to its layout - the rendering code should take care of rendering sub.mustache in the layout.mustache
the content of sub.mustache is no longer a specific substitution, instead there is a dedicated place for it in the layout.
there is no default value for any substitution

Thanks for the feedback. sub.mustache is actually a mustache template, the example usage failed to show that in all its glory. Another example that has come up for us is for having similar widgets that are displayed differently in different places but use the same base template. I do agree though that within the {{<super}}, the top level has to be replacements. However, nothing stops you from using the {{<}} syntax anywhere within any template. {{<}} means "replace any blocks in this partial with blocks that i have specified within this tag, then include the partial".
The RoR solution lacks the ability to have multiple layouts in a single page, also I need some external mechanism to connect the two templates. It also doesn't seem to support a large hierarchy of template "classes". For example, we might have:
General Twitter Page < Settings style page < Profile settings page
Re: characters. This scheme can be implemented without introducing new characters through some elaborate lambda gymnastics but it wouldn't be very clear what was going on when you looked at the template. Could be ameliorated by using a naming convention e.g. {{#<super}} and {{#$content}}. My personal opinion is that this is such a widespread issue with real life mustache usages that it just might be worth it.

{{<}} means "replace any blocks in this partial with blocks that i have specified within this tag, then include the partial".

Thanks for this sentence. It's quite clear, now.
It's also clear that {{<}} can be seen as a kind of inheritance, and as well as a function call with arguments. More people should know Handlebars.js. In Handlebars, no need for new punctuation, no need for an extension to the spec: I quite think you could just write a simple 'content_for' helper, and write : https://gist.github.com/1855986.
Really, Yehuda Katz's announcement is refreshing: http://yehudakatz.com/2010/09/09/announcing-handlebars-js/

That is pretty close. The only gap is that when you use multi-level inheritance the higher one in the stack wins instead of the lowest one in the stack. Default content also doesn't work with this model.

OK, yeah, that functionality wasn't clear to me from the initial example. That does leave the "templateness" of sub.mustache intact. However, as you say, there is still the way that the inside of the ">" tag is basically putting forth a map notation for mustache. Mustache has always seemingly been happy to leave maps to the language implementing it, so I still feel a little uncomfortable with this and can't say why. I just wish that the inside of that tag didn't require so much special meaning and new parsing. That is why I still really like the idea of letting this just be a partial with template substitutions for interpolation tags. The spec is already vague about where exactly the implementation should go looking for partials, so this could be very easily slipped into current implementations with a minimum of new syntax, just an expansion of what partials mean, in that partials might have associated files that create substitutions when they are loaded.
Regarding the other character, I was saying that I think you could leave off the $, and just jam the template into the ordinary interpolation tags. This is already kind of what lambdas do, in that a lambda passed in for an interpolation tag will return mustache code that is then parsed and rendered in that context. So there is already a precedent for a regular interpolation tag being used as a site for more template code to be inserted. The difference here would be that instead of this happening due to a function passed into the context map, it's happening because a derived template is instantiating itself (and thus wouldn't need to happen every time the template is called).

The biggest problem with using # is that the identity transformation is so goofy. Even though it looks like you are replacing it with the same thing, they would be interpreted much differently:
https://gist.github.com/1881261

I'm not at all following what you're saying... What is it about the # tags? What is the identity transformation?

Ah sorry, by "identity transformation" I mean a replacement that leaves you with the original. If we reuse # tags we have the issue that to replace the section with something that means the same thing you have to double them.

OK, I see now what you're saying. If we grant the syntax you want for using sections as keys and their values in a partial, the repetition in that example seems clear enough to me. Certainly the identity transformation is not a motivating use case for this feature, and having the source code for it, when it is necessary, shake out in this way doesn't strike me as sufficient motivation for the addition of a new tag type. There are no current semantics defined for sections when they are being used as the sites of template inheritance replacements, so there is some leeway here to posit new rules if that's necessary.

So currently the semantics for # sections is such that if they are not present in the backing code, they are "false". It might make more sense to use {{^}} instead in this case to get something that is closer to the {{$}} behavior. Not sure if I am explaining this well enough, but {{#}} has very different semantics.

I don't know if you're explaining it well enough, but I'm not quite following you. I assume you're talking about the resolution rules hereabouts: https://github.com/mustache/spec/blob/master/specs/sections.yml#L11 Those rules name a few different places to look for the contents of a section tag... is there some reason another place to look couldn't be added to those rules? I'm not seeing it.

So lets take the example where you want a page to evaluate two different ways, once with the default text and once with your modified text. In order for that to work, based on Rule 2), you have to have a singular valued hash matching the name of the section in the top level template so the section renders at all:

Walk the context stack from top to bottom, finding the first context
that is a) a hash containing the name as a key OR b) an object responding
to a method with the given name.

If you don't have that, the section won't be evaluated. When I extend the top level template, suddenly that "section" is satisfied and it doesn't even reference the section code that I use in the super template. Having to offer specific backing code in order for the template to render properly is what I am trying to avoid by suggesting a declarative template inheritance feature. This is why the sub template isn't just a hash of values but literally replaces the template code in the super template. The real issue is the default for {{#}} is false, not true -- which is why I suggested that {{^}} would be more appropriate.

OK, gotcha, thanks. I had not been considering the use case of wanting to have default code for when nothing is provided by a derived template and there is no desire to have a flag in the context to control whether that default code renders or not. I agree that that's not desirable.
If a partial has {{something}}, {{#something}}{{/something}} or {{^something}}{{/something}}, I thought we'd like to have a way to replace that tag with an arbitrary expansion, but Mustache doesn't currently have a way to name groups of code other than sections and inverted sections. So then you'd have to say: when you're replacing, match the tag names regardless of the tag type. That way you could get a section/inverted section named "something" inserted in the place of {{something}} or whatever. Here we're using sections/inverted sections just for their code-grouping semantic to define the key/value syntax for the replacement part of the {{<}} tag. Although you're right that {{^}} would be more natural for the replacement sites, it's not clear to me which is more natural and meaningful in the key/value replacement syntax, especially when you add on the question of whether they should match the replacement site or be the opposite for the replacement to avoid confusion about "double negatives."
That makes me unhappy, and you already had a solution to that, which was to define a new tag type that allows you to name a group of code, but without the implied "here or not" choice between a section or inverted section. So {{#}} means "Render this group of code if we are given a truthy value for its name," {{^}} means "Render this group of code if we aren't given a truthy value for its name," and {{$}} means "Render this group of code if we aren't given a substitution from a partial, and if we are, render that replacement." This expansion of syntax allows more information to remain "in mustache," as mustache is then expanded to mean more things. This trades away the ability to "replace anything" in the parent template for one that has to be prepared with replacement sites and requires a special idea of what is "default" in the template that is above and beyond just the template itself as it would already exist in mustache as it is today.
I also had a solution to that, which was to not get into the key/value syntax business in the first place and leave those to the implementation's maps. In that world, {{}}, {{#}}, and {{^}} can all be replaced equally naturally when the super (actually just a partial, now) is loaded by the implementation with the use of a map of the tag name->source code replacements that the implementation knows how to find. All that's needed is some rules for how those replacements should be made from the map, assuming this map already exists. This offers "replace anything" capability and retains implementation simplicity by avoiding the addition of any new syntax, but it comes at the cost of requiring that derived templates have some external storage format outside of mustache to store their replacement maps (the spec itself shows that mustache can be very naturally embedded in YAML, for one example). I think we assign different weights to how much we value these things; I don't value having a mustache version of a key/value map that much.
Anyhow, I think your idea for derived templates is great, and I think I'm gonna try adding my version to my mustache implementation just to check it out. I just hope the mustache language stays small and provides a small number of very flexible primitives. The language is pretty much defined by the 5 or 6 special characters that start tags, and I hope the language doesn't expand to many more tags than that, or it'll stop being easy to learn and keep in your head.

Anyone know if there are any plans for this to make it into the Ruby version?

There's a pull request open right now. The original author seems to have hit a roadblock, if you want to jump in and help out :)
mustache/mustache#186

@bobthecow Will take a look.

In some Mustache libraries, templates are objects. One loads templates, and then feeds them with values, gets a rendering back.
The GRMustache implementation allows a user to feed a template with many values, strings, numbers, etc. and also other templates. A regular variable tag {{template}} would then render just the same as a partial tag {{<partial}}. Except that the included partial is not hard-coded: it is provided with the rest of the rendered data (aka ViewModel). This is called "dynamic partials".
Now, why not having {{#template}}...{{/template}} be to {{<layout}}...{{/layout}} what {{template}} is to {{>partial}}? Well, this feature has landed in GRMustache.swift. Other implementors may be interested.

Would somebody mind explaining the basic steps they took (in English) to add this feature to their Mustache implementation? I am looking to add the feature to my own implementation.

Hi @thelucid. It's rather tricky to explain in English :-)
Pick words. For example: "partial override tag" for {{<layout}}...{{/layout}} and "block" for {{$content}}...{{/content}}. Blocks are not sections: their names are not data identifiers.
Pick the correct data structures. I guess you already have a type for Partial tags which fundamentally is a pair (name, content). OK. So partial override tag are a pair (parent partial, inner content). Blocks are a pair (name, inner content).
Update your rendering algorithm (reference implementation):

When the rendering engine renders a partial override tag, enqueue it, render the content of the parent partial, and then dequeue.

When the rendering engine renders a block, resolve it against the queue of partial override tags (1), and render the inner content of the resolved block.

The resolving algorithm (1) of a block against the queue of partial override tags is (reference implementation):

Iterate all partial override tags in the queue, starting from the last enqueued one. Step: If the parent partial has not been used yet, resolve the block against the inner content of the partial override tag (2). If the block has actually been overriden, flag the parent partial as used so that it is not reused.

The resolving algorithm (2) of a block against some content is (reference implementation):

For each element in the content:

If it is a block with the same name, the resolved block is that block, and flag the block as actually overriden.

If it is a partial override tag, resolve the block against the content of the parent partial, and then against the inner content of the partial override tag. If one of those two resolutions has actually overriden the block, well carry this flag.

If it is a partial tag, resolve the block against the content of the partial. Carry the actually overriden flag.

And you're done. You get a super duper implementation which makes sure that any part of a template can be refactored with partials and partial override tags, without requiring any modification anywhere else (in other templates that depend on it, or in application code).
If you need a test suite, pick https://github.com/groue/GRMustache.swift/blob/Swift2/MustacheTests/Public/SuitesTests/GRMustacheSuite/GRMustacheSuite/partial_overrides.json, the tests from mustache.java, hogan.js, and mustache.php.

@groue Awesome, thanks for the detailed explanation… it's a little more complicated to implement than I'd hoped but I'm sure I'll get there. If be interested to see a set of examples added to this spec.

Hey guys, I'm in the process of implementing this in Tache, so there will finally be a Ruby implementation that supports layouts! Whilst I've been working on it, I've found a common use case that seems far too verbose and that could benefit from a syntactical shortcut, I would propose that this:
{{<partial}}
{{$block1}}
Block 1 content
{{/block1}}
{{$block2}}
Block 2 content
{{/block2}}
{{/partial}}

...should also be able to be written as:
{{> partial block1="Block 1 content" block2="Block 2 content" }}

Much more concise for simple partials. This is actually very similar in syntax to handlebars but doesn't require any complex parsing as they are all just string literals.

@thelucid I'd strongly prefer not to get into parsing strings, or anything that looks at all like "arguments" to mustache tags :)

@thelucid
+1 for layout support
-1 for adding new syntax for a small use case that current syntax supports

@thelucid I'm all in favor of literals, especially for mustache implementations that have filters that go beyond pipes, as in {{ format(date, "yyyy-MM-dd") }}, or {{ repeat(separator, 80) }}.
I have another issue with your proposal: it's not clear if those strings are unescaped HTML or escaped text. In your example, they look like they are HTML since they are equivalent to explicit blocks (which contain unescaped HTML). But your intent is more likely to pass in escaped text. It's not clear.
Second issue: you talk about handlebars but this is actually very confusing. Handlebars arguments alter the context stack. In Handlebars, argument names define new keys, not block names, which are not the same at all at all at all. There are two "foo" in {{#foo}}...{{/foo}}...{{$foo}}...{{/foo}}`, one that triggers a context stack lookup, and one that is involved in inheritance. When you mix apples and oranges, a kitten dies! Again, this needs clarification.

Third issue: your idea is all good and shiny when you want to replace blocks with static text (assuming you intented text). But users love dynamic stuff. Bob asks:

Why doesn't {{> partial block1="{{content1}}" block2="{{content2}}" }} work?

What would be your answer?
Later, Alice asks:

Why doesn't {{> partial block1="{{> partial2 foo="{{bar}}" }}" }} work?

So... your short syntax may only cover a very narrow use case. Do you think it could be improved?

I'm fundamentally against anything that looks like function calls,
arguments, pipes, formatters, etc.
On Wed, Nov 4, 2015 at 7:29 AM, Gwendal Roué<EMAIL_ADDRESS>wrote:

Third issue: your idea is all good and shiny when you want to replace
blocks with static text (assuming you intented text). But users love
dynamic stuff. Bob asks:
Why doesn't {{> partial block1="{{content1}}" block2="{{content2}}" }}
work?
What would be your answer?
Later, Alice asks:
Why doesn't {{> partial block1="{{> partial2 foo="{{bar}}" }}" }} work?
So... your short syntax may only cover a very narrow use case. Do you
think it could be improved?
—
Reply to this email directly or view it on GitHub
https://github.com/mustache/spec/issues/38#issuecomment-153763375.

Why are you so negative? @thelucid may come up with something. He has already been a very helpful contributor to mustache. Even if your implementations do not use his ideas, others do. Both his "Safe views", and "dynamic partials" for example have been very inspiring to my own GRMustache, even if they are surely implemented not exactly as his original idea. If you don't even know what I'm talking about, check https://github.com/thelucid/tache, https://github.com/groue/GRMustache.swift#dynamic-partials.

Safe views (mustache.java has this) and dynamic partials (I think someone
is contributing this) are perfectly fine and don't fall into the logic trap
that the features I am referring to fall into. Basically, I don't want code
in my templates. As soon as you do things like I18N, share templates
between view languages / mustache implementations, etc. those features put
you in a really bad place. For example, that implementation supports
filters which allows moving code from the view class to the template. It
just isn't a good idea and a slippery slope to JSP/RHTML/Freemarker, etc. I
would put most of the Goodies in this category:
https://github.com/groue/GRMustache.swift/blob/master/Docs/Guides/goodies.md
So yes, there are some great ideas in your implementation and I'm happy to
implement the ones that don't move code into the template.
On Wed, Nov 4, 2015 at 9:12 AM, Gwendal Roué<EMAIL_ADDRESS>wrote:

Why are you so negative? @thelucid https://github.com/thelucid may come
up with something. He has already been a very helpful contributor to
mustache. Even if your implementations do not use his ideas, others do.
Both his "Safe views", and "dynamic partials" for example have been very
inspiring to my own GRMustache, even if they are surely implemented not
exactly as his original idea. If you don't even know what I'm talking
about, check https://github.com/thelucid/tache,
https://github.com/groue/GRMustache.swift#dynamic-partials.
—
Reply to this email directly or view it on GitHub
https://github.com/mustache/spec/issues/38#issuecomment-153795545.

All valid points, I was thinking aloud on a purely syntactical shortcut for cases when the block syntax is a little too verbose. I guess prefixing with a $ would clarify that the content was for the placeholders i.e.
{{ > partial $heading="Hello World" $body=content }}

The body assignment would be equivalent to $body="{{content}}" and would allow for easier partial reuse but agree that it all would need more thought.
I'm working with a client who manages their own backend and I'm just providing a set of Mustache templates and assets to be rendered. This clear separation turns out to be quite a good testbed for Mustache's flexibility.

That is exactly the conditions under which Mustache.java was developed. The
designer created the templates and via handlebar rendered them with mock
data. He then passed off the templates and the mock data to me and I filled
in the blanks with the real backend. Worked great.
On Thu, Nov 19, 2015 at 8:11 AM, Jamie Hill<EMAIL_ADDRESS>wrote:

All valid points, I was thinking aloud on a purely syntactical shortcut
for cases when the block syntax is a little too verbose. I guess prefixing
with a $ would clarify that the content was for the placeholders i.e.
{{ > partial $heading="Hello World" $body=content }}
The body assignment would be equivalent to $body="{{content}}" and would
allow for easier partial reuse but agree that it all would need more
thought.
I'm working with a client who manages their own backend and I'm just
providing a set of Mustache templates and assets to be rendered. This clear
separation turns out to be quite a good testbed for Mustache's flexibility.
—
Reply to this email directly or view it on GitHub
https://github.com/mustache/spec/issues/38#issuecomment-158103291.

@spullara Did you use helpers with Handlebars?

Sorry, naming confusion. Mustache.java has a small server that renders templates backed by static json files named handlebar. Should probably change the name now.

On Nov 21, 2015, at 7:59 AM, Jamie Hill<EMAIL_ADDRESS>wrote:
@spullara Did you use helpers with Handlebars?
—
Reply to this email directly or view it on GitHub.

@groue Based on your description, I've tried to write it as pseudo code to help my understanding... finding it really hard to implement in Tache. Would you mind telling me if I'm on the right track. My main problem is that it's hard for me to keep a global queue as my implementation makes use of template objects for each partial etc; as you will see here.
Render
  If <
    Enqueue
    Render partial
    Dequeue
  If $
    Iterate reversed partial queue
      If not been used yet
        resolver(content of partial override)
      If used
        Flag partial as used

resolver(content):
  For each element in content
    If $
      If name matches
        Flag this as overriden
        return block
    If <
      resolver(parent partial content) or resolver(inner content)
    If >
      resolver(partial content)

Hello @thelucid.

My main problem is that it's hard for me to keep a global queue as my implementation makes use of template objects for each partial etc.

I don't use a global queue, and I use objects for partials as well, so this should not be a problem.
When I compare the pseudo code below to https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L94-L99 I think it's OK. You use a queue, when I derive a temporary extended context, but I think it should eventually be the same, and that's why I used the word "queue" in my initial explanations. I hope I was not wrong.
Render
  If <
    Enqueue
    Render partial
    Dequeue

However the code below does not exactly look like the same algorithm as https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L87-L92, because I don't see that you resolve the $block before rendering it.
  If $
    Iterate reversed partial queue
      If not been used yet
        resolver(content of partial override)
      If used
        Flag partial as used

Your resolver function looks pretty similar to https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L251-L319, so it looks OK.
Maybe that the issue you are facing is that those functions should be pure functions, and not modify any shared state. Especially, the flags "block is overridden" and "partial is used" should remain local to each function.
This algorithm is very difficult to track mentally, I agree.
Maybe you can try to make it run in your head with a very simple case, and make your ruby code very verbose so that you can check that it performs as expected.
The Swift code contains tests for the complex cases (why must we test that a partial has not been used yet, for example: https://github.com/groue/GRMustache.swift/blob/Swift2/Mustache/Rendering/RenderingEngine.swift#L221-L233).

@groue I have a working Ruby implementation now (not pushed to Github yet). It passes all your tests except for those where partials and layout partials are place directly within blocks — this causes complications as mentioned by @bobthecow in https://github.com/mustache/spec/pull/75. I'm not sure the additional complexity and additional processing is worth it of the negligible benefit that you get from allowing '>' and '<' tags within '<' content.
@bobthecow where did you get to on allowing this?

Mustache.php does not allow partials as direct children of parent tags, for the reasons outlined in #75.

What kind of input can't your handle?

@bobthecow Out of interest, have you had any requests for that functionality or needed it in any of your projects to date? Just trying to determine if the additional processing and complexity is worth it, as I currently I have a very simple, clean and fast implementation.
@groue I'm not sure if you mean input on the topic, or input with regards the mustache template?

Sorry @thelucid, it's OK, I had a better re-reading of [#75](https://github.com/mustache/spec/pull/75.
OK, so... welcome to the club of the Mustache engines with template inheritance!

@thelucid No, I haven't needed nor had any requests for that.

Ok, so finally there is a Ruby Mustache implementation with support for layouts! Tache is actually quite a bit faster than the original implementation and just uses an AST vs. evaling generated code.
These are the tests it passes: https://github.com/thelucid/tache/blob/master/test/fixtures/tests/layouts.json (adapted from GRMustache, thanks @groue).

Well done, and thank you, @thelucid ! :-)

I have an issue. I am doing the inheritance
{{<../email_base}}
{{$email_title}}Your  Title here{{/email_title}}
{{$email_content}}
{{/email_content}}
{{/../email_base}}
The only thing is I have two separate folders and my email_base is outside that folder. When I run my project on Intellij I can see everything properly. But mvn fails
mvn test -DskipTests=false passes
mvn verify -DskipTests=false fails
giving me an error
com.github.mustachejava.MustacheNotFoundException: Template developer/../email_developer_base.html.mustache not found
P.S. Very new to mustache

Sounds like this is specific to mustache.java. I would post to the google
group:
https://groups.google.com/forum/#!forum/mustachejava
On Tue, Dec 22, 2015 at 5:31 PM, nkurothec<EMAIL_ADDRESS>wrote:

I have an issue. I am doing the inheritance
{{<../email_base}}
{{$email_title}}Your Title here{{/email_title}}
{{$email_content}}
{{/email_content}}
{{/../email_base}}
The only thing is I have two separate folders and my email_base is outside
that folder. When I run my project on Intellij I can see everything
properly. But mvn fails
mvn test -DskipTests=false passes
mvn verify -DskipTests=false fails
giving me an error
com.github.mustachejava.MustacheNotFoundException: Template
developer/../email_developer_base.html.mustache not found
P.S. Very new to mustache
—
Reply to this email directly or view it on GitHub
https://github.com/mustache/spec/issues/38#issuecomment-166773861.

Guys, how do you get around situations like this? In this case I want to omit an the 'id' block but have no way of doing it:
{{! partial.mustache }}
<section id="{{$id}}{{/id}}" class="content__block">
{{$content}}
  <p>Default content.</p>
{{/content}}
</section>

{{! index.mustache }}
{{<partial}}
{{$content}}
The content.
{{/content}}
{{/partial}}

I'm wondering if we should have a way to conditionally render content, only when a block is supplied (I'm not sure if the '?' is a good idea but not sure what other syntax would make sense), for example:
{{! partial.mustache }}
<section{{$id?}} id="{{$id}}{{/id}}"{{/id?}} class="content__block">
  ...
</section>

Any ideas?

@thelucid {{#renderId}}id="{{id}}"{{/renderId}}. If it depends on whether the id value is provided or not, {{#id}}id="{{.}}"{{/id}}.

@thelucid: why don't you just add a regular {{#conditional}} section around the block that you want to optionally render?

In this case I'm not specifying any data i.e. a front end guy who is only in charge of templates wants to use that layout partial and not specify an id... it should then somehow omit the id html attribute. There is currently no way to do this.

Seems like a common use case.

just include the id= in the the content.
<section {{$id}}{{/id}} class="content__block">

I kind of think of your example is an abuse of the idea of "layout".

@spullara I'm doing that as a workaround but it's not very DRY.
I think it's the perfect use case for layout patials i.e. when you have a chunk of markup that you want to reuse etc. Otherwise I'd just have to repeat all that markup every time meaning that if there were significant changes, they would need to be done with a project wide find/replace.

Hello, can we get a working TL;DR? Thanks

Implemented in #75 and #125.


qutebrowser: add support for list values in settings

Description
Add support for list values in qutebrowser settings.
Input:
settings.spellcheck.languages = [ "en-US" "sv-SE" ];

Output:
c.spellcheck.languages = ["en-US", "sv-SE"]

Checklist

[x] Code formatted with ./format.

[x] Code tested through nix-shell --pure tests -A run.all.
(Partly lie, I get an unrelated error
lieer-service: FAILED Expected home-files/.config/systemd/user/lieer-hm-example-com.service to exist but it was not found., also got this same error on master)

[x] Test cases updated/added. See example.

[x] Commit messages are formatted like
{component}: {description}

{long description}

See CONTRIBUTING for more information and recent commit messages for examples.

If this PR adds a new module

[ ] Added myself as module maintainer. See example.

[ ] Added myself and the module files to .github/CODEOWNERS.

Thanks! Rebased to master in 5b7b9821e0dd69d2538f67a48b0d7c641bcb0471.
The lieer error is a bit odd, I haven't seen that before and the CI system doesn't encounter it. Would you mind pasting the output of nix-shell -p nix-info --run "nix-info -m"?

Thank you. Yes, this is the output:

system: "x86_64-linux"
host os: Linux 5.5.15, NixOS, 20.09.git.39247f8d04c (Nightingale)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.3.3
channels(root): "nixos-18.09.2574.a7e559a5504"
channels(terje.larsen): ""
nixpkgs: /home/terje.larsen/src/github.com/terlar/nix-config/external/nixpkgs

I guess you can ignore the channels and nixpkgs since that is my local stuff, I am running a checkout of the latest nixos-unstable as you can see from the host. E.g commit 39247f8d04c04b3ee629a1f85aeedd582bf41cac
Let me know if I could do something else to help debug this. I was also trying to disable my overlays (by setting NIX_PATH to just nixpkgs=... but still failed with the same reason.


Need to decide whether to supply default values from constants

Is your feature request related to a problem? Please describe.
Before we introduced the new system for constants we wrote a lot of functions that defined default values imported from the relevant constants.py module, e.g.
def calculate_max_sorption_capacity(
    bulk_density: NDArray[np.float32],
    percent_clay: NDArray[np.float32],
    coef: MaxSorptionWithClay = MaxSorptionWithClay(),
) -> NDArray[np.float32]:

If constants are supplied like this then they will not change when the value of the constant is changed during simulation configuration. This introduces the risk of us forgetting to populate a constant in a function call, and then the default value getting used (in e.g. a sensitivity analysis) without the user knowing. On the other hand it's poor code style to not set defaults for functions with a large number of arguments.
Describe the solution you'd like
We could decide to avoid suppling default values for constants to functions, to avoid this risk.
Describe alternatives you've considered
We could still supply default constant values and just be careful to never invoke functions without all constants specified in the main function flow.
Alternatively, there might be a way to populate these default values from the constants registry, which would avoid the problem entirely.
I think this is something we should discuss as a group so we can agree upon a common strategy, and then document accordingly.

Closing - see #364


Join tackle-controls and tackle-application-inventory

This PR is done to start the discussion of whether or not we should/can join tackle-controls and tackle-application-inventory as an alternative solution for https://github.com/konveyor/tackle-ui/pull/242 .
The current PR has migrated all the code, including tests, from tackle-controls into this repository.

Closing this PR as this was just a prototype.


Cherrypick Fix nightly large_vector test caused by incorrect with_seed path (#18178)

@ciyongch @leezu
This unblocks the v1.7.x nightly test for binaries pipeline.

@mxnet-bot run ci [unix-gpu]

Hangs after np_empty. Known issue.
Retriggering @mxnet-bot run ci [unix-gpu]

Yet another hang at np empty test.
@mxnet-bot run ci [unix-gpu]

@mxnet-label-bot add [pr-awaiting-merge, pr-awaiting-review]

@ChaiBapchya any reason this wasnt committed to 1.x branch?

Missed it. Thanks for catching.


Protocol [x] is unavailable

About You:
Name: Abdullah Elabasery
Company: Incorta
random error saw in the tdvt result  is protocol [x] is unavailable where x is the protocol number , so what does this error mean ?

Hi @AbdallaElabasery,  can you share with us the log files where this error was seen? The files should contain names like jprotocolserver or stdout_jprotocolserver and this article should be able to help you locate them. There are a few potential causes that can lead to this error and we will need the information from the logs to further identify.

Hi @AbdallaElabasery, other than the page I linked above for log collection. It would greatly help us investigate if you can follow this page to share a few other artifacts with us. Thanks!

Hi @AbdallaElabasery, just wanted to follow up and see if there might be any update on your end?

Hi @AbdallaElabasery, just a kind note that I'll be closing this issue tomorrow if we haven't heard anything by then. If you have any further update feel free to comment on the issue and we can certainly look into it again. Thanks!

Closing as no logs received.


Implement msg.data

What is wrong?
We do not yet implement reading msg.data
https://github.com/ethereum/fe/blob/177ee35ba4b12150f5cb14152e066d364b1a0161/compiler/src/yul/mappers/expressions.rs#L438
How can it be fixed
Start here:
https://github.com/ethereum/fe/blob/177ee35ba4b12150f5cb14152e066d364b1a0161/compiler/src/yul/mappers/expressions.rs#L438
Background:
https://docs.soliditylang.org/en/develop/units-and-global-variables.html#block-and-transaction-properties

I can take this.

Happy to review a PR!

Hi @georgeroman, thank you very much for volunteering.
There's a bit of a holdup with this issue right now, since we do not support dynamically sized bytes (see #280). Without support for dynamically sized bytes, I'm not quite sure how this will be implemented.
There's some open discussion about bytes support, which you'll see if you follow the linked issue, but I think we can get away with handling bytes the same way we handle strings for the time being.
If you're still interested in taking this on, let me know. Otherwise, we could find something else for you.

Thank you for letting me know! In this case, I'll take https://github.com/ethereum/fe/issues/278 as it seems easier to start with. However, I would still be interested in helping out with this one.

Great! That one is alright.

Just citing from #280 that there might be a better way to read msg.data using a special cbytes type that could have read or slice APIs to read slices of the calldata into a fixed size array.
https://github.com/ethereum/fe/issues/280#issuecomment-799901764

How about something like a decode_calldata function, which is generic in its return type?
impl Msg:
  pub fn decode_calldata<T>(offset: u256 = 0) -> T:
    // magic

(x, y, z): (Array<u8, 10>, u256, address) = Msg::decode_calldata(offset: 32)
// x is Array<u8, 10, Location=Calldata> or Calldata<Array<u8, 10>>

// Get a pointer to some raw bytes (This is a bit weird, I admit. Maybe getting a pointer to raw bytes should be a different fn)
x: Array<u8, 128, Encoding=Static> = Msg::decode_calldata()


Restore files and directories times (access and modification times).

This pull request depends on #275.
As the syscalls to restore a symlink modification times are not available (os.Chtime changes the linked file times) I used the github.com/docker/pkg/system that contains this functions and many other that can be very useful and tests (for example makeblockdev can be replaced with system.Mknod and used for various os).

@sgotti I'm hesitant to drag in this huge pile of code until it's clearer that we absolutely need it. Alternatively if this functionality is available in libcontainer, I think that would make more sense as a dependency.
/cc @vbatts for any ideas

@jonboulle The needed functions are system.UtimesNano and system.LUtimesNano
In future I see that probably that also   ```system.Lsetxattrvarious extended attributes functions should be needed
I don't see docker/pkg/system imported by libcontainer.
Would you prefer if I copy only these needed functions inside pkt/tar or in another pkg under rocket (needs a special copyright note?)?

@jonboulle @vbatts I think we should also choose if to follow the cgo track or the syscall track. I personally prefer the syscall track (like docker/pkg/system and libcontainer does) as it should avoid possible cgo problems and it will be clearer to implement it on multiple architectures

I am two ways about adding those deps. We've evolved that system library was grown that way for cleaner abstraction. And some of this feels like the kind of copied around code, similar to why boost became such a defacto for C++. Maintaining support for both cgo and gccgo is important to me.
I would say, evolve in baby steps, and as needed, keep it simple. and attempt to not make api limiting decisions. Perhaps pulling all that in, is not a requirement right now.

@vbatts thanks. So If I understand it correctly will you prefer to add just the needed functions inside pkg/tar?
These funcs are different between Linux and other unixes as there isn't a posix func to change symlink times (see docker/pkg/system/: utimes_unsupported.go, utimes_linux.go, utimes_freebsd.go, utimes_darwin.go). Should they just be copied? If so, all or just only the linux and unsupported parts and the other in future?
Or you'll prefer a cgo solution (using utimensat) just now for linux instead of the syscall solution?

I don't have a hard stance regarding this PR, I am saying make baby steps as needed. Sometimes making a _posix.go consolidation makes sense, but sometimes you will need _linux.go, _freebsd.go, etc.
For this PR perhaps pulling in all this may be too much.

@vbatts @jonboulle I updated the PR with your suggestions. I just copied the needed bits from docker/pkg/system. I really think that the ending solution will be that https://github.com/golang/sys/ will extended it's implementation of utimensat to accept also the flags argument and then add a LUtimesNano function (I'll try to create a PR for this).
The symlink times are restored only if the underlying platform supports it. So I'm using the same pattern used by docker/pkg/system: the function LUtimesNano is implemented on linux (utime_linux.go) while for the other platforms a utime_unsupported.go is used returning and ErrNotSupportedPlatform that will be ignored by ExtractTar and ExtractFile. Additional implementations of LUtimesNano for bsd and darwin can be added in future (or just now, let me know).
docker/pkg/system also uses the same pattern with UtimesNano but, as it's used on all systems by os.Chtime I directly used it without the unsupported pattern like with LUtimesNano. I'm not directly using os.Chtime as it needs to be wrapped by a function that does something like timeToTimespec as if the tar header's times are 0 the call will fail (as time.UnixNano() gives undefined results with a zero time).

much more concise.
Also, while I like golang.org/x/sys, vendoring it would add 3Mb, so we can hold on that for a minute, I'm sure.

LGTM, could you please rebase so we can merge this? thanks!

Ping

@iaguis @jonboulle Rebased.

LGTM, thanks a lot.


Fix test regression with enable-fips and enable-weak-ssl-ciphers

This is rather a proof of concept than a real fix. I assume the real fix would be to add a gettable param to the 3DES in FIPS_MODULE that would indicate that it is a decrypt-only cipher.

@slontis @paulidale - we need to do something with this.

On a related note should we really be cross testing a PR against master, that seems like a strange combination.
Which seems equivalent to testing an older branch with the newest provider on that branch (Not sure why you would do that).

Also what is with these cross testing PR against branch results. Were they just recently added? There have been multiple times where issues are not seen until the next merged PR after the PR that caused the issues...

The cross testing has been around for a while.  It's been update recently and made easier to run against PRs.

@nhorman please reconfirm

Ack, approval holds

An alternative to this would be using a fipsmodule.cnf that allows tdes encrypt?

No, not really. We really should handle this case gracefully in libssl and not use a cipher that is unusable.

@mattcaswell please re-review

This pull request is ready to merge

Merged to the master branch. Thank you for the reviews.


Lavague gets stuck in endless loops for an action

Describe the bug
When we use the Getting started Colab notebook, and run the Gradio demo, then Lavague goes in endless loops for clicks.
To Reproduce
Steps to reproduce the behavior:

Run the Colab notebook
Run the below cell : `

driver = SeleniumDriver(headless=True, no_load_strategy=True)
action_engine = ActionEngine(driver)
world_model = WorldModel()
agent = WebAgent(world_model, action_engine)
agent.get("https://huggingface.co/docs")
agent.demo("")`

In the Gradio notebook, enter "Click on the Models link"
Expected behavior
Models link should be clicked ONLY ONCE. Here Lavague goes in endless of trying to click it thrice and more.

Screenshots

I think it might be because it has difficulty understanding it finished the task. Maybe we should add the URL as the list of observed data to provide higher signal about where it landed

Yes, we should update the Action engine and capture the URL, so that it doesn't go into a "hallucination" well.

@dhuynh95 - I think we already have the URL in the observed data?
obs = {
            "html": html,
            "screenshots_path": str(current_screenshot_folder),
            "url": url,
            "date": datetime.now().isoformat(),
        }

This is now fixed so I will close this @smilinrobin - thanks for flagging this


[6.0][concurrency] Mark the global _emptyDequeStorage as nonisolated(unsafe).

Explanation: This is just an internal global that is only written to during initialization. I validated together with @ktoso that there aren't any write uses to it. This just turns off a warning in the concurrency library due to strict concurrency.

rdar://130171701

Original PRs:

https://github.com/apple/swift/pull/74149

Risk: Low.
Testing: N/A.
Reviewer: @ktoso

@swift-ci test

@swift-ci test macOS

Ci had an outage, restarted mac CI


missing alpine linux dependency py-pillow

Hi Vladimir,
can you please add the package py-pillow to your image?
Without this package trying to add an avatar to an account, seafile crashes with the following message:
Page unavailable
Sorry, but the requested page is unavailable due to a server hicup.
Our engineers have been notified, so check back later.

I was able to resolve this issue running apk add python2-pillow.

Ouch, thanks for report, I'll fix it in couple of minutes

Fixed with commit 30de55f077cc29c1be3ef331eb70df1c44d894ca

thank you! 👍


The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

Hello Guys. I got this infinitive warning in the console in Google chrome. I'm using Nuxt.

I have the same problem, do you solve it?@derakhshanfar


Allow items to consume their "Quantity" without being considered ammunition. (Javelins, grenades, etc.)

Currently there is no way to create a consumable weapon such as a javelin or grenade. There are many situations where a player may have a small collection of single-use weapons such as these, but there is currently no native way to create such an item.
It is possible to simulate this behavior by giving a weapon "charges" but this does not affect the quantity (and therefore weight) of the item being used. The presumed way of creating a consumable weapon would be by creating a Consumable with itself as the ammunition, but this causes the strange behavior of rolling the damage formula twice, since the system seems to consider the damage roll of a piece of ammunition to be included with the item that fired it (like a bow and arrow).
Currently the best way around this seems to be to create a weapon called "Throw" and set various consumables as its ammunition, but this is clunky and not ideal, requiring players to open up the Details of the item and change the ammo themselves.
Suggested feature: EITHER add "Item Quantity" to the Resource Consumption dropdown for items/weapons/etc. that simply decrements the quantity of the item its assigned to, without any additional functionality OR add "Thrown" to the Consumable Type dropdown with similar behavior.

I struggled with this somewhat myself, but found out how to do it based on the way other consumables in the SRD compendium work. It's a bit unintuitive, but it is very possible to make an item like this work, similar to the way Potions of Healing work in the SRD:
How to make a single use consumable that depletes itself on use (like a grenade):

Create your consumable (Optional: Set the Consumable Type to trinket or whatever best fits your vision)
Set your desired Activation Cost
Under Limited Uses, select Charges, and set the desired number of charges (usually 1 for a single-use consumable)
Check the box that says Destroy on Empty.

When you use the item, if Uses Prompt is checked, it will allow you to expend a use of the item. When the charges are all used up, it will destroy one item from the stack (freeing up weight) and refill the charges with the next item in the stack. On the last usage of the item, it will warn you that it's the last one before using it, and will delete the item from your inventory once it's used. (Make sure to keep a backup of the item sheet in your Items page or a Compendium.)

Seems like "Resource Consumption" should support "Self" as an option, as a more idiomatic way to configure weapons. Or just always include the weapon itself as an Ammunition option.

I struggled with this somewhat myself, but found out how to do it based on the way other consumables in the SRD compendium work. It's a bit unintuitive, but it is very possible to make an item like this work, similar to the way Potions of Healing work in the SRD:
How to make a single use consumable that depletes itself on use (like a grenade):

That is wacky, but does work! Thank you!
Though I do think a more of a self-evident way to configure thrown expendable weapons is called for.

Is a more self-evident way of doing this being considered?


Question about the Windows Home version and the use of rdpwrap

My windows is Windows Home Version, which has no remote desktop function, just like the screenshot:

But before I installed the rapwarp, when I search the "Remote Desktop" in the windows search box (or Cortana), I can also find the software:

I wonder to know why my home version windows still has the "Remote Desktop" software?
After I installed the rdpwarp, how do I use it?
Should I still use this software shown above? If so, I don't have to install the rdpwarp, isn't it?
Thanks a lot for your answers.

You must install the Rdpwrap in order to have ability to use multiple connections to your PC.  Please scroll down here https://github.com/stascorp/rdpwrap and read what it is, and how it works.


AOT Reflection issue when compiling against .NET 4.x runtime in Unity 2018.1

Hello! I've been using your excellent framework extensively for our UI, but I've hit a snag.
It looks like reflection methods are more restrictive under the .NET 4.x scripting backend than it was with 3.x, and I'm seeing the following when running on iOS having compiled under 4.x in Unity 2018.1.
[Crashlytics] Recording exception: ExecutionEngineException: Attempting to call method 'System.Reflection.MonoProperty::GetterAdapterFrame' for which no ahead of time (AOT) code was generated.
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
Fabric.Internal.Runtime.Utils:Log(String, String)
Fabric.Internal.Crashlytics.CrashlyticsInit:HandleLog(String, String, LogType)
UnityEngine.LogCallback:Invoke(String, String, LogType)
UnityEngine.Application:CallLogCallback(String, String, LogType, Boolean)
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityWeld.Binding.Internal.PropertySync:SyncFromSource()
UnityWeld.Binding.<>c__DisplayClass17_0:<Connect>b__0()
System.Action:Invoke()
UnityWeld.Binding.Internal.PropertyWatcher:propertyOwner_PropertyChanged(Object, PropertyChangedEventArgs)
System.ComponentModel.PropertyChangedEventHandler:Invoke(Object, PropertyChangedEventArgs)
MyCustomViewModel:OnPropertyChanged(String)

Actually, having looked around, it might be as a result of this known issue with Unity 2018.1.
https://issuetracker.unity3d.com/issues/uwp-executionengineexception-is-thrown-on-system-dot-reflection-dot-monoproperty-getteradapterframe

Yeah, it's an issue with Unity 2018 ): ... Hopefully it is fixed soon

I'll close this and keep an eye on patch releases