Allow for preflight requests with OPTIONS method

Links to comments explaining the CORS issue

https://github.com/Expensify/App/issues/14227#issuecomment-1406800255
https://github.com/Expensify/App/issues/14227#issuecomment-1410089720

Summary
We are removing the auth token from the URL part of an attachment/image and instead placing it in a HTTP header called X-Chat-Attachment-Token
We're doing this to improve file caching (when the token is part of the URL, the URL changes often and cannot be cached)
As a consequence (for adding this header) browsers start to make preflight (OPTIONS method) requests
(This also applies to Desktop since under the hoods it's using Chromium)
The preflight requests are being made because the API url (we request the image from) is from a different origin
than the place where webapp is hosted
e.g. website is new.expensify.com while API is www.expensify.com
Right now the OPTIONS request, seems unexpected and the API redirects with a response to an error page and we get the following error
Web:
[Error] Preflight response is not successful. Status code: 302
[Error] Fetch API cannot load https://www.expensify.com/chat-attachments/4198796077944938521/w_6f21f50b3af2ee93c4e6e288ba5b986858166367.jpg.1024.jpg due to access control checks.
[Error] Failed to load resource: Preflight response is not successful. Status code: 302 (w_6f21f50b3af2ee93c4e6e288ba5b986858166367.jpg.1024.jpg, line 0)
[Error] Preflight response is not successful. Status code: 302

Desktop:
Access to fetch at 'https://www.expensify.com/chat-attachments/962115596/w_e989bfe450cb8e872f52dac2d082c391f35e2e0a.jpg.1024.jpg' from origin 'http://localhost:8081'
has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.

What can we do
I think this needs to be planned internally
We can start a conversation in Slack to get a general consensus on how to deal with the CORS issue
Perhaps the simplest thing would be to update the backend to handle cross origin request coming from these origins

localhost
desktop app
staging url
prod url

Typically servers support a CORS configuration that can specify to allow and properly respond to request (including preflight) from these origin
Alternatively we can avoid CORS altogether if the API is resolved from the same origin where the app is hosted
this can be a simple proxy that takes requests coming to new.expensify.com/api and resolves them from www.expensify.com/api
Perhaps more alternative exist and someone can propose something better

cc @kidroca

Triggered auto assignment to @johnmlee101 (ring0), see https://stackoverflow.com/c/expensify/questions/6102 for more details.

👋 I have created this ring0 issue since we will have to update the way we handle requests with attachments so we dont run into CORS issues. I dont have much knowledge on how this can be updated in our backend, I think we need to allow for the OPTIONS request with the X-Chat-Attachment-Token in headers but dont know much about how to execute it.
Happy to learn and help you here! Making this issue Weekly as its not a blocker right now but we have some PRs which will need to be on hold until this will be enables in backend.
Thanks! Also @kidroca is probably best to explain the issue if there is anything not clear from the OP.

Can you detail the pipeline for the request ideally and what is currently happening now?
Kind of like
Source: localhost
GET
Url: www.expensify.com/somepath
Headers:
    xxxx: xxxx
Body:
    xxx:xxx:

And where is it being rejected?
Is it the fact its making an expensify.com request from a new.expensify.com source?

This PR changes the way we load attachments using header for web/desktop and can be used to recreate the CORS issue

https://github.com/Expensify/App/pull/13036

You can easily recreate the problem from the Desktop app
Recreating the problem for web is a bit more tricky as you have to host or simulate hosting the app on a staging URL
The requests against the API are still considered CORS because the staging app is hosted on staging.new.expensify.com while the API the requests are targeting is staging.expensify.com
Here's what I did to resolve the hostname new.expensify.com from my local server: https://github.com/Expensify/App/pull/13036#issuecomment-1406733537
(Thought this only works in Safari)
To see how the backend reacts to an OPTIONS request (like the one used in preflight) right now you can make the following request
curl -X OPTIONS \
https://www.expensify.com/chat-attachments/4198796077944938521/w_6f21f50b3af2ee93c4e6e288ba5b986858166367.jpg.1024.jpg \
--dump-header -

@johnmlee101

Can you detail the pipeline for the request ideally and what is currently happening now?

Origin: new.expensify.com
Method: GET
URL: www.expensify.com/chat-attachments/{path}.jpg
Headers:
  - X-Chat-Attachment-Token: {TOKEN_VALUE}

Due to the difference in Origin and the target URL (and the presence of a header) browsers initiate a preflight request like:
Origin: new.expensify.com
Method: OPTIONS
URL: www.expensify.com/chat-attachments/{path}.jpg
Headers:
  - access-control-request-headers: x-chat-attachment-token
  - access-control-request-method: GET
  - origin: http://new.expensify.com
  - referer: http://new.expensify.com/

It seems the backend doesn't expect an OPTIONS request for /chat-attachments/{path}.jpg and responds with a redirect to an error page
RESPONSE
Status Code: 302
RESPONSE HEADERS
  - content-type: text/html; charset=UTF-8
  - server: cloudflare
  - location: https://www.expensify.com?exitTo=%2Fchat-attachments%2F4613573186077378841%2Fw_8c1e154469ef960408a0d7d2d6ce682fbe4e189d.jpg.1024.jpg

The browser considers this a non OK response and doesn't proceed with the initial GET request

Is it the fact its making an expensify.com request from a new.expensify.com source?

Yes that's the reason access control checks (CORS) are being made - www.expensify.com and new.expensify.com are not the same origin and requests between them are subject to CORS

Ah cool I see what you mean. So the issue is that specific path doesn't support this bridge.

Okay a few things:
We currently set the CORS policy for known hosts like new.expensify.com and staging.new.expensify.com, not localhost so that's why we're hitting that.
So when I run an OPTIONS network request from staging.new.expensify.com from the console I get
access-control-allow-origin: https://staging.new.expensify.com in the response, which should be expected. However since I'm not passing a proper authToken its doing a 302 redirect to www.expensify.com which responds in CORS (as expected).
Are you suggesting we implement full OPTIONS support with our resources to determine if CORS is allowed with certain requests? We don't currently support that in any capacity and would require more work to implement.

We currently set the CORS policy for known hosts like new.expensify.com and staging.new.expensify.com, not localhost so that's why we're hitting that.

I'm not making the request from localhost
Here's an example from the browser console (on https://new.expensify.com/)

If we call fetch with a header, an OPTIONS request is being made (and stuff breaks)
fetch('https://www.expensify.com/chat-attachments/5307596164054552186/w_32652718d4f32c75792c468e94404d4eeb603bb6.pdf', {
  headers: {
    'X-Chat-Attachment-Token': 'aub12umOOqee9bbyQcLFXqVgyizNIT%2FRe2CGqgQ%2Bpu58NFMZ%2Faba3POWx9Bv04gApkd3aBnkoQowQ54iIDcqQu9szKyQA%2BXZpz4vkPLmYl%2FmYb2lEUAvRzzNuWPNGBOEsBTxWa7%2Bn5Dkk9J7JxzEgdgyegbiT1qrlbgjnkWKmYVgV9PLf4jKOcg%2FVMRqTA5h3kvA2CKgsoHVrc%2Ba2IiLzh6%2F%2FLPE1LVRVrGF5ar3QKZXb1jviLOq%2BwdVUOQJiaEWeKMatoxUPoa6RiRnp0hfcsTN2XGUEwl6iPq9OfbKtq1bx5DtToD%2FjMXqCok%2FGEsV5k%2F81pickHzmT%2BgokWSwvemtE%2FdTalz8ghVTQmm616LLTsRJBFh4liD6bjn3vd5qHJrlgKZHlGWeQO54AKYVdDKOuuZPKmuWEjUMyOmY8uyl%2Fr1%2FY2J5Z8%2FHGUWfzfwtCNAYE6c3infaT7YUo0sHHmoDSSIVciSDWo6TJwPFAQTCbIKRnYEk3Z6rkZ9NdjryQQSqgcS6wNXu4hrsrmls41ds7cB49DCwknlweCIxXTnB2wUNY46dw7l84HfFqwpps3pi6W7sQKkbfu5euUPXoRijWkBnLPjvVJoARoi9ewCB70jBvoq4hyfjEgOFFL1GvMopAg1MeMqOPIKZW%2F7Kx3mYG51alGJ9v01hfbz%2Boaq%2BDTJuRsfiXqGRfA3g2QARCG1LhfZMIpBIRu1PmRGYuzxOycA%2FAIiY2BC1O6WHlhLE1Ix6FVZXyuK41cIpOJBBzwQdHgMIvfJ5M%2BTtOTcFwEhQ54rRWZRjF1c%2BSZe2u0av%2FNqCnb2M3pabMjQPtE7%2BtyJ3%2FxewZBBATlSg8QBkdkKIkj7J1c8ylrR30h0A%2FtH8FMp1K2x%2FekksCICfiGD5XQbCfeFOBGgGbF5MAGQhQCzRk4ivfZvCRknoT5G%2FyoZSu0KUA9ETx%2BYXsqFNhSPr%2FGD48umfYsONz3rVa8X8lQ%3D%3D'
  }
})

If we instead call fetch without a header (the old way)
fetch('https://www.expensify.com/chat-attachments/5307596164054552186/w_32652718d4f32c75792c468e94404d4eeb603bb6.jpg.1024.jpg?encryptedAuthToken=AP8vz1B3TKytIYZ0BcuegJCDyD68Iqlf1jY3NHHp296%2B4mAPvRCW1MpRmlmnkXnWN9q4cxwYkJN%2FTJ%2FBdmqVQDxhetkoMOvqgs6txk6AzGVRmraXcQLpSoyAgyQ%2FV0nsX0%2BLKZdSuN%2BjX%2Bn8EkBJAi0wcuXKa9ZH0LNoAmASbook9PdCn7LJS2Gx5CoPogB4Y6%2BgCFcvMvgYgFIH18d0UoosIqG8IAhAW%2BABj20K4JBBR59G8%2FhKFyrefFKyoD%2BMcc47wJKNsa4wkkapQH1bsLpf%2Bro4pORXlQtjYtyX2vHxvPXtA%2B6lRqU9pX10ekQ1nkwpKOmYSyLhd%2FB%2BzV2BDGKbonpfOMDp4dT93wJy7FC%2BPxersm4BKnkiv8v0bTsa3muDOJNcRffQ8Wf%2FesJECLkYJhLaC2Uajf4EXhKsehLTrSrsaikbxiv9mihaMVLopIHaVUWm5dmrQAh%2BeHe%2Bu7gv8OcqAvmsZ0f4xZLqPTVBTjgT8PUZfU3FHbEAvfMz66hIuFNyllK986GquSapVEmqeCLGP3k7Syd4118CMqXZHxJIwKS4IAxGQJXcn4VoeR5e3plxDk%2BuhThR4GRX5V4WVoGaM%2B4xFlICSPcHThZ5KxDkRGkNUbpbTRWvzEeyc6Gt%2FKz1sqGr1S9T7uicfmuUNiDX5br9yE20ETQ2ERylwQyAFp96PV0O9taHfqn1frYvOSB%2Bx%2BPtPWFUNOoYKTw%2FAjhDclgc5o1jvmWiPk3Qc1LNfQk6lPXLsS8FbweeFAnJx%2BK4CRXWa8ynCdA7MoIOmQMwoX%2FRHd3A14zzpDmvhALZnz5YBGNUTwCRrevsECwNGDxcWDKsGglMzhL4Yd%2FmBeFUjFgJYVS0w0juJiSxVHHe5hQeSo7k9nzQKBHSZ7F60BA2DSsPZvFnpnldPA3XfA6gogWS2B%2FseJU%2BvqrERUsRhB0bZoqoFiaNy2AC%2FnsYNLaV0Gut2Sum6FW%2FxQ%3D%3D')

It's considered a SIMPLE CORS request - no preflight request is being made. The browser uses the GET response and searches for access-control-allow-origin on the response headers satisfying the CORS requirement
The backend responds correctly for this case
But when a header is involved, browsers first make OPTIONS request to ask the backend whether they can use x-chat-attachment-token in a follow-up CORS request
(access-control-request-headers: x-chat-attachment-token)
The backend (currently) responds with a redirect to an HTML page to the OPTIONS request
IMO OPTIONS requests cannot be answered with a redirect to a page - we're not responding to a human that can interpret the page, but to a machine that needs to know whether it can use the requested options or not

So when I run an OPTIONS network request from staging.new.expensify.com from the console I get
access-control-allow-origin: https://staging.new.expensify.com in the response, which should be expected. However since I'm not passing a proper authToken its doing a 302 redirect to www.expensify.com which responds in CORS (as expected).

Even if you use a valid token the OPTIONS request results in a redirect (add method: 'OPTIONS' to the above code sample), but browsers would not add the custom header (and it's value) to the OPTIONS request anyway - if we send the header it defeats the purpose of the options request
Browser uses OPTIONS to ask whether it can use a certain header(s)  to make a GET request to for a certain resource

Are you suggesting we implement full OPTIONS support with our resources to determine if CORS is allowed with certain requests? We don't currently support that in any capacity and would require more work to implement.

Only for the resources that expect/allow requests from different origins
Usually CORS is configured on the server setup level and it's relatively easy to manage
But if the backend setup does not allow to change this easily you can bypass CORS altogether by making the API accessible from the same origin - the proxy idea
request to the API are made against the same origin, but a proxy server resolves them from the API. We do a similar thing for localhost development where a proxy server runs and channels requests

Gotcha, I've never done non-simple requests so that makes sense. And this is all coming from the need to add the custom header for caching.

When you call fetch manually it doesn't include your own authToken in the standard cookie request, right?

When you call fetch manually it doesn't include your own authToken in the standard cookie request, right?

IDK maybe it doesn't include cookies for CORS requests as a precaution
AFAIK you can tell fetch to send or not send cookies, but you can't select which ones

BTW the proxy idea is not about bypassing CORS, but to make requests use the same origins without CORS restrictions or the overhead of a preflight request
If the request is not coming from an allowed origin the proxy can reject it, or redirect it to the original API (and let it get a CORS error)

The backend (currently) responds with a redirect to an HTML page to the OPTIONS request

This happens because no valid authToken is passed with the request btw

So if we can somehow pass the authToken in the right parameter we expect (plus other base necessary cookies) I feel like we could simulate a request properly and the preflight would just work, since the redirect is basically seeing that the user is making an unauthenticated request outright. Alternatively we could try and fetch the x-chat-attachment-token and do some replace on the api side to then use that as the token itself

But in that case we'll still want to send the accountID and email for the account its associated with. There's no hard rules in our backend preventing this from happening, but it just doesn't like that this is an unauthenticated request without the expected token

The backend (currently) responds with a redirect to an HTML page to the OPTIONS request

This happens because no valid authToken is passed with the request btw

Browsers would not add body or headers to the OPTIONS request being made
They would not add the x-chat-attachment-token value
We don't have any control over the OPTIONS (preflight) request, can't alter it in any way
Preflight requests simply ask the server "what are the options for this resource, can I use this header"
Auth state shouldn't matter for the answer to this question
E.g. the OPTIONS request is supposed to be safe, and not include sensitive data, passing the token value as part of the request is a violation of that spec
account IDs and any other information can be passed over the actual request, once the browser is satisfied by the OPTIONS response

Cool, seeing if this is something we should just handle in nginx or in our backend. Exploring that now, appreciate you being patient with my questions!

From what I know about CORS server configuration
Regardless of the underlying architecture or language
CORS is set up on a config / initialization level - it's not coded for each controller/route/resource individually
Usually a library or a framework solution exists where you specify a list of allowed origins and a list of allowed headers
you can be more granular to specify a mapping of resources/urls and allowed options

@Beamanator @thomas-coldwell
How locked in are we with the custom header route for this? The more I investigate the more we'll have to create a custom route/ruleset for this endpoint on our OldDot API and I want to be super certain this is the route we want to take before putting extra effort into this. If we could use simple requests it would simplify this all immensely.

Oof I mean this was the only solution we came up with for caching chat attachment images... I'm not sure how we'd work around this unless we scrap the caching idea 😅

Oofda probably in some old slack thread - there were some image enhancements predesigns on the topic, I'm done working though so won't be near my computer for the night sryy

Have we looked into CloudFlare caching for these assets? We can extract certain params from the URL to create the caching keys

https://developers.cloudflare.com/cache/ we have all of the enterprise features

Have we looked into CloudFlare caching for these assets? We can extract certain params from the URL to create the caching keys

Current logic adds the token to the URL (as query param /chat-attachments/{path}?encryptedAuthToken={token})
The token expires frequently (e.g. every 2 hours) and this create a new URL for the attachment invalidating cache
No CloudFlare caching would help in this case

How locked in are we with the custom header route for this?

There are alternatives, though I think the header idea is optimal
Moving the token to a header keeps the URL stable and allows for built in caching to work (on all platforms not just browsers)
Alternatively we can create custom caching where if we have cached value for /chat-attachments/{path} we don't make a request for /chat-attachments/{path}?encryptedAuthToken={token}
If we want to drop the header idea entirely and go with custom cache, each platform might have to have a custom implementation (at least web vs native)
We can build custom cache only for web/desktop, and leave the rest of the platforms use headers, yet we have to manually

invalidate cache
evict cache
maintain custom logic

Stuff that comes out of the box for the built in cache (for stable URLs)

Sorry for the delay here, I think the scope of this project is larger than originally intended, and I think we should do some pre-design and create a doc on how we want to approach this. There's multiple solutions IMO and each one has trade-offs. Since this was originally Ring0 assigned for me, so going to unassign. I'm happy to hop on a call to discuss the difficulties in our system for this, and the architecture required on our end.

Job added to Upwork: https://www.upwork.com/jobs/~01a085d801f68f616f

Triggered auto assignment to Contributor Plus for review of internal employee PR - @mollfpr (Internal)

Looking for someone to take over 🙏

Some questions we need answers to:
Why is the API on a different origin than NewDot?
Can we use a proxy for routing so NewDot & the API are on strictly the same origin?
Local proxy (for dev) was created for that problem

@johnmlee101 Do you think we can talk about a few of ^ these questions in person in Iceland next week?

For sure! 👍

Maybe it makes sense to HOLD this until we move NewDot web hosting to our servers

Yeah I think that does make sense @roryabraham - I'm pinging @justinpersaud as well about this since he's very involved in that change

Still on hold

Still on hold

Updated the title to reference the issue number in E/E.

still on hold, making monthly

Still on hold

still

Still on hold

still on hold

Is this still important to do? The migration project has been de prioritized lately. If it is, happy to help come up with an alternate solution so we aren't waiting for a long time.

It would be great to finish the image improvements once and for all, the remainder of which as I understand it is blocked on image caching as the source of the problems left to fix from the OP here.
If you have an alternative to the migration project to unblock that it sounds good to me, though I'm sure others on this issue might have something to share on where we got to with exploring alts before.

There's a lot to read through. Maybe someone can summarize? It seems like we just aren't supporting OPTIONS method at the chat attachments path, or is there more to it? How does moving NewDot to our own environment help this? My gut has me thinking we can solve this via Cloudflare request transformations or our worker that we have setup?
If someone wants to catch me up here I can probably come up with a solution.

@kidroca & @johnmlee101 maybe?

Bump guys to maybe furnish Justin with a summary on why we ended up pausing to tackle the server migration project first!

Hey @trjExpensify, everybody
Here's a summary of our discussion so far:
Problem: The domain new.expensify.com is attempting to fetch chat attachments from www.expensify.com but encounters CORS (Cross-Origin Resource Sharing) issues.
Details:

Our goal is to shift attachment authentication to a custom header, eliminating the need to append a token to the URL. This change allows for more efficient caching since the URL remains consistent.
However, when new.expensify.com initiates a GET request to www.expensify.com for chat attachments, browsers trigger a preflight OPTIONS request. This is due to differing origins and the introduction of the custom header.
The server's reaction to the OPTIONS request is a 302 redirect. Since the browser sees this as a non-OK response, the initial GET request is halted.
To elaborate, the server/attachments endpoint isn't set up to handle the preflight OPTIONS request. Instead, it perceives it as an error and redirects to an error page.
Browsers differentiate between www.expensify.com and new.expensify.com, which results in the CORS (OPTIONS) checks.

Potential Solutions:

CORS Config: Servers or routes usually allow for basic CORS configurations in place. This supports the utilization of specific headers, origins, and other CORS-related elements.
Proxy Server: We could establish a straightforward proxy server that presents both the App and the API from an identical origin. When both entities operate from the same origin, CORS are notbeing made.
Custom Caching: We could develop a unique caching method, potentially necessitating distinct implementations across platforms (e.g., web vs. native). This method poses challenges like manual cache invalidation and eviction. However, the server code would remain unchanged.

From my perspective, the custom header strategy appears to be the most beneficial for inherent caching. Nevertheless, if the architectural obstacles are sizable, it might be wise to consider alternative solutions.

The server's reaction to the OPTIONS request is a 302 redirect. Since the browser sees this as a non-OK response, the initial GET request is halted.

It sounds like this is the main issue.
Is there an example request, perhaps via cURL I can use to reproduce this?

Is there an example request, perhaps via cURL I can use to reproduce this?

Check out this console log capture:

Open a chat with attachment

Select the image and inspect the html to see the src attribute

The auth token is moved from url query params to headers

Make a fetch call like
   fetch(`https://www.expensify.com/chat-attachments/${reportId}/${attachmentPath}`, {
     headers: {
       'X-Chat-Attachment-Token': authToken,
     },
   })

I guess you can even use made up attachmentPath and X-Chat-Attachment-Token because they would trigger an OPTION request and the request will fail, the actual attachment doesn't yet metter at that point it's all about the OPTIONS request

perhaps via cURL I can use to reproduce this?

Keep in mind that CORS is particular to browsers and cURL won't initiate an OPTIONS request, browsers do that as a precaution before the actual request, cURL and other utils just perform the actual request
The OPTIONS request should be a plain GET OPTIONS https://www.expensify.com/chat-attachments/${reportId}/${attachmentPath} for the resource (I guess you can run that in cURL)

Question: would this still be a problem if the chat attachments were served off the new.expensify.com domain instead of www.expensify.com? I think that is what is presenting the problem in the browser right? We can do some sever/cloudflare magic to manipulate the hostname and have the attachments be accessible off the NewDot domain as well if that would help?

I've just tried it, now and I no longer get a CORS error, neither an OPTION request is being made, but instead of getting an attachment I'm receiving an html document with Expensify App's logo
From origin https://staging.new.expensify.com/
var response await fetch('www.expensify.com/chat-attachments/5720532303966235030/w_862e89b6f6f0afeec9a6986a366c786aa4ef17d7.png', {
    headers: {
        'X-Chat-Attachment-Token': 'HznVDJAe%2Be8ta%2FjAePLypOqkQtxmk5SOR6bN0OEU%2F6kvfmg5%2BvpfqeV9K%2BbuqADS7OLULKrcVx7iIX5XrYp1vpVYiKpNKSlxyFORgB1DgLaFHas58cKmyllrIBR4T9hQXm98ywDLiGTqKt1WgcLT2vu3FUeA3FBpj7s%2FcHc9wYIdaro89Mx5qaO9xa9hrh2kXWEqRzX19PLtRUzqLPvjcb8qWzeyPNotjMb2rrdMcA2XEQYgqJ3baezC8TaR5vzihoXJTsTCmaX7gtFfDvnL7nboFdrYE2lUewuGkutr%2BKw7cmRxUYYlWh%2F9TIbBIRG6RNZirAZFvJee8t1UEAwev78zs%2BW5RMHpf2su4sei4fqndq0G1NrWIiINnJj7h2xwqychUqQ5pmmEbJ80Hj4PT9WTfCOGRCaZD0JhK3cofqA95i6a7pph8wQXC7Emahs%2BZ%2B71USBRQ6yy1PnARQsl3j1fRgq1Db1iAJipYJEBV0co2zHmGQQydcBhsYJLsuo95QrL9Gia5csNrjs12MRvKwASnv1XVdLXlqlUBFAWG6j4Ly%2BKh4jGARPSgZTixVa14BF2SlOW83byqr4UaRFApjpnAim64%2B4j2HltONEr1x4d6d8JtUcb%2BosAbAy3506ofUM02vGuXXDB256e23ig%2BwQeN9N4dLZMfWQcZyGpiFG19DcEzrRgZ%2BT3FkO9WmTOW1IHuSL3Bd%2BfIZEatOXipRdwO5Kt1erdHkkrnhARuxpDDCU%2BzUPuxLqzQPbaTwIHOfm6XB8bVYIB9kWQByjQ9vmh6pI%2BRp%2BzT15uMpn92wUHI89ZrFOjV3bV1324fNTsRdGQHOvRGIHQpxjcVdZLnMKLokyFFpJx0SsYQT%2BpxlfucEv2Mhnkz%2FWmIc7WGdZY1P%2FwbA%2BU0nDux3VqZESyxAiRZw1ZeX0MVvCzL2z3%2Bj%2Fekab8SmB2gxqhGVEk55JIhxPx38Qe29wvDELvheQPxWrEV0ShnZaGV9xIaKNrubRGZVi7%2FSkaFjT%2B05j6PeTP'
    }
})

console.log(response.status) // 200
console.log(response.headers.get('content-type')) // text/html

For reference in the past this used to result in a preflight OPTIONS request and then an error: https://github.com/Expensify/App/issues/14991#issuecomment-1424720123
Now it seems the request is re-routed to the path on the same origin, which does not result in CORS error, but also doesn't return the attachment, but a document like "One moment while we connect you to Expensify"

Question: would this still be a problem if the chat attachments were served off the new.expensify.com domain instead of www.expensify.com?

It should no longer be a problem
When both entities operate from the same origin, CORS request are not being made.

Thanks, @kidroca for the overview. Super helpful! @justinpersaud, do you a better idea of where we're at now and any suggestions for our next steps? Thanks!

Yeah, I have some ideas. Let me put them together and run them by some other infra folks to validate them. What's the priority on this? Is it still monthly or are you hoping for sooner?

I'm always hoping for sooner 😆 ! Jests aside, somewhere in the ballpark of weekly to monthly sounds about right. It's not attached to a wave, but it unblocks fixing our image upload experience, which isn't really premium as we gear up to get existing customers migrated across.

Got it. I was able to reproduce the issue using @kidroca's steps. I have an idea on how to solve this and posted it for some feedback https://expensify.slack.com/archives/C094TGUTZ/p1698098986137179
I will try and test it out locally tomorrow and see if it works.

I have a working fix for this in my dev environments
https://github.com/Expensify/Expensidev/pull/809
Once reviewed/approved, I'll work on getting it out to staging/production to test further.

I submitted a PR to get this into staging/production. Once deployed, maybe someone here can help test this out again and help validate.

This is working in staging and production now using the example @kidroca provided.
Can anyone else help confirm and see if we can close this?
Request:
OPTIONS /chat-attachments/8923768874934211380/w_2bbeb8dd4d2843915d9bb8c6c0e085c14dfa8225.png.1024.jpg HTTP/2
Host: www.expensify.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Access-Control-Request-Method: GET
Access-Control-Request-Headers: x-chat-attachment-token
Referer: https://new.expensify.com/
Origin: https://new.expensify.com
Connection: keep-alive
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
TE: trailers

Response:
HTTP/2 204
date: Thu, 26 Oct 2023 14:19:29 GMT
access-control-allow-origin: https://new.expensify.com
access-control-allow-methods: GET, OPTIONS
access-control-allow-headers: X-Chat-Attachment-Token
cf-cache-status: DYNAMIC
set-cookie: __cf_bm=LiO2iiAyH5JuJoh0KH2wVGGlZru0oDRx2b1SAOJzhFk-1698329969-0-AciTxSxU2tcuHgwnMVBDsIO4KLiIrfky8/J4Br4MACcI+cicej0bsangkTWXiE1n4tuwEhU9CHB5Pi/1zLEoGLM=; path=/; expires=Thu, 26-Oct-23 14:49:29 GMT; domain=.expensify.com; HttpOnly; Secure; SameSite=None
set-cookie: _cfuvid=mLPpEYXWmCK8lOb6vUCUZrFu7pwfDkdIbfLI7MqE3.w-1698329969788-0-604800000; path=/; domain=.expensify.com; HttpOnly; Secure; SameSite=None
strict-transport-security: max-age=31536000; includeSubDomains; preload
server: cloudflare
cf-ray: 81c355a679c939c3-YYZ
cf-team: 1c0df3dc08000039c35c1cd400000001
X-Firefox-Spdy: h2

Niceeeee! @Beamanator @mollfpr perhaps for a second confirmation?

Looking at where we left off in this PR, I think the purpose of this issue has been completed so let's close! We'll continue the App work / discussion in https://github.com/Expensify/App/issues/12603. Thanks @justinpersaud & everyone involved!


[Addition]: spoofed clusters of heltec miners in Philippines (owner 14i7VBwFAWrffqmP4e9Lpjir9vUJzCsFcHCowyEUud9WJZgBGao)

Hotspot b58 Addresses
112wdwT9srqyiccypU3oxZV8zvZq4YXCy1der4tyfDCPhXwbkVS5
11xJgFnG2gdpTqFNwL7ot2x4PXR9ZhVaF4ysqxS8j2qbyRphvTJ
112nWj5LKQ7tKA5De7Um9KwCBxVaw9ff9eyitkGTB5vGB8LMFrAi
11DNuNhdkh8WaSg8MMFbMVnPpjHnWJFoaq7rYtnc1v3f9D8XJXd
112qrEYA2gPHHG5TqWcjzfanazv6mRiybiyRQPNGbw5pGSe4SwNi
11yG41WA6hAEaDJ21WQsy7zY7AaSw7LUfLEoep3VUgbj9DvCjbC
112GeMP7jahvYbiPez4aGANRYvSVRVZT9JTo2Ucc4FEmBx2XeALp
11xGKyZyDbX3FcAHL8Nz8eXsreXZieyMSd2CsNWFgYowRDXarcw
Discord Handle
screwdriver#9187
Reason(s)
spoofed cluster of miners in a remote provincial area in the Phlippines. IPs are Hongkong.

We have taken into account this addition request in the next iteration of the denylist. Some or all of the Hotspots listed may be added to the denylist. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


Integrate STAR data into roster front-end

Current status:

Check STAR & MCAS administration dates.

STAR: [date_taken]

Raw data files unique by [student_id, date_taken] i.e., multiple assessment results per student by test date

MCAS: March/April

How are you pulling in re-tests? Are you taking the first or most recent (to account for retests) MCAS assessment result?

Ah, thank you @marimuraki!

STAR: Will add date_taken as an attribute and assign on import.
MCAS: Not pulling in retests at all right now.

What this looks like
GIF:

Will you build in search by subject e.g., search for "Math" shows STAR Math and MCAS Math?

@marimuraki All about what we decide to prioritize as a team! Let's discuss together on Monday. We should also figure out what we want to learn from our Healey teacher friends by showing them this prototype. That will help us set priorities.

Yesterday we demoed the roster with STAR data for the Healey 5th grade team. Successor issues are #56 (richer data) and #59 (more flexible sorting).


Cannot create composite servers

name
about
labels
assignees

Bug report
Creating composite servers
bug
mihaela-mh

Description
Cannot create composite servers (a server that has nics/cdroms/volumes attached).
Expected behavior
When trying to create a server that has any entities attached, the operation should succeed.
Environment
Java version: 1.11

SDK version: 5.0.0

OS: Ubuntu 20.04

Configuration Files
N/A
Error and Debug Output
The SDK returns NullPointerException when calling the datacentersServersPost method.

I am currently working on it.
In the Cloud API swagger file the AttachedVolumes, Cdroms and Nics items property is read-only, which means that when the Java SDK is generated, it does not generate setters for the items field. To solve this problem, I opened a PR for the swagger file to make the specified parameters editable.
The next step is to regenerate the SDK and release a new version of it.

new sdk has been generated and published.


[BUG] ARM getLogContent() doesn't return the full log contents.

Describe the bug
I am using getLogContent method of ContainerGroups interface to get the complete log of azure container instance and save it before it gets deleted. But It does not fetch me the complete log. It just gives me the last part of the log. It does not have the data of when ACI got started.
Exception or Stack Trace
There is no exception. But it does not give the end to end log.
To Reproduce
1- Create an ACI
2- let it load the docker image and run for sometime.
3- collect the log by using getLogContent method
4- Log wont have the loggers got executed very first. Means you will not get end to end log.
Code Snippet
@Autowired
private AzureResourceManager azureResourceManager;
String logContent = azureResourceManager.containerGroups().getLogContent(,
, );
Expected behavior
I expect end to end log of operations performed inside the azure container instance. not only the last part of the log.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):

OS: Linux (ubuntu 20.04)
IDE: Eclipse
Library/Libraries:

com.azure.resourcemanager
azure-resourcemanager
2.14.0

Java version: jdk-13.0.2
App Server/Environment: [Deployed in Azure VM - Linux (ubuntu 20.04) ]
Frameworks: Spring Boot

@weidongxu-microsoft could you please follow up with @smita-mohapatra?

This seems to be backend behavior, on returning latest log, instead of all history.
@XiaofeiCao to investigate.

Hi @smita-mohapatra
Have you tried the overload with tailLineCount parameter?
https://github.com/Azure/azure-sdk-for-java/blob/2cd30e97440042b17a94db3438865ed530fb4c08/sdk/resourcemanager/azure-resourcemanager-containerinstance/src/main/java/com/azure/resourcemanager/containerinstance/models/ContainerGroup.java#L154-L162
Pass in Integer.MAX_VALUE as second parameter. My test shows it does return more logs than the one without tailLineCount. (3000 lines of logs)
If you did and still can't get the full log, we may want to let service team confirm this behavior.


Using RandomizedCV with xgboost

Hi all,
I am trying to use randomized cross validation on dask-xgboost.
Here is the snippet of code I am trying to get to work:
from dask_ml.model_selection import RandomizedSearchCV
from dask_ml.datasets import make_classification
from scipy.stats import uniform

X, y = make_classification(chunks=50)

model=dxgboost.XGBClassifier()
client=Client()
metric="f1"
data=X
target=y
bounds={"base_score": uniform(0.3, 0.7), "max_depth": uniform(3, 40), "learning_rate": uniform(0.05, 0.4),
                       "n_estimators": uniform(50, 200), "gamma": uniform(0, 10)}
results=RandomizedSearchCV(model, bounds,cv=3,scoring=metric,scheduler=client.scheduler).fit(data,target).best_estimator_

While running this code, I get the error TypeError: 'Future' object does not support indexing.
I think that I am misusing the api. If somebody can confirm that and point me to the right direction, I will be gratefull.
Thanks

FYI, I think you want scheduler=client, not scheduler=client.scheduler.

That gives a different error
ValueError                                Traceback (most recent call last)
<ipython-input-10-6b15c84df156> in <module>()
      9                            scoring=metric,
     10                            scheduler=client,
---> 11                            ).fit(data,target).best_estimator_

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/dask_searchcv/model_selection.py in fit(self, X, y, groups, **fit_params)
    865         scheduler = _normalize_scheduler(self.scheduler, n_jobs)
    866
--> 867         out = scheduler(dsk, keys, num_workers=n_jobs)
    868
    869         results = handle_deprecated_train_score(out[0], self.return_train_score)

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/client.py in get(self, dsk, keys, restrictions, loose_restrictions, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, **kwargs)
   2157             try:
   2158                 results = self.gather(packed, asynchronous=asynchronous,
-> 2159                                       direct=direct)
   2160             finally:
   2161                 for f in futures.values():

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/client.py in gather(self, futures, errors, maxsize, direct, asynchronous)
   1560             return self.sync(self._gather, futures, errors=errors,
   1561                              direct=direct, local_worker=local_worker,
-> 1562                              asynchronous=asynchronous)
   1563
   1564     @gen.coroutine

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/client.py in sync(self, func, *args, **kwargs)
    650             return future
    651         else:
--> 652             return sync(self.loop, func, *args, **kwargs)
    653
    654     def __repr__(self):

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/utils.py in sync(loop, func, *args, **kwargs)
    273             e.wait(10)
    274     if error[0]:
--> 275         six.reraise(*error[0])
    276     else:
    277         return result[0]

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/utils.py in f()
    258             yield gen.moment
    259             thread_state.asynchronous = True
--> 260             result[0] = yield make_coro()
    261         except Exception as exc:
    262             error[0] = sys.exc_info()

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/tornado/gen.py in run(self)
   1097
   1098                     try:
-> 1099                         value = future.result()
   1100                     except Exception:
   1101                         self.had_exception = True

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/tornado/gen.py in run(self)
   1105                     if exc_info is not None:
   1106                         try:
-> 1107                             yielded = self.gen.throw(*exc_info)
   1108                         finally:
   1109                             # Break up a reference to itself

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/client.py in _gather(self, futures, errors, direct, local_worker)
   1437                             six.reraise(type(exception),
   1438                                         exception,
-> 1439                                         traceback)
   1440                     if errors == 'skip':
   1441                         bad_keys.add(key)

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    690                 value = tp()
    691             if value.__traceback__ is not tb:
--> 692                 raise value.with_traceback(tb)
    693             raise value
    694         finally:

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/dask_searchcv/methods.py in fit_and_score()
    278     y_test = cv.extract(X, y, n, False, False)
    279     est_and_time = fit(est, X_train, y_train, error_score,
--> 280                        fields, params, fit_params)
    281     if not return_train_score:
    282         X_train = y_train = None

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/dask_searchcv/methods.py in fit()
    214         try:
    215             est = set_params(est, fields, params)
--> 216             est.fit(X, y, **fit_params)
    217         except Exception as e:
    218             if error_score == 'raise':

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/dask_xgboost/core.py in fit()
    273         3. The ``classes_`` and ``n_classes_`` attributes are not learned
    274         """
--> 275         client = default_client()
    276         xgb_options = self.get_xgb_params()
    277         self._Booster = train(client, xgb_options, X, y,

~/miniconda3/envs/dask-xgboost/lib/python3.6/site-packages/distributed/client.py in default_client()
   3609         return c
   3610     else:
-> 3611         raise ValueError("No clients found\n"
   3612                          "Start an client and point it to the scheduler address\n"
   3613                          "  from distributed import Client\n"

ValueError: No clients found
Start an client and point it to the scheduler address
  from distributed import Client
  client = Client('ip-addr-of-scheduler:8786')

There may be two issues.
"No clients found", and a secondary issue once that's resolved:
It seems like by the time it gets to dask_searchcv.methods.fit_and_score the values are NumPy arrays, while dask-xgboost only expects dask arrays (or dataframes).
I'm debugging with
from dask_ml.model_selection import RandomizedSearchCV
from dask_ml.datasets import make_classification
from scipy.stats import uniform
import dask_ml.xgboost as dxgboost
from distributed import Client

def main():
    client=Client(processes=False)
    X, y = make_classification(chunks=50)

    model = dxgboost.XGBClassifier()
    metric="f1"
    data=X
    target=y
    bounds={"max_depth": uniform(0.3, 0.7)}
    results=RandomizedSearchCV(model,
                               bounds,
                               cv=3,
                               scoring=metric,
                               scheduler=client,
                               ).fit(data,target).best_estimator_

__spec__ = 'bug.py'

if __name__ == '__main__':
    main()

I likely won't have time to debug this further right now. Would welcome additional help here if you have time @dma092

I would just leave scheduler= blank and let the default behavior handle
things
On Thu, Jun 28, 2018 at 10:49 AM, Tom Augspurger<EMAIL_ADDRESS>wrote:

There may be two issues.
"No clients found", and a secondary issue once that's resolved:
It seems like by the time it gets to dask_searchcv.methods.fit_and_score
the values are NumPy arrays, while dask-xgboost only expects dask arrays
(or dataframes).
I'm debugging with
from dask_ml.model_selection import RandomizedSearchCVfrom dask_ml.datasets import make_classificationfrom scipy.stats import uniformimport dask_ml.xgboost as dxgboostfrom distributed import Client
def main():
client=Client(processes=False)
X, y = make_classification(chunks=50)
model = dxgboost.XGBClassifier()
metric="f1"
data=X
target=y
bounds={"max_depth": uniform(0.3, 0.7)}
results=RandomizedSearchCV(model,
                           bounds,
                           cv=3,
                           scoring=metric,
                           scheduler=client,
                           ).fit(data,target).best_estimator_

spec = 'bug.py'
if name == 'main':
main()
I likely won't have time to debug this further right now. Would welcome
additional help here if you have time @dma092 https://github.com/dma092
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-xgboost/issues/21#issuecomment-401061498,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszK8SdsZ457KYOrxs0ZGJXh0f3QZbks5uBOz5gaJpZM4U7etR
.

Long term it looks like dask-searchcv needs to update to the newer
scheduler= keyword handling in dask
On Thu, Jun 28, 2018 at 10:53 AM, Matthew Rocklin<EMAIL_ADDRESS>wrote:

I would just leave scheduler= blank and let the default behavior handle
things
On Thu, Jun 28, 2018 at 10:49 AM, Tom Augspurger<EMAIL_ADDRESS>
wrote:

There may be two issues.
"No clients found", and a secondary issue once that's resolved:
It seems like by the time it gets to dask_searchcv.methods.fit_and_score
the values are NumPy arrays, while dask-xgboost only expects dask arrays
(or dataframes).
I'm debugging with
from dask_ml.model_selection import RandomizedSearchCVfrom dask_ml.datasets import make_classificationfrom scipy.stats import uniformimport dask_ml.xgboost as dxgboostfrom distributed import Client
def main():
client=Client(processes=False)
X, y = make_classification(chunks=50)
model = dxgboost.XGBClassifier()
metric="f1"
data=X
target=y
bounds={"max_depth": uniform(0.3, 0.7)}
results=RandomizedSearchCV(model,
                           bounds,
                           cv=3,
                           scoring=metric,
                           scheduler=client,
                           ).fit(data,target).best_estimator_

spec = 'bug.py'
if name == 'main':
main()
I likely won't have time to debug this further right now. Would welcome
additional help here if you have time @dma092 https://github.com/dma092
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-xgboost/issues/21#issuecomment-401061498,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszK8SdsZ457KYOrxs0ZGJXh0f3QZbks5uBOz5gaJpZM4U7etR
.

@TomAugspurger
I would be glad to!

Any idea when this will be resolved?

@TomAugspurger Is there currently any other approaches to run Hyperparameter tuning for XGBoost in Dask-ML where the workload is primarily "memory constrained"? It seems both GridSearchCV and RandomizedCV with XGBoost in Dask-ML ends up in the "No clients found" error. Does XGBoost support partial_fit API to run in IncrementalSearchCV? Thanks for all the work related to Dask-ML.

I don't think XGBoost implements partial_fit, but it's possible to add on
your own by subclassing their estimators and using
xgb_model=self._booster on subsequent calls. I don't have an example
handy at the moment unfortunately.
On Mon, Jun 8, 2020 at 4:15 PM jcuda000001<EMAIL_ADDRESS>wrote:

@TomAugspurger https://github.com/TomAugspurger Is there currently any
other approaches to run Hyperparameter tuning for XGBoost in Dask-ML where
the workload is primarily "memory constrained"? It seems both GridSearchCV
and RandomizedCV with XGBoost in Dask-ML ends up in the "No clients found"
error. Does XGBoost support partial_fit API to run in IncrementalSearchCV?
Thanks for all the work related to Dask-ML.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-xgboost/issues/21#issuecomment-640890791,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKAOIVNCODPJGUI427TUBDRVVIIBANCNFSM4FHN5NIQ
.

@TomAugspurger - Thanks. Just to confirm, is there any hyperparameter tuning modules in Dask/Dask-ML which works currently for algorithms without partial_fit already implemented?

Yes, GridSearchCV and RandomizedSearchCV both work with estimators that
implement .fit. They just don't work with the latest version of xgboost's
estimators because of that bug in the type of exception raised, which is
fixed on xgboost master.
On Tue, Jun 9, 2020 at 7:17 AM jcuda000001<EMAIL_ADDRESS>wrote:

@TomAugspurger https://github.com/TomAugspurger - Thanks. Just to
confirm, is there any hyperparameter tuning modules in Dask/Dask-ML which
works currently for algorithms without partial_fit already implemented?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-xgboost/issues/21#issuecomment-641250900,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKAOIT3RMAKLIYBD7EOCFTRVYR4TANCNFSM4FHN5NIQ
.


emulate: save/restore host flags in fastop_dispatch

fastop_dispatch isn't saving/restoring host flags before overwriting with guest flags, which can cause some interesting issues. This fixes that by just pushing and poping host flags before/after executing fastop instruction.

OK to verify

emmm I'm investigating another interesting issue in PR#204 recently and finally came to the same root cause as described in this PR, that RFLAGS are corrupted after fastop.
I have some comments to the change, and I'll also add Alex to the reviewer.

Looks good to me! Thanks for the patch @jarveson, I also agree with the new pushf/popf placement suggested by @coxuintel.

sure, so something like this then?

@jarveson would you mind update the commit message in your own branch? I'll merge it.

like this? otherwise im not fully sure what you mean

like this? otherwise im not fully sure what you mean

Exactly. Unlike PR comments, detailed commit msg in original commit will remains in git repo which helps future development.


It told me to report this error here

Describe the bug
Message: Magick: zero-length blob not permitted `' @ error/blob.c/BlobToImage/446
Stack Trace: undefined
Steps to reproduce

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
No response
Screenshots
No response
Self-hosted instance?
No
Additional context
No response

What image/command did you use?

https://cdn.discordapp.com/attachments/846298836301643776/963075225212309554/IMG_20220411_155425.jpg
&caption srasz

Duplicate of #238


Prevent locking funds when using sendMessage with the target is an EOA and value > 0

In the L1XDM contract, we prevent sending ETH to this contract when using sendMessage function
In the L2XDM contract, we prevent sending the native token and the _target is an EOA when using sendMessage function, because in the relayMessage, the _target must transfer the native token when that is called.

Please don't make changes in the contracts. We are all preparing the external audit (resolving test code in contracts and NAT spec...)

These changes are from the internal audit. Related to this problem we discussed before

I changed to the Draft PR, wait for the external audit report to decide.


Rasa X - Migration database error

Rasa version:1.4.3
Rasa X version:0.22.1
Python version:3.6
Operating system:windows
Issue:
The Rasa X can't finish the database migration to oracle.
Error (including full traceback):
Traceback (most recent call last):
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
cx_Oracle.DatabaseError: ORA-00906: parentese esquerdo n?o encontrado

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "migrate_tracker.py", line 220, in <module>
    args.max_trackers,
  File "migrate_tracker.py", line 102, in _migrate_tracker_store_to_rasa_x
    sql_migrations.run_migrations(db_session)
  File "C:\Users\x01214707173\AppData\Roaming\Python\Python36\site-packages\rasax\community\sql_migrations.py", line 25, in run_migrations
    _run_schema_migrations(session)
  File "C:\Users\x01214707173\AppData\Roaming\Python\Python36\site-packages\rasax\community\sql_migrations.py", line 50, in _run_schema_migrations
    command.upgrade(alembic_config, "head")
  File "C:\Program Files\Python36\lib\site-packages\alembic\command.py", line 276, in upgrade
    script.run_env()
  File "C:\Program Files\Python36\lib\site-packages\alembic\script\base.py", line 475, in run_env
    util.load_python_file(self.dir, "env.py")
  File "C:\Program Files\Python36\lib\site-packages\alembic\util\pyfiles.py", line 90, in load_python_file
    module = load_module_py(module_id, path)
  File "C:\Program Files\Python36\lib\site-packages\alembic\util\compat.py", line 177, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Users\x01214707173\AppData\Roaming\Python\Python36\site-packages\rasax\community\database\schema_migrations\alembic\env.py", line 89, in <module>
    run_migrations_online()
  File "C:\Users\x01214707173\AppData\Roaming\Python\Python36\site-packages\rasax\community\database\schema_migrations\alembic\env.py", line 79, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "C:\Program Files\Python36\lib\site-packages\alembic\runtime\environment.py", line 839, in run_migrations
    self.get_context().run_migrations(**kw)
  File "C:\Program Files\Python36\lib\site-packages\alembic\runtime\migration.py", line 362, in run_migrations
    step.migration_fn(**kw)
  File "C:\Users\x01214707173\AppData\Roaming\Python\Python36\site-packages\rasax\community\database\schema_migrations\alembic\versions\migration_2019_08_30_add_unique_id_to_entitysynonyms_d45a0bf21e89.py", line 72, in upgrade
    sa.PrimaryKeyConstraint("id"),
  File "<string>", line 8, in create_table
  File "<string>", line 3, in create_table
  File "C:\Program Files\Python36\lib\site-packages\alembic\operations\ops.py", line 1248, in create_table
    return operations.invoke(op)
  File "C:\Program Files\Python36\lib\site-packages\alembic\operations\base.py", line 345, in invoke
    return fn(self, operation)
  File "C:\Program Files\Python36\lib\site-packages\alembic\operations\toimpl.py", line 101, in create_table
    operations.impl.create_table(table)
  File "C:\Program Files\Python36\lib\site-packages\alembic\ddl\impl.py", line 252, in create_table
    self._exec(schema.CreateTable(table))
  File "C:\Program Files\Python36\lib\site-packages\alembic\ddl\oracle.py", line 30, in _exec
    result = super(OracleImpl, self)._exec(construct, *args, **kw)
  File "C:\Program Files\Python36\lib\site-packages\alembic\ddl\impl.py", line 134, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\sql\ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1050, in _execute_ddl
    compiled,
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\util\compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\util\compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "C:\Program Files\Python36\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00906: left parenthese not found
[SQL:
CREATE TABLE entity_synonym_value (
        id INTEGER NOT NULL,
        entity_synonym_id INTEGER,
        name VARCHAR2,
        PRIMARY KEY (id),
        FOREIGN KEY(entity_synonym_id) REFERENCES entity_synonym (id) ON DELETE cascade
)

]
(Background on this error at: http://sqlalche.me/e/4xp6)

Command or request that led to error:
python -m rasax.community.server

Thanks for raising this issue, @amn41 will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗

Hi @DiegoProtec ,
thanks for raising this issue. We already fixed this and will include this in the next patch release :-) I'll keep it open for now, until we have released the version including this change.

Fixed in 0.22.2 🙂

Images

Rasa: rasa/rasa:1.4.3
Rasa X: rasa/rasa-x:0.22.2
Rasa SDK: rasa/rasa-sdk:1.4.0
RabbitMQ: rabbitmq:3.8.1-management

Problem

When I run docker-compose, the following error occurs:
rasa-x_1           | [SQL: ALTER TABLE entity_synonym ADD id INTEGER NOT NULL]
rasa-x_1           | (Background on this error at: http://sqlalche.me/e/4xp6)

When run the "migrate.py", with environment variable "DB_URL":
DB_URL=oracle://admin:admin@localhost:1521/db

occurs this:
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00936: expression not found
[SQL: select name, synonym from entity_synonym]
(Background on this error at: http://sqlalche.me/e/4xp6)

@DiegoProtec https://github.com/RasaHQ/rasa/issues/4754 covers your latest comment, right?


Revert "bump version to 2.6.0-SNAPSHOT (#2358)"

What problem does this PR solve?
This reverts commit a411fbda
What is changed and how it works?
we don't use revision now, because it doesn't work when install or deploy

/run-all-tests

/merge


Implement Zipper.map

Allows to map all labels in a Zipper and preserve focus. See doc examples in src/Tree/Zipper.elm.
Also I've upgraded elm dependency in package.json to latest elm-0.19.0-no-deps. Without it npm install would fail for me.
Use case
I was looking for a way to display a Rose Tree Zipper as Html msg and highlight the focused sub-tree.
Currently there is Tree.restructure (https://package.elm-lang.org/packages/zwilias/elm-rosetree/latest/Tree#restructure), but I can't think of a way to pass the information about what is currently focused. I thought about mapping the tree into another tree that contains highlighted flag, and then updating the focused label to set highlighted to True, but again I don't know how to preserve the focus.
With Zipper.map it is possible as follows:
type alias Notebook =
    Zipper String

type alias Visual =
    { highlighted : Bool
    , text : String
    }

display : Notebook -> Element msg
display notebook =
    notebook
        |> Zipper.map (Visual False)
        |> Zipper.mapLabel (\visual -> { visual | highlighted = True })
        |> Zipper.toTree
        |> Tree.restructure identity displayItem

displayItem : Visual -> List (Element msg) -> Element msg
displayItem item children =
    Element.column
        [ Border.width 1
        , Element.padding 10
        , Element.spacing 5
        , Background.color
            (if item.highlighted then
                Element.rgb 1 1 0.6

             else
                Element.rgb 1 1 1
            )
        ]
        [ Element.text item.text
        , Element.column [ Element.spacing 5 ] children
        ]

I'm using mdgriffith/elm-ui in the example, but it's trivial to translate it to elm/html

Hey! Any thoughts on this? It's OK if you are busy.

It's sad that such amazing projects are abandoned like this.


BetterSafariView Opens URL Twice

Description
BetterSafariView is opened twice on the same URL.
Project Configuration
Xcode 15.1 targeting iOS 17.2. The containing view does not have a navigation stack.
Sample Project
SafariDemo.zip
Demo Video
https://github.com/stleamist/BetterSafariView/assets/384288/c1dcc68b-26fe-40c8-8659-7aa6aa9330a3
Sample Code
The relevant code is in this listing.
import SwiftUI
import BetterSafariView

enum UseCaseButtons: CaseIterable, Identifiable {
    var id: Self { self }

    case networkKit
    case betterSafariView
    case reviewKit
    case urlEncodedForm

    var url: URL {
        switch self {
        case .networkKit:
            URL(string: "https://swiftpackageindex.com/sabapathyk7/NetworkKit").unsafelyUnwrapped
        case .betterSafariView:
            URL(string: "https://swiftpackageindex.com/stleamist/BetterSafariView").unsafelyUnwrapped
        case .urlEncodedForm:
            URL(string: "https://swiftpackageindex.com/interactord/URLEncodedForm").unsafelyUnwrapped
        case .reviewKit:
            URL(string: "https://swiftpackageindex.com/FlineDev/ReviewKit").unsafelyUnwrapped
        }
    }

    var title: String {
        switch self {
        case .networkKit:
            "NetworkKit"
        case .betterSafariView:
            "BetterSafariView"
        case .reviewKit:
            "ReviewKit"
        case .urlEncodedForm:
            "URLEncodedForm"
        }
    }
}

struct ContentView: View {
    @State var selectedUseCase: UseCaseButtons?

    var body: some View {
        ScrollView {
            VStack {
                ForEach(UseCaseButtons.allCases) { message in
                    Button(message.title) {
                        selectedUseCase = message
                    }
                }
                .padding()
                .safariView(item: $selectedUseCase) { useCase in
                    SafariView(url: useCase.url)
                }
            }
        }
    }
}

#Preview {
    ContentView()
}

This was as scoping issue, applying the safariView to a parent (the ScrollView) instead of the ForEach fixes the problem.


dockerize the benchmark for resources control (WIP)

[x] docker compose
[x] rust REST server
[ ] REST source
[x] CI

@YassinEldeeb close or leave this one open?
I think since then we made some fixed and adjustments to the benchmark, so it might be fine to run it on the GH runner.

@dotansimha I think still, controlling the resources used by each binary is useful.
the purpose of this is to predictably control the allocated resources for each of the following:

upstream source rust server
conductor gateway

And currently, we just freely make them allocate whatever they need, when it can be sometimes unfair.
Also, it mimics how it would run for most large production infras, where they allocate specified memory and cpu resources for their K8s' pods.


planning/execution should be operation aware

Currently planning & execution is not aware of the possibility of having multiple operations within a document. Planning should be extended to plan for multiple operations. Execution should take the operationName from the request and only execute this particular operation.

Done


Replace the onKeyDown & onKeyUp events

Replace the onKeyDown and onKeyUp events with a single BABYLON observable that I name _onKeyBoardObservable.
So it will always be compatible with future versions of browsers.

It has been tested and it works. Besides, I also tested with a glTF character and it works much better in terms of collisions with the ground.
However, I noticed a problem here:
let myWalkAnimationGroup:AnimationGroup = ...;
let agMap = {
     "walk":myWalkAnimationGroup,
     "run" : {"ag": myRunAnimationGroup,"rate": 1},
     "idle" : {"ag": myIdleAnimationGroup,"loop": true,"rate": 1},
};

"walk": myWalkAnimationGroup, This works
But this not works :
"run" : {"ag":myRunAnimationGroup,"rate":1},
"idle" : {"ag":myIdleAnimationGroup,"loop":true,"rate":1},

Maybe I used it wrong. while doing it like this, it work :
setIdleAnim(animationGroups.idle, 1.0, true);
Maybe I should open a new issue for this?

Sorry for the delay in looking into this but I am not sure why we should complicate things with observable. Why add another library dependency?. The current code is simple, will not become deprecated anytime soon and do not require any additional library

There are no additional libraries.
Observables are built into Babylon and its more modern use of events.

Yes it is in Babylon but is still additional library. It is not part of
standard JavaScript. Browser don't support it natively
On Wed, Nov 16, 2022, 11:29 AM David @.***> wrote:

There are no additional libraries.
Observables are built into Babylon and its more modern use of events.
—
Reply to this email directly, view it on GitHub
https://github.com/ssatguru/BabylonJS-CharacterController/pull/54#issuecomment-1317391957,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAVRJDIZZAETBM7SRUXH7VLWIUKXPANCNFSM54AOJMSA
.
You are receiving this because you commented.Message ID:
@.***>

The CharacterControler refers to Babylon. It can only work with Babylon. Your whole class is not just from Javascript, but from Babylon. So no additional library anyway because your class will not work without the Babylon library.
You are using BABYLON.Vector3... There is nothing more different or additional than using a BABYLON.Observable.
Observables are the future, more modern than of simples calls onKeyDown onKeyUp and all the code around it.
But you do as you wish. But I think it's a good improvement to use observables in the future. Javascript is starting to integrate them too, because they are more efficient.

I looked into this more.
Babylonjs does not use any library to provide observable.
They have implemented this pattern themselves.
You can see this here
https://github.com/BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Misc/observable.ts#L99
Though I am not very keen on adding this, I will research this further to see if it provides any additional value.


Add default value for go-mod-path

I noticed that we don't have a default value for the go-mod-path, even though it's usually at the root of the repository. I propose adding the default value here.

@febuiles Do we have automation pushing a release or do we need to cut that manually?

@jhutchings1 didn't get any feedback about this when inquiring earlier, I'll PR some instructions in the contribution doc this week, fix the tags and push a new version. If you want to do it by yourself these instructions should "just work" ™: https://github.com/actions/dependency-review-action/blob/main/CONTRIBUTING.md#cutting-a-new-release

I've manually pushed a new release. With the latest action added the v1 tag has been automatically sync'd.


How to rescue SocketError when using RestClient?

I am facing a problem rescuing SocketError I have detailed the issue here http://stackoverflow.com/questions/40779163/how-to-rescue-socket-error-in-ruby/40779800#40779800
Can someone help me with this?

I am using RestClient to make a network call in the ruby class. I am getting a SocketError whenever I am not connected to the internet. I have added a rescue block to catch the exception still I am not able to do so.
the error message is: SocketError (Failed to open TCP connection to api.something.com:443 (getaddrinfo: Name or service not known))

module MyProject
  class Client
    def get_object(url, params={})
      response = RestClient.get(url, {params: params})
    rescue SocketError => e
      puts "In Socket errror"
    rescue => e
      puts (e.class.inspect)
    end
  end
end

The broad rescue gets called and print SocketError, but why the previous rescue SocketError is not triggered!
Do you see something that I am missing?

I'm not able to reproduce your issue. When I run your code, I see In Scoket errror printed.
What version of rest-client are you running?

Sorry for the typo :sob:
and I am using 'rest-client', '~> 2.0', I believe this is the latest version.

Hmm, do you have full sample code that I could run to demonstrate the issue?

I'm working on a project, give me some time, I'll ask someone and let you know if I can share the code with you. I'll update to ASAP.

Ideally this wouldn't include project code, just a minimal amount of code needed to reproduce.
For example, this code:
#!/usr/bin/env ruby
require 'rest-client'
require 'socket'

module MyProject
  class Client
    def get_object(url, params={})
      response = RestClient.get(url, {params: params})
    rescue SocketError => e
      puts "In Socket errror"
    rescue => e
      puts (e.class.inspect)
    end
  end
end

MyProject::Client.new.get_object('http://nonexistent.nodomain')

Results in this output:
In Socket errror

I thought it'll be difficult on you part as it involves facebook auth. Let me know if you need more clarity.
module FbAds
  class Client
    attr_reader :access_token

    def initialize(access_token)
      @access_token = access_token
    end

    def get_object(endpoint, params={})
      params.merge!({access_token: @access_token, format: 'json'})
      url = FbAds::Utils.build_url(endpoint)
      response = RestClient.get(url, {params: params})
      return JSON.parse(response.body)
    rescue RestClient::Exception => e
      if response_has_error?(e.response)
        handle_fb_api_errors(e.response)
      else
        raise FbError::RestClientException(e.response, e.message)
      end
    rescue SocketError => e
      puts "In Socket errror" # doesn't go here
    rescue => e
      puts (e.class.inspect) # still puts SocketError here
    end
  end
end

The same happens to me, not when my internet connection is disabled, but for a typo in the url - on purpose during testing. I mistyped some.url.de as some.url.der during testing - any non-existing url in a nested resource will bring up a SocketError for me, a non-existing page for an existing url will give a normal 404, that I can catch with rest-client.
To catch the resulting exception, I tried to go up thru your exceptions inheritances to the top (from RestClient::RequestFailed to RuntimeError class), not one of them grips SocketError. Maybe that helps a bit finding the reason?

RestClient doesn't wrap most of the exceptions that may be raised by underlying Net::HTTP libraries, so you will want to handle them separately. (Network errors, OpenSSL errors, etc.)

I want to handle them separately if you can just help how can I catch them. As I couldn't catch it.


Search queries stacked in the URL

I was playing around with the search function. I'm not entirely sure how (or how to reproduce), but I got my queries to stack somehow in the URL. Predictably, this lead to no results.

This looks correct, one is location and one is query. Confusing UX is already tracked


how to switch next/prev track?

I see there are stop, play, pause and seek methods.
Is it possible to skip next/prev track?
Thanks!

PR created:
https://github.com/thibauts/node-castv2-client/pull/89


[FEA] Support First() in windowing context with double type

Is your feature request related to a problem? Please describe.
This is a feature request to support first() function in windowing context with double type.
For example:
import org.apache.spark.sql.Row
import org.apache.spark.sql.types._
import org.apache.spark.sql.functions._

val data = Seq(
    Row(Row("Adam ","","Green"),"1","M",1000.1, "2019-01-01",List("Java","Scala")),
    Row(Row("Bob ","Middle","Green"),"2","M",2000.2, "2019-01-02",List("Java","Python")),
    Row(Row("Cathy ","","Green"),"3","F",3000.3, "2019-01-03",List())
)

val schema = (new StructType()
  .add("name",new StructType()
    .add("firstname",StringType)
    .add("middlename",StringType)
    .add("lastname",StringType))
  .add("id",StringType)
  .add("gender",StringType)
  .add("salary",DoubleType)
  .add("birthdayStr",StringType)
  .add("language",ArrayType(StringType))
             )

val df = spark.createDataFrame(spark.sparkContext.parallelize(data),schema)
df.withColumn("birthday", to_date(col("birthdayStr"))).write.format("parquet").mode("overwrite").save("/tmp/testparquet")
val df2 = spark.read.parquet("/tmp/testparquet")
df2.createOrReplaceTempView("df2")
df2.printSchema

sql("""SELECT gender, first(salary,true) OVER (PARTITION BY gender ORDER BY salary) FROM df2""").collect

Driver log message:
      !Expression <WindowExpression> first(salary#174, true) windowspecdefinition(gender#173, salary#174 ASC NULLS FIRST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) cannot run on GPU because the type of orderBy column is not supported in a window range function, found DoubleType

Actually not support first function in window context.
CUDF does not support this currently.

@viadea  Order by in window only supports int and timestamp, does not supports double float, it's a constraint of CUDF.
Like Count does not support double, you can test:
         SELECT gender, count(salary) OVER (PARTITION BY gender ORDER BY salary) FROM df2
So this issue will be:  Support First() in windowing context with int and timestamp.

All of these are going to require some CUDF changes to work.
nth_element_aggregation is not supported as a rolling_aggregation in cudf so we cannot implement it in a way that will be able to support null checks like the query above wants.
Cudf does not support range queries on floats or doubles yet.  We can add this in though.

Note: Still depends on cuDF: https://github.com/NVIDIA/spark-rapids/pull/6000#issuecomment-1188512673

[ ] For IntegerType
[ ] For Float/Double Type

Split this issue into 2, one for Integer, one for Double #6192

#6000 already supported Integer type.


ICE Connectivity issues

src:- https://github.com/ashumeow/MeowRTC/tree/master/MeowRTC/cloud/stack/python/demos/meowrtc-p2p
No fault seen in main.py
Problem in index.html inside the script
The appengine demo had been working successfully...
So where is the problem seen?
2 users from different browsers were able to successfully connect to eachother...
But, the users weren't able to listen to hear eachother's voice.
So, the problem is seen in the ICE side. :-1:

With respect to line (https://github.com/ashumeow/MeowRTC/blob/master/MeowRTC/cloud/stack/python/demos/meowrtc-p2p/index.html#L106), a commit (https://github.com/ashumeow/MeowRTC/commit/e7bd9bd32fe91a0782182a170a133d6ad577af8f) has been pushed. Need to check-up with the Google AppEngine! :|

Upgrading to 1.9.24 from 1.9.18 --- Requires to update

It's working! :+1:

Running in localhost... Updates needs to be deployed.. :|

Done! Deployed ;) ICE Connectivity issue solved...