Cool down when hitting rate limit with online processors

Fixes #252
We had cooldown before for openai: https://github.com/bespokelabsai/curator/blob/40aa7df91a7e8a82e461b2b2b07a7c2ee38e5b5b/src/bespokelabs/curator/request_processor/openai_online_request_processor.py#L456-L473
However, the way the main loop is running has changed so we need to change it a bit

Test
python tests/simple_online.py --log-level DEBUG --model deepinfra/meta-llama/Llama-3.3-70B-Instruct  --n-requests 5000 --max-requests-per-minute 3000 --max-tokens-per-minute<PHONE_NUMBER>0000 --max-retries 0

Observations:
This cool down only helps you if you are close to the correct rate limit and just need to pause sometimes.
If you are well over the correct rate limit, then as soon as you are un-paused you will send as many requests as possible and then once again be hit with tons of errors.
Because there is a delay between sending a request and getting a response, you can sends tons of requests even though you are already over the rate limit without knowing it.
Having something that is more responsive, as discussed in #233, like exponential retries that intelligently adapts and finds the correct rate limit would be better.
Right now we are relying on the local rate limit being correctly set. This PR helps mitigate failures when we hit rate limits randomly in patches (e.g. connection limit isn't actually a RPM rate limit so longer requests cause our estimation of RPM to be too large and we hit rate limits. This is also potentially better solved by #253)

Tested on claude 3.5 haiku, with a manual high rate limit, expected behaviors and logs
Command
(bespokelabs-curator-py3.12) (base) ➜  bella git:(ryanm/rate-limit-cool-down) ✗ python tests/simple_online.py --log-level DEBUG --model claude-3-5-haiku-20241022  --n-requests 5000 --max-retries 0;;

Some of my local logs
Pausing to cool down for 9 seconds
Seconds since rate limit error: 9.77731466293335
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: True
Pausing to cool down for 0 seconds
2024-12-14 21:42:28,402 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Pausing to cool down for 0 seconds
Pausing to cool down for 0 seconds
Seconds since rate limit error: 10.002909183502197
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.0047025680542
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.007076501846313
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.010509729385376
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.012789964675903
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.014800310134888
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.016777753829956
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.019784450531006
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.022103071212769
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.024208068847656
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.031663656234741
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.034281969070435
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.036867618560791
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.039520263671875
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.042527198791504
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.045552492141724
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.05044412612915
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.05648422241211
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.06947636604309
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.079070329666138
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.084420442581177
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.088730573654175
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.094618320465088
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.098593950271606
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.104597091674805
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.117860555648804
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.127331495285034
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.130967140197754
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.1381094455719
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.14184308052063
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.147015571594238
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.152969598770142
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.162304878234863
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.168402194976807
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.175575494766235
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.17983102798462
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.185556888580322
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.191916465759277
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.200887680053711
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.207088947296143
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.213093519210815
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.21965503692627
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.226195573806763
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.232726573944092
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.239461421966553
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Status tracker time of last rate limit error:<PHONE_NUMBER>.655001
2024-12-14 21:42:28,655 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 228 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,655 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 228 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Seconds since rate limit error: 0.0009074211120605469
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: True
Pausing to cool down for 9 seconds
Status tracker time of last rate limit error:<PHONE_NUMBER>.65985
2024-12-14 21:42:28,659 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 220 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,660 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 220 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.6647618
2024-12-14 21:42:28,665 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 232 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,665 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 232 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.6858182
2024-12-14 21:42:28,685 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 236 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,686 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 236 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.6901386
2024-12-14 21:42:28,690 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 234 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,690 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 234 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.694756
2024-12-14 21:42:28,694 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 233 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,695 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 233 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7018754
2024-12-14 21:42:28,702 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 235 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,702 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 235 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7118537
2024-12-14 21:42:28,711 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 237 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,712 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 237 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7199416
2024-12-14 21:42:28,720 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 238 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,720 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 238 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.724836
2024-12-14 21:42:28,724 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 239 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,725 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 239 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7375472
2024-12-14 21:42:28,737 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 241 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,737 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 241 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.744753
2024-12-14 21:42:28,744 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 242 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,745 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 242 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7602687
2024-12-14 21:42:28,760 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 240 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,760 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 240 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7695148
2024-12-14 21:42:28,769 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 243 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,769 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 243 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7756224
2024-12-14 21:42:28,775 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 244 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,775 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 244 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.781321
2024-12-14 21:42:28,781 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 255 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,781 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 255 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.7926455
2024-12-14 21:42:28,792 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 246 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,792 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 246 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.799815
2024-12-14 21:42:28,799 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 245 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,799 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 245 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.8107188
2024-12-14 21:42:28,810 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 247 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,811 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 247 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.8162231
2024-12-14 21:42:28,816 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 248 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,816 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 248 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.8222675
2024-12-14 21:42:28,822 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 249 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,822 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 249 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.8436055
2024-12-14 21:42:28,843 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 252 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,843 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 252 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.850122
2024-12-14 21:42:28,850 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 251 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,850 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 251 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.864091
2024-12-14 21:42:28,864 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 250 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,864 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 250 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.8703935
2024-12-14 21:42:28,870 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 253 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,870 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 253 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.9305582
2024-12-14 21:42:28,930 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 254 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:28,930 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 254 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Processing LiteLLMOnlineRequestProcessor requests:   2%|▋                            | 120/5000 [00:59<21:33,  3.77it/s]2024-12-14 21:42:38,662 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Pausing to cool down for 9 seconds
Pausing to cool down for 9 seconds
Seconds since rate limit error: 9.733609676361084
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: True
Pausing to cool down for 0 seconds
2024-12-14 21:42:38,932 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Pausing to cool down for 0 seconds
Pausing to cool down for 0 seconds
Seconds since rate limit error: 10.003473043441772
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.00525712966919
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.00659465789795
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.00950026512146
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.012519359588623
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.01540207862854
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.01784610748291
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.020509719848633
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.022376298904419
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.025001049041748
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.02773380279541
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.030853986740112
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.033184289932251
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.0364248752594
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.039759874343872
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.042680740356445
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.04600477218628
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.051254987716675
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.059996843338013
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.071706771850586
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.081772327423096
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.090312004089355
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.09453797340393
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.10104250907898
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.109780550003052
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.120887517929077
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.128454685211182
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.135640859603882
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.141072034835815
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.147302865982056
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.154208183288574
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.1580810546875
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.167264223098755
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.179534673690796
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.183794736862183
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.191077709197998
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.19514799118042
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.200405359268188
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.206177473068237
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.213131666183472
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.225002527236938
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.23212718963623
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.24032735824585
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Seconds since rate limit error: 10.246054649353027
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: False
Status tracker time of last rate limit error:<PHONE_NUMBER>.1813638
2024-12-14 21:42:39,181 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 278 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,181 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 278 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Seconds since rate limit error: 0.006224393844604492
if seconds_since_rate_limit_error < SECONDS_TO_PAUSE_ON_RATE_LIMIT: True
Pausing to cool down for 9 seconds
Status tracker time of last rate limit error:<PHONE_NUMBER>.1921756
2024-12-14 21:42:39,192 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 276 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,192 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 276 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2025673
2024-12-14 21:42:39,202 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 279 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,202 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 279 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.219472
2024-12-14 21:42:39,219 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 280 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,219 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 280 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2276735
2024-12-14 21:42:39,227 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 282 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,227 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 282 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2321038
2024-12-14 21:42:39,232 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 281 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,232 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 281 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2428505
2024-12-14 21:42:39,242 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 284 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,243 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 284 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2473881
2024-12-14 21:42:39,247 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 283 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,247 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 283 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2573447
2024-12-14 21:42:39,257 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 286 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,257 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 286 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2642558
2024-12-14 21:42:39,264 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 285 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,264 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 285 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.27671
2024-12-14 21:42:39,276 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 287 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,276 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 287 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2813115
2024-12-14 21:42:39,281 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 288 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,281 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 288 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.2903912
2024-12-14 21:42:39,290 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 289 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,290 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 289 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.295375
2024-12-14 21:42:39,295 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 300 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,295 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 300 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3054955
2024-12-14 21:42:39,305 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 277 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,305 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 277 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3112357
2024-12-14 21:42:39,311 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 291 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,311 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 291 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3181143
2024-12-14 21:42:39,318 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 290 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,318 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 290 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3278427
2024-12-14 21:42:39,327 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 292 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,328 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 292 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3329508
2024-12-14 21:42:39,333 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 301 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,333 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 301 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3407898
2024-12-14 21:42:39,341 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 293 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,341 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 293 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3492148
2024-12-14 21:42:39,349 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 294 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,349 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 294 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3543978
2024-12-14 21:42:39,354 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 295 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,354 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 295 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3633752
2024-12-14 21:42:39,363 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 296 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,363 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 296 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.368515
2024-12-14 21:42:39,368 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 297 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,368 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 297 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.3914592
2024-12-14 21:42:39,391 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 298 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,391 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 298 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']
Status tracker time of last rate limit error:<PHONE_NUMBER>.4104059
2024-12-14 21:42:39,410 - bespokelabs.curator.request_processor.base_online_request_processor - WARNING - Request 299 failed with Exception litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}, attempts left 0
2024-12-14 21:42:39,410 - bespokelabs.curator.request_processor.base_online_request_processor - ERROR - Request 299 failed permanently after exhausting all 0 retry attempts. Errors: ['litellm.RateLimitError: AnthropicException - {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 output tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}']


Owin startup getting confused between upgrades

See https://our.umbraco.com/forum/extending-umbraco-and-using-the-api/92917-owin-startup-and-2fa

Here's what've done for this case: https://github.com/Offroadcode/Umbraco-2FA/pull/15


Feature request: Support advertising IPv6 Service/Cluster IPs

Feature described here, does not seem to support IPv6
https://docs.projectcalico.org/v3.8/networking/service-advertisement
In the config reference https://docs.projectcalico.org/v3.8/reference/node/configuration it states that CALICO_ADVERTISE_CLUSTER_IPS expects a IPv4 CIDR.
(For the sake of curiosity, putting an IPv6 range into the variable makes nodes failed with: bird: /etc/calico/confd/config/bird_aggr.cfg, line 5: This is an IPv4 router, therefore IPv6 addresses are not supported)
Your Environment

Calico version: 3.8
Orchestrator version: Kubernetes 1.15
Operating System and version: CentOS 7.5

@lwr20 Lance, could you please take a look at this and see if it's a documentation problem or a bug?

Since the docs clearly state that its expecting an IPv4 CIDR and OP has tried putting an IPv6 CIDR in (and it didn't work), this is clearly a feature request, no?
What's happening is that the value in this variable is being copied into the ipv4 bird config.  There's a separate bird config for IPv6.
Speculating now, but for dual-stack operation, it might be required to have both IPv4 and IPv6 CIDRs - so the enhancement would probably be to add an additional env var for IPv6 and have that value make its way into the IPv6 bird config.  @neiljerram would likely know more.

@DennisGlindhart @lwr20 @rafaelvanoni Yes, I can confirm that service cluster IP advertisement is currently for IPv4 only.  It shouldn't be hard to extend to IPv6, so long as all the other ingredients play ball.  For example, does k8s svc.Spec.ClusterIP now allow an IPv6 address instead of IPv4?

@lwr20 yes it is a feature request, but I knew you had done some IPv6 testing and wanted to make sure that we really didn't support it yet (I couldn't remember who had added service advertisement).
@neiljerram Thanks for confirming.


'TypeError: init_from_dict() got an unexpected keyword argument 'matrix'

I am using panpipes to analyze CITEseq data, and managed to run the ingest workflow and get the resulting h5mu file. However, when I try to load the resulting 'x_unfilt.h5mu' file in a jupyter notebook using 'muon.read_h5mu()', I get an error that says: ' 'TypeError: init_from_dict() got an unexpected keyword argument 'matrix'.

Hi @Yohana-Berhe i can't replicate your issue with the environment you provided in issue #126.
have you checked if the mudata is not corrupted? thanks!


davmail: 4.8.6 -> 5.2.0

Motivation for this change
5.2.0 announcement:
https://sourceforge.net/p/davmail/news/2019/02/davmail-520-released/
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

The UI appears when running davmail, but I haven't tried pointing at a
proper server or otherwise actually seeing it in action.

(reviewers suggested by github, wasn't sure :innocent:)


CustomEvent type cannot be used with addEventListener

I have a function that dispatches a CustomEvent.  I then do things based on what the detail is from the custom event.  Unfortunately EventTarget#addEventListener's EventHandler only accepts an Event.
Example that might make more sense:
// @flow

function eventHandler(e: CustomEvent) {
  if (e.detail.body === 'Specific Text') {
    // perform a certain action
  }
}

document.addEventListener('eventType', eventHandler);

const ce = new CustomEvent('eventType', {
  detail: { body: 'Specific Text' },
});

document.dispatchEvent(ce);

I either get an error on my usage of eventHandler when I e is a CustomEvent, or an error checking e.detail if e is an Event.
I'll have a crack at changing this, but I've only just started using flow, so I'll probably come up short.

This is because of the in-built types for document.addEventListener.
You have a few ways of working around it.

You could override the types for document.addEventListener:

(document.addEventListener: Function)('eventType', eventHandler);

You can just use a  // $FlowFixMe comment and ignore it.

You use a more open-ended type for your function:

function eventHandler(e: mixed) {
  if (e instanceof CustomEvent){
    if (e.detail.body === 'Specific Text') {
      // perform a certain action
    }
  }
}

I get that.  But I think that surely, if EventHandler can accept the CustomEvent (which is a subclass of Event that just takes a detail), then the EventHandler definition should take either Event or CustomEvent.

@deecewan I guessed I missed the part where CustomEvent is a subclass of Event. Then it might be an error to do with property variance: https://flowtype.org/blog/2016/10/04/Property-Variance.html
Still, this is something that I'll look into. Reopening.

Why is this closed? @nmn. document.addEventListener() should accept any string as event type for custom events.

This can't be fixed either any string will be always custom event or just event

This can't be fixed either any string will be always custom event or just event

Here is the fix (see revisions): https://gist.github.com/doasync/00fd970ce06dffc6ef9c32c5f5909d17


AutoData attribute doesn't work with "NUnit Test Adapter"

The AutoData attribute doesn't work when tests are executed by the NUnit Test Adapter Visual Studio extension. Here is how I found out:

Install the NUnit Test Adapter extension using the Extension Manager in Visual Studio.
git clone https://github.com/AutoFixture/AutoFixture.git
devenv AutoFixture\Src\AutoFixture.NUnit2.sln
Ctrl+R, A or Test -> Run -> All Tests in Visual Studio.
All tests in Ploeh.AutoFixture.NUnit2.UnitTest.Scenario class fail.

I suspect a similar problem may exists for the NUnit Test Adapter NuGet package, however I haven't verified it.

Is this a known problem? I've looked at the discussion in the original pull request #158, but couldn't tell if this was meant to work.

As far as I can tell, NUnit has no intrinsic knowledge of the AutoDataAttribute and AutoFixture currently relies on the AutoFixture.NUnit2.Addins to make it work. The original pull request mentions the need to have NUnit installed with an MSI and have the AutoFixture.NUnit2.Addins.dll deployed in the C:\Program Files (x86)\NUnit 2.6.2\bin\addins directory.

Was the option of subclassing NUnit's TestCaseSourceAttribute considered as an alternative? This would require changing NUnit's ProviderReference to support passing the MethodInfo to the target method.

I can't answer any of these questions. If someone else (@gertjvr?) can, I'd appreciate the help.

You dont need nunit installed anymore should work OOTB, the required/missing dll was included in the extention, @olegsych what version of NUnit and NUnit Test Adapter are you using?

Latest version, released on visual studio gallery on 4/7.

@olegsych can you confirm your using the versions below?
NUnit 2.6.4
NUnit Test Adapter <IP_ADDRESS> - Targeting 2.6.4

Visual Studio Ultimate 2013 Version 12.0.40629.00 Update 5
NUnit 2.6.2
NUnit Test Adapter <IP_ADDRESS>

Closing due to lack of activity.


onPopPage is deprecated

Hi there!
Just when I thought this package was feature-complete, the new Flutter update 3.24.0 changes the Navigator's API 😅
Documentation for reference: https://docs.flutter.dev/release/breaking-changes/navigator-and-page-api
Basically, the onPopPage should now only be used for cleaning up the page's stack.
However, the change also does not send the popped Route's result back to the callback, so we cannot complete the LocationState's Completer (and thus we are unable to correctly send the navigation result back to the caller).
I tried updating it to the new API, but I'm a bit lost on how to now correctly send the result back, so I'm calling on the hive mind for any suggestions ;)

Issue for go_router dealing with the same: https://github.com/flutter/flutter/issues/152504

Closed in #28