memory is increasing until out of memory when training ShareGPT4v

memory is increasing until out of memory when training ShareGPT4v .
File "/search/odin/licong/InternLM-XComposer/projects/ShareGPT4V/share4v/train/train_mem.py", line 14, in
train()
File "/search/odin/licong/InternLM-XComposer/projects/ShareGPT4V/share4v/train/train.py", line 1019, in train
trainer.train()
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train
return inner_training_loop(
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 1809, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 2665, in training_step
self.accelerator.backward(loss)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/accelerate/accelerator.py", line 1847, in backward
self.deepspeed_engine_wrapped.backward(loss, **kwargs)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 167, in backward
self.engine.backward(loss, **kwargs)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1955, in backward
self.optimizer.backward(loss, retain_graph=retain_graph)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 2135, in backward
self.loss_scaler.backward(loss.float(), retain_graph=retain_graph)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 63, in backward
scaled_loss.backward(retain_graph=retain_graph)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/function.py", line 274, in apply
return user_fn(self, *args)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/utils/checkpoint.py", line 157, in backward
torch.autograd.backward(outputs_with_grad, args_with_grad)
File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.90 GiB (GPU 0; 39.59 GiB total capacity; 28.77 GiB already allocated
; 3.20 GiB free; 34.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid
fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CON

memory is increasing until out of memory when training ShareGPT4v .
File "/search/odin/licong/InternLM-XComposer/projects/ShareGPT4V/share4v/train/train_mem.py", line 14, in train() File "/search/odin/licong/InternLM-XComposer/projects/ShareGPT4V/share4v/train/train.py", line 1019, in train trainer.train() File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train return inner_training_loop( File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 1809, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/trainer.py", line 2665, in training_step self.accelerator.backward(loss) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/accelerate/accelerator.py", line 1847, in backward self.deepspeed_engine_wrapped.backward(loss, **kwargs) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 167, in backward self.engine.backward(loss, **kwargs) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn ret_val = func(*args, **kwargs) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1955, in backward self.optimizer.backward(loss, retain_graph=retain_graph) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn ret_val = func(*args, **kwargs) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 2135, in backward self.loss_scaler.backward(loss.float(), retain_graph=retain_graph) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 63, in backward scaled_loss.backward(retain_graph=retain_graph) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/_tensor.py", line 487, in backward torch.autograd.backward( File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/init.py", line 200, in backward Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/function.py", line 274, in apply return user_fn(self, *args) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/utils/checkpoint.py", line 157, in backward torch.autograd.backward(outputs_with_grad, args_with_grad) File "/search/odin/anaconda3/envs/share4v/lib/python3.10/site-packages/torch/autograd/init.py", line 200, in backward Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.90 GiB (GPU 0; 39.59 GiB total capacity; 28.77 GiB already allocated ; 3.20 GiB free; 34.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CON

In our experiments, we utilized 2x8 A100 (80G) GPUs and the ZERO2 setting. Maybe you can use the ZERO3-offload setting and the gradient accumulation technique. Do not forget to keep the original global batch size.


Add support for external Alertmanager

A lot of orgs have kube-prometheus-stack deployed and already have alertmanager running. It seems redundant to run 2 separate deployments of alertmanager and have to manage routes, templates, etc for both.
Can you provide support for specifying external alertmanager? Also it would be great if there is a programatic way to create alerts from yaml files.

agree

Hi @bsgrigorov,
SigNoz does not support integration with prometheus/alertmanager yet. The SigNoz alertmanager is a customized fork of the prometheus/alertmanager.
You can request support for prometheus/alertmanager here: https://github.com/SigNoz/signoz/issues


ARROW-4629: [Python] Pandas arrow conversion slowed down by imports

The local imports slow down the conversion from pandas to arrow significantly (see here)

Codecov Report

Merging #3706 into master will decrease coverage by 21.31%.
The diff coverage is 57.14%.

@@             Coverage Diff             @@
##           master    #3706       +/-   ##
===========================================
- Coverage   87.79%   66.48%   -21.32%
===========================================
  Files         688      323      -365
  Lines       84280    48123    -36157
  Branches     1081        0     -1081
===========================================
- Hits        73994    31993    -42001
- Misses      10175    16130     +5955
+ Partials      111        0      -111

Impacted Files
Coverage Δ

python/pyarrow/pandas_compat.py
96.3% <100%> (-0.04%)
:arrow_down:

python/pyarrow/array.pxi
68.75% <50%> (-0.45%)
:arrow_down:

cpp/src/arrow/pretty_print.h
0% <0%> (-100%)
:arrow_down:

cpp/src/parquet/murmur3.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/testing/util.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/io/hdfs-internal.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/util/sse-util.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/testing/random.h
0% <0%> (-100%)
:arrow_down:

cpp/src/parquet/hasher.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/util/int-util.cc
0.39% <0%> (-99.21%)
:arrow_down:

... and 512 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e61bcf...eb5c8ba. Read the comment docs.

I'm surprised we're importing Pandas inconditionally. We probably shouldn't do that, as Pandas is quite slow to import:
$ time python -c "import numpy"

real	0m0,097s
user	0m0,077s
sys	0m0,020s

$ time python -c "import pandas"

real	0m0,300s
user	0m0,272s
sys	0m0,028s

Here is a comparison of PyArrow import time with and without Pandas:
$ time python -c "import pyarrow"

real	0m0,360s
user	0m0,305s
sys	0m0,037s

$ time python -c "import sys; sys.modules['pandas'] = None; import pyarrow"

real	0m0,144s
user	0m0,124s
sys	0m0,020s

=> more than twice faster without.

@pitrou Any ideas on how to avoid these local imports and also have the benefit of only loading pandas when needed?

Imports are reasonably cheap once the module is already loaded, but it's probably better to avoid doing them in a tight loop. So hoisting the import outside of critical loops should be sufficient.
>>> %timeit import pandas
102 ns ± 0.497 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
>>> %timeit import pyarrow.pandas_compat as pdcompat
253 ns ± 7.27 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This is in pure Python, though. Cython does not seem to implement the same optimizations as CPython does:
>>> %timeit lib._noop_bench()
66.3 ns ± 0.379 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
>>> %timeit lib._import_bench()
928 ns ± 19.2 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

@pitrou Are you happy with the change here and will deal with the Pandas import issue separately or should this patch be adapted before merging?

Yes, I've created ARROW-4637 for the pandas import.

@xhochy @pitrou we could create a singleton object that exposes the pandas APIs that we need. The imports will be performed on creation of the singleton


use C++ I/O instead of C I/O

We are using a mix of C and C++ I/O, we should stick to one in order to
be consistent.

@ThibaultRiviere Because std::endl is not equivalent to '\n': it also call std::flush.
That's not an issue on std::cout (which is usually line-buffered, so '\n' trigger a flush anyway) and std::cerr (which is usually not buffered) but it can be a performance killer when writing into files & co.
So in order to be consistent (and to "say what I mean and mean what I say") I tend to use '\n' when I want to insert a newline and explicitely flush when I want it. No surprise whatever the output stream.


File Manager share play delete

So I was thinking we might have a file manager interface on the bottom bar. The user can play / share / delete recordings, as well as use it as a "backing track" for playing solos (yay!)
implement using exoplayer

Add volume slider

File manager  / delete implemented in 2bc688d

Done except Triggered Recording opened separately #11


add Attach documents feature

@osBins pls assign this issue to me

Sounds like a good addition to the chatApp. Assigning this to you @sushovanb02!


community: introduce mlflow output parsers

In MLflow 2.13.0, we will introduce RAG model signatures defined as dataclasses to support extensibility.
This PR introduces matching output parsers for those model signatures, such that the users don't have to reason about how to transform their chains into MLflow RAG-compatible formats.
return (
    {
        "question": itemgetter("messages") | RunnableLambda(extract_question),
        "chat_history": itemgetter("messages") | RunnableLambda(extract_history),
    }
    | RunnablePassthrough()
    | {
        "relevant_docs": generate_query_to_retrieve_context_prompt
        | chat_model
        | StrOutputParser()
        | retriever,
        "chat_history": itemgetter("chat_history"),
        "question": itemgetter("question"),
    }
    | {
        "context": itemgetter("relevant_docs") | RunnableLambda(format_context),
        "chat_history": itemgetter("chat_history"),
        "question": itemgetter("question"),
    }
    | question_with_history_and_context_prompt
    | ChatDatabricks(
        endpoint=rag_config.get("llm_model"),
        **rag_config.get("llm_parameters"),
    )
    | ChatCompletionsOutputParser()
)

@efriis @baskaryan could one of you take a look at this PR please? Thank you!

Howdy! Could you add some documentation of how this might be used?
Because these are so lightweight, I actually think these will introduce more maintenance cost than ease of use. Instead, it might be better to just add some documentation where this is intended to be used and recommend using a Runnable Lambda like:
from mlflow.models.rag_signatures import StringResponse

chain = llm | lambda x: StringResponse(text=x)

Hey @efriis, I tried out the approach you described, but I think its a little harder because you need to now extract data from the AIMessage yourself into the StringResponse object. As I understand it, using the BaseTransformOutputParser makes that part easy.
I wrote a chain like this:
      ...
      | chat_model
      | RunnableLambda(lambda x: StringResponse(content=x))
    )

but that will create an object like this:
ChatCompletionResponse(choices=[ChainCompletionChoice(index=0, message=Message(role='user', content=AIMessage(content='\nDatabricks provides several tools for exploratory data analysis (EDA), including:\n\n1. Databricks notebooks: These are collaborative, interactive coding environments where you can perform data manipulation, analysis, and visualization using Python, SQL, Scala, and R.\n\n2. Data visualization tools: Databricks notebooks support various data visualization libraries like matplotlib, seaborn, and Plotly, enabling you to create charts, plots, and graphs to better understand your data.\n\n3. Delta Live Tables: A no-code/low-code data pipeline creation tool that simplifies ETL and data transformation processes, ensuring data quality and reliability.\n\n4. Structured Streaming: A powerful tool for building streaming, incremental, and real-time workloads, allowing you to analyze and process data as it arrives.\n\n5. Summary statistics generation: Databr', response_metadata={'prompt_tokens': 1572, 'completion_tokens': 200, 'total_tokens': 1772}, id='run-61dbb62c-51cb-4ffe-a02e-426a6dd1dbe9-0')), finish_reason='stop')])

Great catch for chat models! Thoughts on:
from mlflow.models.rag_signatures import StringResponse

chain = llm | lambda x: StringResponse(text=x.content)

for chat models (and the previous one for llms)

Great catch for chat models! Thoughts on:
from mlflow.models.rag_signatures import StringResponse

chain = llm | lambda x: StringResponse(text=x.content)

for chat models (and the previous one for llms)

@efriis , i think the RunnableLambda approach does not work for streaming. See this screenshot -- when i call list(full_chain.stream(..)) I get just one object with the whole response.

if i use the ChatCompletionsOutputParser from my PR, then i am able to get each chunk when i call list(full_chain.stream(..))

Responding in slack

discussed in slack

Oh and if you want to use streaming, you can do
from langchain_core.runnables import chain
from mlflow.models.rag_signatures import StringResponse

@chain
def string_response_output_parser(input):
    yield StringResponse(text=x.content

my_chain = llm | string_response_output_parser


New VS ability to publish without Docker Desktop

This entire end-to-end is now possible with VS 17.5 and .NET 7 without using Docker Desktop.
.NET 7 is now able to produce docker images without using Docker Desktop, learn more here:
https://devblogs.microsoft.com/dotnet/announcing-builtin-container-support-for-the-dotnet-sdk/
VS 17.5 has added support for this capability:

Let me know if you have any more questions, I'm the person to work with.
Thank you
Angelos

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b92e9376-e663-c07a-6091-6d700e20fbf9
Version Independent ID: bde516e6-2e34-72f0-2195-0d7124d2994b
Content: Deploy to Azure Container Apps using Visual Studio
Content Source: articles/container-apps/deploy-visual-studio.md
Service: container-apps
GitHub Login: @alexwolfmsft
Microsoft Alias: alexwolf

@AngelosP
Thanks for your feedback! We will investigate and update as appropriate.

@AngelosP  Is a request to write an article about building containers?  How would it tie into Container Apps?

@AngelosP I can update this to remove the dependency on docker desktop

This update has been included in an upcoming PR.
#please-close


RealSense D455 Simulataneous RGB streams

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
{ R200 / F200 / SR300 / ZR300 / D400 }

Firmware Version
(Open RealSense Viewer --> Click info)

Operating System & Version
{Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS

Kernel Version (Linux Only)
(e.g. 4.14.13)

Platform
PC/Raspberry Pi/ NVIDIA Jetson / etc..

SDK Version
{ legacy / 2.. }

Language
{C/C#/labview/nodejs/opencv/pcl/python/unity }

Segment
{Robot/Smartphone/VR/AR/others }

Issue Description
Does the new RealSense 455 support simulantenous RGB streams from the left and right cameras?  If so, could you point me to how I can enable them?

Hi @Hyjale  You can obtain RGB from the left infrared of the D455 with firmware version <IP_ADDRESS> or newer.  The right infrared only offers infrared Y8 or Y16.
Left infrared

Right infrared


Review attack surface reduction events in Windows Event Viewer

In the section "Review attack surface reduction events in Windows Event Viewer" controlled folder access is mentioned few times,
am I wrong or could it be that we are referring to the ASR rules?
Thank you

@msbemba

@MatteoC5 Thank you for your feedback. I hope you are referring to https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/attack-surface-reduction?view=o365-worldwide. If yes, all are related to controlled folder access.
Hope this helps!
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @MatteoC5 Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Is there a way to count how many people have been detected in faceDetect.cpp

I am running the code faceDetect.cpp , i need to count how many people are beign detected , i can detect how many people are in in the same frame by incrementing a variable for each face detected but in the case for example if the person appear , disapear and re-appear  i couldn't find a way to count it as just one person
any hints ?
poke again :  @avdmitry and @vpisarev  :vertical_traffic_light: please stop here :fast_forward:

You should direct questions like this to the OpenCV Q&A forum on https://answers.opencv.org/questions/ where a lot of people will help you way sooner then here. Normally if you apply face detection, then the size of the returned objects vector is the amount of people detected. For tracking a person over time you will need a tracker that remembers positions and gets updated by detections, like the Kalman filter.

yes i'm not interrested in the size of the returned objects because ir can give more than 10 persons for only one face beign detected so it's not the way to do it ..... ++ i'm very interrested to the second solution using kalman filter for tracking a person over time , it's a lot complex , do you have any ressources on that or implementation .
thanks :neckbeard:

"yes i'm not interrested in the size of the returned objects because ir can give more than 10 persons for only one face beign detected so it's not the way to do it ....."
I do not agree, if you built a robust face detector, then it will not yield 10 detections for each individual... Just look for good parameters and apply some post filtering on detections to know if this is actually a face image or not.
As to the tracking stuff, look at this Kalman filter example: http://opencvexamples.blogspot.com/2014/01/kalman-filter-implementation-tracking.html

yes you right i must try to filter out the frames maybe using kalman filter or just a simple filter , thanks you

Then I suggest closing this topic down :)

Ook :100:


"visit_in_email" and "click_first_link_in_email" throw ArgumentError

Using email_spec with cucumber like so:
[…]
Then I should see "Sign in"
And I should see "You will receive an email with instructions […]"
And I should receive an email
When I open the email
Then I should see "Change my password" in the email body

When I follow "Change my password" in the email

Then I should […]

will throw
wrong number of arguments (1 for 2) (ArgumentError)
Replacing
When I follow "Change my password" in the email
with
When I click the first link in the email
will throw
wrong number of arguments (0 for 1) (ArgumentError)
The corresponding steps come straight from the generator:
When(/^(?:I|they) follow "([^"]*?)" in the email$/) do |link|
visit_in_email(link)
end
When(/^(?:I|they) click the first link in the email$/) do
    click_first_link_in_email
end

As a work–around I've specified "current_email" explicitly and changed the steps like so:
When(/^(?:I|they) follow "([^"]*?)" in the email$/) do |link|
    visit(parse_email_for_link(current_email, link))
end

When(/^(?:I|they) click the first link in the email$/) do
    click_first_link_in_email(current_email)
end

— which seems to work so far. Any ideas?
Cheers.

were you able to resolve that? i tried including email_spec after pick is loaded and still getting the same error

@ilyakatz I think I resolved it by updating either pickle and/or email_spec. We're using email_spec 1.2.1 and pickle 0.4.8.


SSL Cert error

Hello,
This is a great project!  Thanks for starting it as I was thinking of doing my own version of this, but I now see I would have been over my head.
I've been able to get HTTP working; however, when I make the same request via HTTPS it has issues (TLS handshake error from <MY HOME IP>:55473: ondemand certificate retrieval is not enabled).  Do you have any pointers?
Client Window
# ssh -p 2222 -R foo:80:localhost:80 tunnel.<REMOVED>.com
Press Ctrl-C to close the session.

Starting SSH Forwarding service for http:80.
Thanks, I truly appreciate the kind words! If you're using dnsrobocert, I assume you're trying to make use of a wildcard certificate with sish? If that's the case, have you setup the the symlinks into the ssl directory that sish uses for certificates? I'm referencing this portion of the readme.
Hope that helps. Feel free to respond here or I'm also idling in IRC.

Closing as @lastcoolnameleft and I resolved this over IRC. The symlinks to the certs weren't correct so they were never being loaded.

Thanks again for your support!
I've created a PR which I think would have helped me during the setup:  https://github.com/antoniomika/sish/pull/104


Use SupportsFloat in step_api_compatibility

The problem is that gym.Env defines step return as Tuple[..., SupportsFloat, ...], and this then can't be passed into functions of step_api_compatibility. Alternative to my fix would be to define centrally what is a step return, and reuse the type alias in both places.
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Shouldn't we be starting to deprecate or remove the step api compatibility thing? It makes sense in gym, but technically speaking, Gymnasium never had a different step API

Shouldn't we be starting to deprecate or remove the step api compatibility thing? It makes sense in gym, but technically speaking, Gymnasium never had a different step API

Yes, but I think we should remove it when we remove the compatibility wrappers in v0.29


Add support for team

I have an open PR for this: https://github.com/shortcut-cli/shortcut-cli/pull/306


readme: tweak signatures to indicate that methods are not curried

Ha ha, third PR, so hard to notice this stuff. 👍

Could someone with write access merge this pull request? I don't like to merge my own pull requests.


Give explanations of memory settings

What do these changes do?

Related issue number

Related to #1121

@lidongze0629 Could you see if these explanations are accurate? And is there anything to add?


Rework log limits

Currently we're only storing 1,000 logs max across all verbosity levels. If a user logs in after a few days or a week, and their app is set to view only level 1 logs, they likely will see no logs at all, or very few, because the 1,000 limit will have been reached by higher verbosity logs and pushed out all or most of the level 1 stuff.
This results in an alarming situation at first, as it appears the bot perhaps wasn't doing anything, or is offline, etc. And then if the user changes their verbosity settings, they will only see the verbose logs, which aren't useful for the use case of periodically checking in on things at a high level.

I don't believe any customer has complained about log availability in a long time. Closing for now.


Speed up SQL IN using SCALAR_IN_ARRAY.

Main changes:

DruidSqlValidator now includes a rewrite of IN to SCALAR_IN_ARRAY, when the size of
the IN is above inFunctionThreshold. The default value of inFunctionThreshold
is 100. Users can restore the prior behavior by setting it to Integer.MAX_VALUE.

SearchOperatorConversion now generates SCALAR_IN_ARRAY when converting to a regular
expression, when the size of the SEARCH is above inFunctionExprThreshold. The default
value of inFunctionExprThreshold is 2. Users can restore the prior behavior by setting
it to Integer.MAX_VALUE.

ReverseLookupRule generates SCALAR_IN_ARRAY if the set of reverse-looked-up values is
greater than inFunctionThreshold.

Benchmarks follow. Overall planning for large IN is much faster. Two new ones are marked with DNF on master, where I gave up and canceled them after they ran for a few minutes. Those same test cases completed in 100ms each with the patch.
InPlanningBenchmark
===================

inClauseLiteralsCount = 1000
inSubQueryThreshold =<PHONE_NUMBER>
rowsPerSegment = 100

## master

Benchmark                                                        Score    Error  Units
InPlanningBenchmark.queryEqualOrInSql                          734.148 ± 46.319  ms/op
InPlanningBenchmark.queryInSql                                 243.272 ± 59.093  ms/op
InPlanningBenchmark.queryJoinEqualOrInSql                      758.371 ± 60.192  ms/op
InPlanningBenchmark.queryMultiEqualOrInSql                     739.495 ± 21.526  ms/op
InPlanningBenchmark.queryStringFunctionInSql                   484.096 ± 46.358  ms/op
InPlanningBenchmark.queryStringFunctionIsNotNullAndNotInSql        DNF
InPlanningBenchmark.queryStringFunctionIsNullOrInSql               DNF

## patch

Benchmark                                                        Score    Error  Units
InPlanningBenchmark.queryEqualOrInSql                           27.063 ±  2.291  ms/op
InPlanningBenchmark.queryInSql                                  24.686 ±  2.113  ms/op
InPlanningBenchmark.queryJoinEqualOrInSql                       29.158 ±  4.165  ms/op
InPlanningBenchmark.queryMultiEqualOrInSql                      29.845 ±  2.914  ms/op
InPlanningBenchmark.queryStringFunctionInSql                    92.489 ±  6.070  ms/op
InPlanningBenchmark.queryStringFunctionIsNotNullAndNotInSql    104.064 ± 31.440  ms/op
InPlanningBenchmark.queryStringFunctionIsNullOrInSql           100.475 ±  9.404  ms/op

SqlReverseLookupBenchmark
=========================

numKeys = 5000000
keysPerValue = 5000
lookupType = immutable

## master

Benchmark                                                        Score     Error  Units
SqlReverseLookupBenchmark.planEquals                           214.932 ±   5.827  ms/op
SqlReverseLookupBenchmark.planEqualsInsideAndOutsideCase      1613.542 ± 182.853  ms/op
SqlReverseLookupBenchmark.planNotEquals                        224.494 ±  19.920  ms/op

## patch

Benchmark                                                        Score     Error  Units
SqlReverseLookupBenchmark.planEquals                            26.214 ±   1.315  ms/op
SqlReverseLookupBenchmark.planEqualsInsideAndOutsideCase       317.464 ±  19.836  ms/op
SqlReverseLookupBenchmark.planNotEquals                         27.020 ±   1.694  ms/op

I think something got messed up when I pulled the commit from github itself: https://github.com/apache/druid/pull/16388/commits/492c80c5cf56561c5534f41d88a9cb795c6a0f47.
I just fixed it up and force pushed. The only change since the original patch is the new test testNotInOrEqualToOneOfThemExpression.

@asdf2014 thanks for reviewing!
@kgyrtkirk thanks as well! please let me know if you have any additional comments; if not I'll merge this.


Makie on Wayland

Arch Version: 5.7.7-arch1-1
Sway Version: 1:1.4-9
Julia Version: 1.4.1
Makie Version: 0.9.5
I am on arch linux, wayland using the swaywm.
I'm having issues installing and running Makie on Wayland.
Running ]build Makie runs into an error whilst building GLMakie:
Error: Error building `GLMakie`:
│ libGL error: MESA-LOADER: failed to open iris (search paths /usr/lib/dri)
│ libGL error: failed to load driver: iris
│ libGL error: MESA-LOADER: failed to open swrast (search paths /usr/lib/dri)
│ libGL error: failed to load driver: swrast
│ init error of GLFW
│ ERROR: LoadError: OpenGL/GLFW wasn't installed correctly. This likely means,
│ you don't have an OpenGL capable Graphic Card,
│ you don't have the newest video driver installed,
│ or the GLFW build failed. If you're on linux and `]build` GLFW failed,
│ try manually adding `sudo apt-get install libglfw3` and then `]build GLMakie`.
│ If you're on a headless server, you still need to install x-server and
│ proper GPU drivers. You can take inspiration from this article
│ on how to get Makie running on a headless system:
│ https://nextjournal.com/sdanisch/makie-1.0
│ If you don't have a GPU, there is also a Cairo software backend
│ for Makie which you can use:
│ https://github.com/JuliaPlots/CairoMakie.jl.
│ Please check the below error and open an issue at:
│ https://github.com/JuliaPlots/GLMakie.jl.
│ After you fixed your OpenGL install, please run `]build GLMakie` again!
│ GLMakie will still load, but will be disabled as a default backend for Makie
│
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] top-level scope at /home/patricka/.julia/packages/GLMakie/gMxEB/deps/build.jl:63
│  [3] include(::String) at ./client.jl:439
│  [4] top-level scope at none:5
│ in expression starting at /home/patricka/.julia/packages/GLMakie/gMxEB/deps/build.jl:31
│ caused by [exception 1]
│ GLFWError (VERSION_UNAVAILABLE): GLX: Failed to create context: GLXBadFBConfig
│ Stacktrace:
│  [1] _ErrorCallbackWrapper(::Int32, ::Cstring) at /home/patricka/.julia/packages/GLFW/g1nX6/src/callback.jl:43
│  [2] CreateWindow(::Int64, ::Int64, ::String, ::GLFW.Monitor, ::GLFW.Window) at /home/patricka/.julia/packages/GLFW/g1nX6/src/glfw3.jl:487
│  [3] GLFW.Window(; name::String, resolution::Tuple{Int64,Int64}, debugging::Bool, major::Int64, minor::Int64, windowhints::Array{Tuple{UInt32,Int64},1}, contexthints::Array{Tuple{UInt32,Integer},1}, visible::Bool, focus::Bool, fullscreen::Bool, monitor::Nothing, share::GLFW.Window) at /home/patricka/.julia/packages/GLFW/g1nX6/src/glfw3.jl:338
│  [4] top-level scope at /home/patricka/.julia/packages/GLMakie/gMxEB/deps/build.jl:34
│  [5] include(::String) at ./client.jl:439
│  [6] top-level scope at none:5
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892
   Building FFTW ────→ `~/.julia/packages/FFTW/qqcBj/deps/build.log`

Output of glxinfo | grep "OpenGL version":
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.1.3

I have installed the community glfw-wayland version 3.3.2-1 from the AUR.
Is it possible to install Makie on Wayland? If so, what needs to happen?

Looks like: https://github.com/JuliaGL/GLFW.jl/issues/198


Add license information.

Hi @slacy, would you mind clarifying the licensing for this project. I wrote a python wrapper for your code and I want to make sure that it's kosher. https://github.com/datamade/doublemetaphone

When reading the README of this GitHub repo, the license of the double-metaphone library seems to be the same as Perl license -- thanks to "This module is free software; you may redistribute it and/or modify it under the same terms as Perl itself." quote from the original codebase. According to https://dev.perl.org/licenses/ Perl is dual-licensed: GPL 1+ or Artistic license.
Artistic license is available at https://dev.perl.org/licenses/artistic.html -- and according to https://choosealicense.com/licenses/artistic-2.0/ (which is referenced to from GitHub licensing HOWTO) it
allows re-distribution of the library even as a part of a commercial product under liberal conditions not too far from MIT or BSD licenses.
I would expect that the Python wrapper for this library could use the Artistic license as well.


cross platform appcast generator based on avalonia

could you make a gui to create appcasts and automaticly upload the updates and the new appcast file to a ftp server

Hi @furesoft!
That's going to be outside the scope of this project for quite a while, I think. It's not a bad idea; in fact, the idea of helping users get their files uploaded was brought up in a recent PR. However, there is enough on my plate that unless it came in as a contribution, this project probably wouldn't support it. I'm not sure that even if a contribution for it came in I'd accept a full GUI into this repo -- it's some serious feature creep and could be a whole project in and of itself.
I might consider something for my work project DotPurple though, since that is already software towards helping the .NET ecosystem...it might fit better into there.
In any case, I won't close this, in case some others want to pitch in ideas or suggestions or someone else makes it happen. For now, I'm marking it as wontfix.


Update description for Step 1 of the Luhn Algorithm project

Describe the Issue
The main() function needs to elaborated. It is used through out in the curriculum and this is a good place to define what main() is intended for in Python.
The description should mention:

main() is the starting point of program execution.

Any other addition can be discussed in the discussion.
Affected Page
https://www.freecodecamp.org/learn/scientific-computing-with-python/learn-how-to-work-with-numbers-and-strings-by-implementing-the-luhn-algorithm/step-1
Your code

n/a

Expected behavior
see explanation above
Screenshots
No response
System

Device: [e.g. iPhone 6, Laptop]
OS: [e.g. iOS 14, Windows 10, Ubuntu 20.04]
Browser: [e.g. Chrome, Safari]
Version: [e.g. 22]

Additional context
No response

Hi @zairahira ,
This is my first time contributing to an open-source project, so I want to clarify the issue. In addition to mentioning that main() is the starting point of program execution should the description also mention Python's unique approach, where the main() function isn't mandatory due to the name convention but including it provides context to those coming from languages where it is?

Hi @MohamedRahimm, this is the first time main() would be introduced in the curriculum. You can incorporate the concepts you mentioned above as long as the instruction is concise and not too wordy. You can also open a PR with the proposed changes and we'll suggest changes where needed. We look forward to your contribution 👍

Sorry for the repeated references I messed up quite a bit trying to create a pull request😓

main is not a specially named function tho, it's just used to match the __main__ special name. It will works exactly as well for any name of function.
Unless you mention the if statement for if __name__ == '__main__' I don't see any reason to give special attention to the main function

I think mentioning  if __name__ = 'main' is out of place since it's the first step and is unnecessary unless you export the module, to my knowledge. So the updated description would look like this:
Start by declaring a function called main, this will be the entry point of the program. Use the pass keyword to avoid an error

After reviewing the discussion, this updated description would be good to go:
In this project, you will implement the Luhn Algorithm. This algorithm is a formula to validate a variety of identification numbers.

Start by declaring a function called `main`, this will serve as the entry point of the program. Use the `pass` keyword to avoid an error.


Adds tupletype to six

Small update to rdkit.six to bring it slightly more up to date

isn't it possible to refer to the tuple type as 'tuple' both in python 2.6+ and 3+ ?

You're right, I was under the impression that this was from a newer six.py, apparently it is not.  I'll close the request.


Error: unsupported driver for Postgres

I'm trying out Atlas on a Postgres DB running inside a Docker container (mapped to port 5433). When I run the command below, I get the following error as output: Error: unsupported driver "".
atlas migrate diff migration_name \
    --dir "./ent/migrate/migrations" \
    --to "./ent/schema" \
    --dev-url "postgresql://postgres:postgres@localhost:5433/postgres"

I tried looking up on the docs and also on the web but wasn't able to find anything related to this issue. Thanks for taking the time to help out!

Hi @ayaanqui
You need to tell Atlas the driver it needs to use for the --dir and --to flags. This is done using a URL scheme that pretends the actual path (ent://, file://)
atlas migrate diff migration_name \
    --dir "file:///ent/migrate/migrations" \
    --to "ent://ent/schema" \
    --dev-url "postgresql://postgres:postgres@localhost:5433/postgres"

For more examples: https://atlasgo.io/versioned/diff

@rotemtam it wasn't clear what file and ent were from the Ent docs so I got confused. thanks for the quick response


ERR_REQUIRE_ESM error

My webpack config is an esmodule, so because eslint-plugin-import-webpack is using require(), I get:
Resolve error: Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: ./webpack.config.js require() of ES modules is not supported.
And things fail.
Any way you can support ESM webpack config files?

@ljharb is there anyway to get arround this, apart from having to revert my webpack config files.

@kemaakpala no. Stick to CJS (or transpiled ESM) for many reasons, including this issue.

I'm also having this issue, I would like to have my Webpack config file written using ES modules as I see no good reason for having to learn and use two module systems in the same project.
I managed to fix it by removing eslint-import-resolver-webpack and replacing it with eslint-import-resolver-typescript which does not seem to have this problem.

@ljharb Do you think it would be a reasonable idea to allow for passing the Webpack config content directly? That might allow users to transpile the config in any way they want. E.g., something like (pseudocode):
settings: {
  'import/resolver': {
    webpack: {
      config: babel.transpile(myWebpackConfig),
    },
},

I feel like that could solve the issues for those who are willing to make a bit of additional effort. The issue is that once you have a "type": "module" package, it's not a great experience to mix the two module types. In that case, for instance, Webpack will expect your config to be using ESM.

Ahh, nevermind. I found out passing a config object is already possible, so something like the following just works:
import webpackConfig from './webpack.config.js';

...

settings: {
  'import/resolver': {
    webpack: { config: webpackConfig },
  },
},

Yep, it needs to be a static (sync) import, but it's a decent workaround.


NEW FEATURES - More download options

Hello everyone,
I absolutely love FastAnime, but while trying to do some specific things i noticied that there are just some small features missing.
Example, I run jellyfin server and was trying to download 'Dragon Ball Z: Battle of Gods' but when i tried to download that it autoselected Dragon Ball Kai, so to solve this i did fastanime anilist, succesfully found the movie i want, but there were options to only stream and not download, so thats one.
Do this for shows too, and the abiliy to select like using start:end format whitch episodes i want to download, what quality, dub or sub on the anilist menu
Second feature i want to have is for it to use like a dowloaded database file, since after I dowloaded entire Dragon Ball Z eps from 1 to 170 i wanted to dowload the rest but on jellyfin i renamed the files to E01.mp4 etc and now its retrying to download these again.
My solution would be when downloading to add option like useDownloadedDB if i enable that then it would do like i want else it just normaly checks if the file exists before downloading.
AND maybe saving into Jellyfin/plex format would be awesome!
THANKS FOR THE GREAT TOOL!!

Glad you enjoying it. However, on your issue, fastanime download is the one used to download anime and if auto select selects the wrong anime you can manually select by fastanime --no-auto-select download ... . Auto select runs a fuzzy match on the titles to make things more automated and convinient so the is a chance that it may be wrong.

I have been thinking about it. And have come to the decision that I will add a db for downloads. This will have the benefit of better local previews. However, I won't include this in the anilist interfaces. Instead I'll introduce fastanime anilist download  which will just be a superset of fastanime download  except with anilist integration and db integration.
So thanks for the idea.
Though don't expect it to be immediately implemented.
On the rest they do sound nice but I don't have that much time to do all of them since am doing everything alone and fastanime's codebase is getting bigger and bigger due to it having so many features.
I will try to explore more on the jellfin and plex format later and see whether it's something I want in the project and worth my time.

I'm glad you liked the ideas and are planning to add them in the future!
It's exciting to see that you're planning to add a database for downloads and enhance local previews. The idea of creating a FastAnime Anilist download as a superset sounds like a smart approach, especially with the integration. I totally understand that implementing everything takes time, especially since you're handling it all alone. I'm looking forward to seeing how you explore the Jellyfin and Plex format—it's great that you're considering what fits best for the project. Keep up the amazing work!

is there an option to download subtitles if using hianime?

is there an option to download subtitles if using hianime to be able to use subtitle translate?

Use fastanime grab like in the docs
to get the subtitle links then download them

how to download subtitle from all episode?

Imposible and out of scope (you could easily create a script that does this with the help of ai if needed); unless you download the whole video and dont merge the video with subtitles.

unless you download the whole video and dont merge the video with subtitles.

how to use?


Filter on DateTime don't use index

Hello, suppose the following document inside an edge collection  :
{
  "task_id": 460,
  "freq": 1,
  "kind": "web",
  "type": "Social activity",
  "unique": "Organization/480786013375-Person/485359770815",
  "dt_first_seen": "2015-06-25T04:56:58",
  "dt_last_seen": "2015-06-25T04:56:58"
}

fields task_id, dt_first_seen, dt_last_seen are inside a Skip-list sparse index
there is also an hash index on task_id filed
the folllowing AQL, don't use the supposed index
LET x = (
for c in Connections
    filter  ((c.dt_first_seen >= "2015-06-11T17:21:39") && (c.dt_last_seen <= "2015-08-14T13:32:13") && c.task_id == 460)
    return c
)

return x

this is the explain output :
Query string:
 LET x = (
 for c in Connections
     filter  ((c.dt_first_seen >= "2015-06-11T17:21:39") && (c.dt_last_seen <= "2015-08-14T13:32:13")
 && c.task_id == 460)
     return c
 )
 return x

Execution plan:
 Id   NodeType          Est.   Comment
  1   SingletonNode        1   * ROOT
  7   SubqueryNode         1     - LET x = ...   /* subquery */
  2   SingletonNode        1       * ROOT
  9   IndexRangeNode     296         - FOR c IN Connections   /* hash index scan */
  4   CalculationNode    296           - LET #4 = c.`dt_first_seen` >= "2015-06-11T17:21:39" && c.`dt_last_seen` <= "2015-08-14T13:32:13" && c.`task_id` == 460   /* simple expression */   /* collections used: c : Connections */
  5   FilterNode         296           - FILTER #4
  6   ReturnNode         296           - RETURN c
  8   ReturnNode           1     - RETURN x

Indexes used:
 Id   Type   Collection    Unique   Sparse   Selectivity Est.   Fields      Ranges
  9   hash   Connections   false    true               0.34 %   `task_id`   [ `task_id` == 460 ]

Optimization rules applied:
 Id   RuleName
  1   use-index-range

any suggestion ?
tnx in advance

Hi @ppiccolo,
it seems reasonable that the skiplist-index should be preferred over the task_id index.
Could you send us the output of
getIndexes()
on your collection?
This would help us to debug the issue.
best
Michael

Sure ...
db.Connections.getIndexes()
[
  {
    "id" : "Connections/0",
    "type" : "primary",
    "fields" : [
      "_key"
    ],
    "selectivityEstimate" : 1,
    "unique" : true,
    "sparse" : false
  },
  {
    "id" : "Connections/480418618559",
    "type" : "edge",
    "fields" : [
      "_from",
      "_to"
    ],
    "selectivityEstimate" : 0.3823936641559592,
    "unique" : false,
    "sparse" : false
  },
  {
    "id" : "Connections/480419011775",
    "type" : "hash",
    "fields" : [
      "task_id",
      "type",
      "unique"
    ],
    "selectivityEstimate" : 1,
    "unique" : true,
    "sparse" : true
  },
  {
    "id" : "Connections/523655535807",
    "type" : "hash",
    "fields" : [
      "task_id"
    ],
    "selectivityEstimate" : 0.0033783783783783786,
    "unique" : false,
    "sparse" : true
  },
  {
    "id" : "Connections/523971353791",
    "type" : "skiplist",
    "fields" : [
      "task_id",
      "dt_first_seen",
      "dt_last_seen"
    ],
    "unique" : false,
    "sparse" : true
  }
]

I think in this case the skiplist index must not be used because the index is declared sparse.
The skiplist index is created on attributes task_id, dt_first_seen and dt_last_seen, in this order.
The filter condition is task_id == 460 && dt_first_seen >= "2015-06-11T17:21:39" && dt_last_seen <= "2015-08-14T13:32:13".
In general, a skiplist index can be used if all index attributes are used in a query or if a left-most prefix of the attributes is used in the filter condition. However, it needs to stop after the first index attribute which is not an equality condition.
In the above example, the skiplist index can be used to scan the range from [ task_id = 460, dt_first_seen = "2015-06-11T17:21:39" ] to [ task_id = 460, dt_first_seen = max value ]. It cannot use the condition on the third index attribute dt_last_seen.
While in general a skiplist index can support such condition and query the range, in this case it won't be used because it's declared sparse. Sparse indexes will (intentionally) exclude all documents in which any of the index attributes are not set or contain the value null. Because the condition only covers two of the index attributes, the optimizer cannot safely determine that the range does not contain any documents in which one of the index attributes would have been null.
For example, an imagined document { task_id: 460, dt_first_seen: "2015-06-11T17:21:39", dt_last_seen: null } would be included in the range, but it won't be present in the index as the index is declared sparse. Therefore using the index for answering the query would potentially suppress documents like this one, so the optimizer chose to not use the index.
If you want the skiplist index to support that query, the index needs to be declared non-sparse.

Ok have a sense, I'll give it a try with non-sparse index

Solved tnx to all


Create a basic class for slide.ts and master.ts to inherit common methods from

Currently, master.ts and slide.ts use a lot of methods that are equal or almost equal. It would be very nice to have a common basic class for both.
Also see the ToDo-comments in master.ts

This is implemented now, see src/classes/has-shapes.ts


Added support for hyper and meh modifiers

There's two popular custom modifier keys called hyper and meh which are cmd+alt+shift+ctrl and alt+shift+ctrl respectively.
I've added two new modifier types to the modifiersMap global variable. This allows for shorthand syntax in the config file: hyper-h instead of having to write cmd-ctrl-alt-shift-h.

nice

Not sure how "standardized" the definitions for hyper and meh combinations are.
I'm using https://hyperkey.app/ and the options are cmd+alt+ctrl or cmd+alt+ctrl+shift but not alt+shift+ctrl.

I think traditionally OP is correct in their definitions for hyp and meh. It's been this way for as long as I can remember and is standard in multiple keyboard firmwares:

https://github.com/qmk/qmk_firmware/blob/master/docs/keycodes.md (Look at KC_HYPR and KC_MEH)
https://github.com/KMKfw/kmk_firmware/blob/main/docs/en/keycodes.md#modifiers (KC.HYPR, KC.MEH)
https://ergodox-ez.com/our-open-source-firmware (Based on QMK, uses hyper & meh as selling points)

To me, that screenshot of the hyperkey app just seems like it's giving you the option of including shift in your hyper key combo or not. It doesn't seem concerned with the meh key at all (alt+shift+ctrl), so I wouldn't say that is evidence against it being standard.
That being said, do we really need a specific name for these key combos in every program as long as they handle cmd-alt-ctrl-shift-* just fine?

@nikitabobko any updates on this? 🤔

Thanks for the proposal, but sorry, I don't like the idea to hardcode hyper and meh. The next thing users are going to ask is to hardcode another combination of keys.
The proper solution is suggested here: https://github.com/nikitabobko/AeroSpace/issues/224#issuecomment-2111556500

@nikitabobko to be honest I think it's far easier to add all the combinations here rather than having some custom parsing logic code to maintain. Especially since all the APIs that are being used here are system ones and they support assigning multiple modifiers. But ultimately it's your call.
I'll try to find some time to do it through key-notation-to-key-code.


OS X: hyper app always running

[x] I am on the latest Hyper.app version
[x] I have searched the issues of this repo and believe that this is not a duplicate

OS X Mojave 10.14.6:
3.0.2:
Link of a Gist with the contents of your .hyper.js:
N/A (CMD+ALT+I on macOS, CTRL+SHIFT+I elsewhere):
is vanilla:

Issue

In OS X hyper terminal is always running unless I right click it in the Dock and manually quit.  Closing the window does not kill the application.  In addition on startup Hyper is always running.
If this is expected behaviour there should be an option to disable background running.  From what I can tell there currently isn't.
I'm running OS X Mojave 10.14.6 and latest Hyper release 3.0.2.

The manual quit behaviour is default in OS X, and it also reopens apps by default on startup.

@bruc3mackenzi3 can you close this, as this is intended behaviour for macOS.


need help to select GPT-4 for a new chat

Any idea how to select GPT-4 model?

A GET parameter seems selecting the model : https://chat.openai.com/chat?model=gpt-4

Is it enough to set the model like that?
cli := chatgpt.NewClient( 		chatgpt.WithDebug(true), 		chatgpt.WithTimeout(120*time.Second), 		chatgpt.WithCookies(cookies), 		chatgpt.WithModel("gpt-4"), )

enough.I have used this for one day.

me too


Add Host Network Attachment Definition

TL;DR
This PR makes some changes to how we handle Network Attachment Definitions (NAD) in the DPU Operator. The goal is to centralize the NAD setup, and automate the deployment of the NAD for the Host. I'm also hoping to discuss if these changes should be moved to the daemon so we can support auto mode as well.
What’s Changed
Host NAD YAML:
Added a manifest for Host NAD to centralize its configuration in the operator itself.
The namespace follows what we use in Cluster Deployment Automation (CDA), so the default resources get nad properly.
NAD Reconciliation:
Changed the check for NAD to ensure both Host and DPU mode have deployed its NADs while skipping it for Auto mode.
Auto Mode Handling:
If the configuration is set to auto, the operator will skip NAD deployment since the controller can’t detect the hardware in this mode. This keeps things cleaner and avoids unnecessary errors.
Why These Changes Matter
By centralizing the Host NAD setup and improving how we handle reconciliation, this makes the operator more reliable and lessens the burden of configuration from the user. I also hope to have a discussion about whether NAD management should be the job of the controller or the daemon, since the latter is the one that can auto detect the hardware.

As part of this PR I think we should remove creating the nad in CDA, and add a bump to CDA as a commit to make sure e2e tests are still working.


Update language mappings (of Romani dialects) when Glottolog 3.5 is released

Glottolog 3.5 will contain a lot more Romani dialects (see https://github.com/clld/glottolog/pull/354), so the mappings in etc/languages.csv can be completed once the next version of Glottolog is released.

Done in https://github.com/lexibank/diacl/commit/ba8dda7fdad90da388e0387f608f6b3cf42a619b.


add MicroBenchMark model

Tips

Thanks very much for contributing to Apache DolphinScheduler.
Please review https://dolphinscheduler.apache.org/en-us/community/index.html before opening a pull request.

What is the purpose of the pull request
add jmh model,we can have better data support for code optimization or code performance. Later, we can also expand on this basis to facilitate users to do performance tests.
Brief change log
Verify this pull request
This pull request is code cleanup without any test coverage.
Do I need to write a test case?
This change added tests and can be verified as follows:
-- Manually verified the change by testing locally

Codecov Report

Merging #2851 into dev will decrease coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev    #2851      +/-   ##
============================================
- Coverage     37.21%   37.07%   -0.14%
+ Complexity     2560     2551       -9
============================================
  Files           435      435
  Lines         20020    20020
  Branches       2426     2426
============================================
- Hits           7451     7423      -28
- Misses        11904    11928      +24
- Partials        665      669       +4

Impacted Files
Coverage Δ
Complexity Δ

...duler/server/entity/DataxTaskExecutionContext.java
35.00% <0.00%> (-60.00%)
7.00% <0.00%> (-6.00%)

...he/dolphinscheduler/common/thread/ThreadUtils.java
68.25% <0.00%> (-9.53%)
13.00% <0.00%> (-1.00%)

...ver/master/consumer/TaskPriorityQueueConsumer.java
54.41% <0.00%> (-4.42%)
16.00% <0.00%> (ø%)

...e/dolphinscheduler/remote/NettyRemotingClient.java
51.07% <0.00%> (-2.88%)
9.00% <0.00%> (-2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f8ddae...83a9355. Read the comment docs.

It's not rational to run the micro-benchmark in the test, this would drastically slow down the CI builds, suggestion is excluding the micro-benchmark from the test phase

thanks ,I will finish him later

It's not rational to run the micro-benchmark in the test, this would drastically slow down the CI builds, suggestion is excluding the micro-benchmark from the test phase

I have excluded it, please check

It's not rational to run the micro-benchmark in the test, this would drastically slow down the CI builds, suggestion is excluding the micro-benchmark from the test phase

I have excluded it, please check

You just exclude the sonar check

It's not rational to run the micro-benchmark in the test, this would drastically slow down the CI builds, suggestion is excluding the micro-benchmark from the test phase

Sorry, I don’t quite understand what you mean, because it does not contain the test module itself, can you please elaborate

It's not rational to run the micro-benchmark in the test, this would drastically slow down the CI builds, suggestion is excluding the micro-benchmark from the test phase

Sorry, I don’t quite understand what you mean, because it does not contain the test module itself, can you please elaborate

Sorry I missed that, please ignore me

is there a document about how to use it ?

is there a document about how to use it ?

Sorry, I will add related documents on the website later

Look good to me once the related doc is added :)

the how to use doc ： https://github.com/apache/incubator-dolphinscheduler-website/pull/125

good job， looking forward for next contribution


Fix slowness problems II

The brew_* rules were really slow (importing them had a lot of side effect) but they are not well tested. Someone should definitely test them (I don't have an OS X).
This will only buy us a few milliseconds.
Ref #334.

I can work on that later today, what you say?
--
Pablo Aguiar (scorphus<EMAIL_ADDRESS>On Wed, Aug 26, 2015 at 4:46 PM, Martin Carton<EMAIL_ADDRESS>wrote:

Thanks, the coverage of that rule is not really good.
—
Reply to this email directly or view it on GitHub
https://github.com/nvbn/thefuck/pull/352#issuecomment-135150688.

Almost there...
============================= test session starts ==============================
platform darwin -- Python 3.4.3 -- py-1.4.30 -- pytest-2.7.2 -- /Users/pablo/.virtualenvs/thefuck-py3/bin/python3.4
rootdir: /Users/pablo/Workspace/thefuck, inifile:
plugins: benchmark, cov, mock
collecting ... collected 6 items

tests/rules/test_brew_install.py::test_match PASSED
tests/rules/test_brew_install.py::test_get_new_command PASSED
tests/rules/test_brew_unknown_command.py::test_match PASSED
tests/rules/test_brew_unknown_command.py::test_get_new_command FAILED
tests/rules/test_brew_upgrade.py::test_match[command0] PASSED
tests/rules/test_brew_upgrade.py::test_get_new_command[command0-brew upgrade --all] PASSED

=================================== FAILURES ===================================
_____________________________ test_get_new_command _____________________________

brew_unknown_cmd = 'Error: Unknown command: inst'
brew_unknown_cmd2 = 'Error: Unknown command: instaa'

    def test_get_new_command(brew_unknown_cmd, brew_unknown_cmd2):
        assert get_new_command(Command('brew inst', stderr=brew_unknown_cmd),
                               None) == ['brew list', 'brew install', 'brew uninstall']

>       assert get_new_command(Command('brew instaa', stderr=brew_unknown_cmd2),
                               None) == ['brew install', 'brew uninstall', 'brew list']
E       assert ['brew instal...ew reinstall'] == ['brew install..., 'brew list']
E         At index 2 diff: 'brew reinstall' != 'brew list'
E         Full diff:
E         - ['brew install', 'brew uninstall', 'brew reinstall']
E         ?                                          ^^ -  ---
E         + ['brew install', 'brew uninstall', 'brew list']
E         ?                                          ^

tests/rules/test_brew_unknown_command.py:27: AssertionError
====================== 1 failed, 5 passed in 0.22 seconds ======================

Just apply:
diff --git a/tests/rules/test_brew_unknown_command.py b/tests/rules/test_brew_unknown_command.py
index 17adb47..86bec5b 100644
--- a/tests/rules/test_brew_unknown_command.py
+++ b/tests/rules/test_brew_unknown_command.py
@@ -25,4 +25,4 @@ def test_get_new_command(brew_unknown_cmd, brew_unknown_cmd2):
                            None) == ['brew list', 'brew install', 'brew uninstall']

     assert get_new_command(Command('brew instaa', stderr=brew_unknown_cmd2),
-                           None) == ['brew install', 'brew uninstall', 'brew list']
+                           None) == ['brew install', 'brew uninstall', 'brew reinstall']

and we are go.

@scorphus Thanks, that might work on OS X but does not work on Linux anymore (including travis) :p
I probably have fixed it now, but it's mostly a hack with a weaken test. This test should be rewritten to do the same thing on Linux and OS X (or we should ask for OS X testing on travis and not test it at all on Linux) and have a better coverage.

No problem, @mcarton :smile:
py.test said:

635 passed, 52 skipped in 5.06 seconds

Tests tell us we're good to python go :+1:
@mcarton said:

[...] This test should be rewritten to do the same thing on Linux and OS X (...) and have a better coverage.

I can try and work on that :construction:

Can I merge this, or it's in progress?

@nvbn You can merge. scorphus helped my fix the tests on mac and I guess the easiest for him would to make another PR if he wants to improve the brew_* tests.

Just give a chance to test this with zsh, as that other issue states it's slow for some reason. I'll test this in about one hour from now.

And count on me for improving the brew tests

I can confirm this changes make TheFuck faster on both Fish and Zsh. Thank you, @mcarton. :+1:
It LGTM, please merge it, @nvbn

@mcarton, @scorphus Thanks!


Add icons to "new", "show", "edit", and "destroy"

Improve UI by adding an icon before "new __" as well as "show", "edit", and "destroy"

Looks good


WWDMM: can't submit meme with only one line of caption

devices info:
iPod Touch 5thGen/ iOS9.3.2/ safari
Droid Bionic/ android4.0.4/ built-in browser
iPad 1stGen/ iOS5.1.1/ safari

When submit meme with only one line of caption, it won't get submitted. Specific process is: fill in one line of caption, clicking post, post button turn dim, no alert pop up, then this user won't be able to resubmit again and have to wait. If any player does this, the host page will not proceed to scoring unless counting end.

devices info:
iPad4thGen/iOS8.1/safari
iPhone6s/iOS10.0/safari
Mac/ safari

Update: problem persists, same as last time


Undo, Redo, Paste-as-text not working in Gmail

Bug description:
When composing a message in Gmail, the Undo & Redo functionality doesn't work right. Undo behaves erratically, and Redo often doesn't work at all.
Pasting-as-text (i.e., paste without formatting for richtext or HTML clipboard) with Shift-Option-Cmd-V does not work.
Environment:

Operating System: macOS 10.15.4
Ferdi Version: 5.4.3 (<IP_ADDRESS>4)

Cannot reproduce this, and we fixed similar issues not long ago.
Please reply in case this still happens.


Added Colorful Interactive Particles using Javascript

Title and Issue number
Successfully added a Javascript project animation of colorful interactive particles for Issue #3761
Preview

Close #3761

Checklist:

[X] I have mentioned the issue number in my Pull Request.
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have created a helpful and easy to understand README.md

@MohitGupta121 @TusharKesarwani  sir kindly review this PR


Fixes Issue 1829

Describe the PR
Fixes the issue described in https://github.com/swaggo/swag/issues/1829
Relation issue
https://github.com/swaggo/swag/issues/1829
Additional context
The previous logic could select the wrong markdown file if one tag was a substring of another. E.g. "objects" and "my-special-objects".

Hi, you have to fix the failed tests，TestParseDescriptionMarkdown and there may be others.

@sdghchj , have fixed the tests. They have actually caught an issue that would have been introduced. Now I also understand why the code was written like that in the first place. It needs to understand "api" and "api.md". Fixed it, to be able to deal with both inputs.
Your review has been dismissed I think, could you re-review or re-approve?


fix(FX-3231): Aligned articles layout

FX-3231
Before

After

The image should be resized according to the max dimensions it can be displayed at. This will be different for each size.
If it's not difficult, can you explain it in more detail?

Ah, sorry I missed this: https://github.com/artsy/force/blob/09cd0cb3e686e1053113b3edb5e32729e75f48bc/src/v2/Apps/Fair/Components/FairEditorial/FairEditorialItem.tsx#L148-L159
So just inlining the component declarations should be fine.


Host via less NSFW domain name?

Love the collection of logos, but I'm only able to view them via github.com because of "porn" in the name of the domain is blocked. Any way to host with a different domain name?
Again, thank you!

Chiming in here for visibility - I've recently transitioned into a more corporate role where I'm reluctant to share links to the site because the optics aren't ideal - we're working hard to cultivate an environment that's considerate of colleagues' respective comfort zones and porn is a term with which many have a negative emotional association.
Speaking personally, I also just don't like being the guy to share the link because at a glance it could be mistaken for something very different - we have 1500+ people in our company slack and the last thing I need is for folks to get the impression that I'm the sharing something inappropriate.
Big :+1: from me for a domain mirror or name change - thank you for all the terrific work building this collection of beautiful icons!

Any chance to have an alternate domain like https://js.org/ ? could be awesome


Hello

Hello,
I don't understand R language
And can you explain the code and algorithm in Chinese?

No further problem.


*If time - FAQ?

About the team
How did we come up with the idea
How did we work together

created page, waiting for approval - #134


Linkifier for domains in manual

Right now, the manual genre exports a JSON-formatted database of link targets for the information in the documentation data domains. It would be great to have a little page like doc-gen's that redirects from a domain/item pair as URL parameters to the actual URL with #ID, to support Zulip linkifiers etc.

Fixed in #183


replace str.decode / str.encode for base64 and quoted-printable. should fix #20

as said. assumed internal encoding of utf-8, but actually i'm not entirely sure if this is how you handle things. therefore please tell me what to change and i'll do extra commits

@prauscher I had to add bytes() before base64 decoding, see https://github.com/arska/vobject/commit/fd07a9f2f06f5862699982ee47d5d789ac7924a2

@arska thank you, but line.value.encode("utf-8") works in python2 too

+1


ImageAnnotatorClient thread safety

Is ImageAnnotatorClient thread safe? I have several working threads sending OCR requests to google vision cloud. Should create a new instance of ImageAnnotatorClient for every single request?

You're absolutely fine to use a single client everywhere, and indeed that's encouraged :)

@jskeet does this apply to all dotnet clients? Found this looking for info about FirestoreDb thread safety

@moander: Yes, it should be fine for all clients.


Error on PC Startup

Microsoft PowerToys version
<IP_ADDRESS>
Running as admin

[X] Yes

Area(s) with issue?
General
Steps to reproduce
Version: <IP_ADDRESS>
OS Version: Microsoft Windows NT 10.0.22000.0
IntPtr Length: 8
x64: True
Date: 15-07-2022 21:31:52
Exception:
System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified.
File name: 'System.Windows.Forms, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b77a5c561934e089'
at PowerLauncher.MainWindow.WindowTop()
at PowerLauncher.MainWindow.InitializePosition()
at PowerLauncher.MainWindow.OnLoaded(Object sender, RoutedEventArgs e)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.UIElement.RaiseEvent(RoutedEventArgs e)
at System.Windows.BroadcastEventHelper.BroadcastEvent(DependencyObject root, RoutedEvent routedEvent)
at System.Windows.BroadcastEventHelper.BroadcastLoadedEvent(Object root)
at System.Windows.Media.MediaContext.FireLoadedPendingCallbacks()
at System.Windows.Media.MediaContext.FireInvokeOnRenderCallbacks()
at System.Windows.Media.MediaContext.RenderMessageHandlerCore(Object resizedCompositionTarget)
at System.Windows.Media.MediaContext.RenderMessageHandler(Object resizedCompositionTarget)
at System.Windows.Media.MediaContext.Resize(ICompositionTarget resizedCompositionTarget)
at System.Windows.Interop.HwndTarget.OnResize()
at System.Windows.Interop.HwndTarget.HandleMessage(WindowMessage msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Interop.HwndSource.HwndTargetFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
✔ Expected Behavior
No response
❌ Actual Behavior
No response
Other Software
No response

Did it occur only once or are you able to repro it? /needinfo
Must be a duplicate of https://github.com/microsoft/PowerToys/issues/18275

Happened only once, and hasn't reappeared since. As such, there was no user input before the error occurred - I booted up my PC, logged in, and this error popped up while my startup programs were loading up. The only input I gave my PC in this entire process was entering my password.

Thank you for the clarification.
Closing this as /dup #19482


V4 new Promise with arrow notation breaks bundle [BUG]

fuse-box  4.0.0-next.156
Writing a new Promise with arrow gets transpiled wrong:
Input
const shutdownHttpServer = () =>
  new Promise((resolve) => {
    server.close(() => {
      resolve();
    });
  });

Output
const shutdownHttpServer = () => new Promise((__req9__.resolve) => {
  server.close(() => {
    __req9__.resolve();
  });
});

Which results in the following error:
const shutdownHttpServer = () => new Promise((__req9__.resolve) => {
                                              ^^^^^^^^^^^^^^^^
SyntaxError: Invalid destructuring assignment target

However writing it like this:
Input
const shutdownHttpServer = () =>
  new Promise(function(resolve) {
    server.close(() => {
      resolve();
    });
  });

Output
const shutdownHttpServer = () => new Promise(function (resolve) {
  server.close(() => {
    resolve();
  });
});

@FKobus let me guess, you have
import resolve from "module"

Yeah, that's a bug in next it has been resolved in alpha. I will see how I can fix master

I don't have any imports to resolve from module.

@FKobus that's impossible. It wouldn't link it to req9 then

4.0.0-next.157


[Fix]: Bank account is not removed from wallet page if we remove it from workspace workflows page

Details

Currently when we remove bank account from the workspace workflows page, it is not deleted from the wallet page, it is only reset, in this PR we fix that by deleting the Bank account
Fixed Issues

$ https://github.com/Expensify/App/issues/39355
PROPOSAL: https://github.com/Expensify/App/issues/39355#issuecomment-2029815993
Tests

Same as QA

[x] Verify that no errors appear in the JS console

Offline tests

Cannot be performed offline
QA Steps

Preconditions
No bank account added in wallet page and the workspace has enabled workflows

Open any workspace.
Go to workflows
Click on Connect Bank Account (If stuck use the slack discussion here)
Add a bank account using chase ending 1111
After Adding bank account, verify that the same bank account is reflected on Wallets Page
Go back to the workspace where you have added the bank account
go to workflows
Delete the bank account from there
Go back
Go to wallet page
Verify that the bank account has been deleted from there too

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/110545952/c84433eb-14e0-4d77-8afc-e1ca73fd800a

MacOS: Desktop
https://github.com/Expensify/App/assets/110545952/219ec0db-2eeb-4db5-bd46-d7de69af5c20

iOS: Native
https://github.com/Expensify/App/assets/110545952/70f04b7a-6b89-436b-823f-6f5e32dceca0

iOS: mWeb Safari
https://github.com/Expensify/App/assets/110545952/a335bd28-a777-4f08-b1ed-09344a646a1d

Android: Native
https://github.com/Expensify/App/assets/110545952/573e6110-dd75-4f1c-87c5-dcdb25123c70

Android: mWeb Chrome

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop


Document the ability to open iOS/iPhone Photos libraries.

You've already implemented it in #791 so why not document it?
For me, ifuse is plenty fast. Maybe you could use pymobiledevice3 to get an easy UX.
Do you have a command for exporting and then deleting photos? What about one that deletes them from the filesystem too?

This ability is documented in the docs for the sync command as that's the only command that can easily make use of an iPhone Photos database. It's not possible, as far as I know, to use the export command from an iPhone as the photo paths are different and osxphotos doesn't account for this.
pymobiledevice3 looks interesting but it's licensed under GPL which isn't compatible with osxphotos so it's not something I can incorporate in osxphotos.
If you have a good workflow for using osxphotos with an iPhone, you're welcome to write it up and I can include it in the docs. I suggest starting with documenting it in our discussions pages.

Do you have a command for exporting and then deleting photos? What about one that deletes them from the filesystem too?

osxphotos cannot delete photos from Photos. What I suggest is to use the --add-to-album feature to add the exported photos to an album in Photos and then you can delete them from there. (If you use Command+Delete on photos in an album, they will be sent to the Recently Deleted folder. If you use only Delete, they are only removed from the album but not the library).

Can Photos on macOS or iOS fix a corrupted database by itself? Will all metadata get lost if osxphotos or anything corrupts or deletes the database? Will all photos get orphaned? How about orphaned photos or photos missing from the filesystem but which are still in the database? Have you tested that?

Can Photos on macOS or iOS fix a corrupted database by itself?

Yes. On macOS, see repair Photos library. No idea about iOS since that's a black box.

Will all metadata get lost if osxphotos or anything corrupts or deletes the database?

osxphotos doesn't write to database normally so querying or exporting with osxphotos cannot corrupt the database. The only osxphotos tool that writes directly to the database is osxphotos timewarp and this warns the user to make a backup then prompts the user to continue. osxphotos import and osxphotos sync can make changes but these are all done through the public Photos AppleScript interface so any corruption would be the responsibility of Photos.
I have seen Photos corrupt a database but I've never seen metadata lost. Usually running the Photos library repair tool is sufficient for fixing the database.

Will all photos get orphaned? How about orphaned photos or photos missing from the filesystem but which are still in the database? Have you tested that?

I don't know what would cause photos to get orphaned. osxphotos does have a osxphotos orphans tool that finds orphaned files (in the filesystem but not referenced in the database) and can export these with osxphotos orphans --export.


Registry tests: use of environment variables in test setup clutters logs

This code configures clients using the old (and deprecated) environment variables. I believe we now have a way to set this directly into the config.
With the current environment variable configuration, many warnings are printed that clutter the test logs:
$ go test ./cmd/registry/cmd/apply -v
2022/12/06 13:21:48 Client will use an embedded registry with a SQLite3 database
  WARN[0000] Slow database operation.  duration=188.747493ms query=CREATE TABLE `projects` (`key` text,`project_id` text,`display_name` text,`description` text,`create_time` datetime,`update_time` datetime,PRIMARY KEY (`key`))
=== RUN   TestApply
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0000] Applying testdata/sample/apis/registry.yaml
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0001] Applying testdata/sample/artifacts/lifecycle.yaml
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0001] Applying testdata/sample/artifacts/manifest.yaml
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0001] Applying testdata/sample/artifacts/taxonomies.yaml
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0001] Applying testdata/sample/artifacts/styleguide.yaml
--- PASS: TestApply (1.40s)
=== RUN   TestApplyProject
=== RUN   TestApplyProject/bundled
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
WARN: deprecated env: APG_REGISTRY_ADDRESS="localhost:38091"
WARN: deprecated env: APG_REGISTRY_INSECURE="1"
  INFO[0002] Applying testdata/sample/apis/registry.yaml
  INFO[0002] Applying testdata/sample/artifacts/lifecycle.yaml  INFO[0002] Applying testdata/sample/artifacts/taxonomies.yaml
  INFO[0002] Applying testdata/sample/artifacts/styleguide.yaml
  INFO[0002] Applying testdata/sample/artifacts/manifest.yaml

I was able to configure the tests without environment variables by adding this code to grpctest.NewServer:
	// set registry configuration for internal client
	addr := fmt.Sprintf("localhost:%d", s.Port())
	config, err := connection.ActiveConfig()
	if err != nil {
		return nil, err
	}
	config.Insecure = true
	config.Address = addr
	connection.SetConfig(config)

All tests pass except for a controller test:
$ go test ./cmd/registry/cmd/resolve -v
2022/12/14 09:02:00 Client will use an embedded registry with a SQLite3 database
  WARN[0000] Slow database operation.  duration=199.00794ms query=CREATE TABLE `projects` (`key` text,`project_id` text,`display_name` text,`description` text,`create_time` datetime,`update_time` datetime,PRIMARY KEY (`key`))
=== RUN   TestResolve
=== RUN   TestResolve/normal_case
 FATAL[0002] Failed to list specs      error=rpc error: code = PermissionDenied desc = Permission denied on resource project controller-demo.
error details: name = ErrorInfo reason = CONSUMER_INVALID domain = googleapis.com metadata = map[consumer:projects/controller-demo service:apigeeregistry.googleapis.com]
error details: name = Help desc = Google developer console API key url = https://console.developers.google.com/project/controller-demo/apiui/credential uid=e747eb20
  WARN[0004] Task failed: Execute command: registry compute complexity projects/controller-demo/locations/global/apis/petstore/versions/1.1.0/specs/openapi.yaml@fe815810 error=failed running command
 FATAL[0002] Failed to list specs      error=rpc error: code = PermissionDenied desc = Permission denied on resource project controller-demo.
error details: name = ErrorInfo reason = CONSUMER_INVALID domain = googleapis.com metadata = map[consumer:projects/controller-demo service:apigeeregistry.googleapis.com]
error details: name = Help desc = Google developer console API key url = https://console.developers.google.com/project/controller-demo/apiui/credential uid=12e1be4d
 FATAL[0002] Failed to list specs      error=rpc error: code = PermissionDenied desc = Permission denied on resource project controller-demo.
error details: name = ErrorInfo reason = CONSUMER_INVALID domain = googleapis.com metadata = map[consumer:projects/controller-demo service:apigeeregistry.googleapis.com]
error details: name = Help desc = Google developer console API key url = https://console.developers.google.com/project/controller-demo/apiui/credential uid=e28ecbfa
  WARN[0004] Task failed: Execute command: registry compute complexity projects/controller-demo/locations/global/apis/petstore/versions/1.0.0/specs/openapi.yaml@6a7aaf2f error=failed running command
  WARN[0004] Task failed: Execute command: registry compute complexity projects/controller-demo/locations/global/apis/petstore/versions/1.0.1/specs/openapi.yaml@4d734993 error=failed running command
    resolve_test.go:296: Returned unexpected diff (-want +got):
          []string{
        -       "projects/controller-demo/locations/global/apis/petstore/versions/1.0.0/specs/openapi.yaml@6a7aaf2f/artifacts/complexity",
        -       "projects/controller-demo/locations/global/apis/petstore/versions/1.0.1/specs/openapi.yaml@4d734993/artifacts/complexity",
        -       "projects/controller-demo/locations/global/apis/petstore/versions/1.1.0/specs/openapi.yaml@fe815810/artifacts/complexity",
          }

The problem is that it execs  the registry tool without setting its configuration.
https://github.com/apigee/registry/blob/f21945d61cc62df5a81369f10c58d6407a6076dc/cmd/registry/controller/local-tasks.go#L72


#13 Create Kadmin that can create topics

Codecov Report
Patch coverage: 62.96% and project coverage change: -3.06 :warning:

Comparison is base (4cb1825) 93.38% compared to head (1652f54) 90.33%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff              @@
##               main      #14      +/-   ##
============================================
- Coverage     93.38%   90.33%   -3.06%
- Complexity       80       88       +8
============================================
  Files            19       21       +2
  Lines           242      269      +27
  Branches          3        4       +1
============================================
+ Hits            226      243      +17
- Misses           15       24       +9
- Partials          1        2       +1

Impacted Files
Coverage Δ

...n/java/su/ptx/emka/clients/admin/CreatedTopic.java
52.94% <52.94%> (ø)

.../src/main/java/su/ptx/emka/clients/admin/Kadm.java
80.00% <80.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


This has not been maintained

The docs say:

In addition to the Vue version of Ping CRM, we also maintain a Svelte version of the application, which you can find on GitHub.

However this repo has not had any updates in years

The project has been updated to Laravel 11, Svelte 4, and Vite ✨


Java verb calls don't have a way to stop when cancelled

Java does not currently have a way to stop execution of the call that gets canceled, so it will always continue even when the async call's lease expires.
Go does this by cancelling the context when the call grpc request ends, so when a verb tries to use the context things should fail.
Speaking with @stuartwdouglas last week, it sounds non-obvious as to the best way to do this in Java.

Test is disabled for this for java here: https://github.com/TBD54566975/ftl/blob/main/backend/controller/pubsub/integration_test.go#L205

Tricky in Java. Will revisit if needed.


ARROW-6286: [GLib] Add support for LargeList type

This PR add support for GArrowLargeListArrayBuilder, GArrowLargeListArray, and GArrowLargeListDataType.

Could you also look into the Travis CI failure on macOS?
(It's not related to this change.)

Sorry, I didn't confirm the CI result. I've rebased this branch on master.
I will confirm the CI result later.


Adding changes for capping description length

#134

don't forget to add desc_cap to readme!

Current coverage is 37.20%

Merging #135 into master will decrease coverage by -0.41% as of f67ac27

@@            master   #135   diff @@
=====================================
  Files           25     26     +1
  Stmts          755    774    +19
  Branches         0      0
  Methods          0      0
=====================================
+ Hit            284    288     +4
  Partial          0      0
- Missed         471    486    +15

Review entire Coverage Diff as of f67ac27

Powered by Codecov. Updated on successful CI builds.

r+