[Bug]: StructuredPlannerAgent always defaults to single task plan.

Bug Description
StructuredPlannerAgent doesn't seem to be creating sub-tasks :(
I tried my own toolset and wasn't getting any success - my setup has multiple tools: txt2sql, visualization, semantic search, etc.
Then I thought maybe my use-case was too complex so I tried a dummy example with tools = [
product_price_tool,
products_available_tool,
product_description_tool,
check_availability_tool
]
and I asked things like "for each product available, get me the their prices and find me the description of the cheapest product", etc.. no sub-tasks.
Finally I gave up and tried the example from the documentation  (see code below) because I thought maybe it didn't find simple function calls 'complex enough' and it may work with query_index better.... but still no sub-tasks (unlike what's shown in the example)... the only difference between my code and the example in the doc is that I'm using VertexAI/Gemini instead of chatgpt4/openai
It simply never creates a multi sub-task plan, I even tried changing the initial_plan_prompt and plan_refine_prompt to emphasize always creating sub-tasks no matter the complexity, etc... but no success
Btw, I do get the answers in the end but it behaves as if I just used a FunctionCallingAgent not as the StructuredPlannerAgent
Version
0.11.18
Steps to Reproduce
from llama_index.core.tools import FunctionTool
        from llama_index.core.agent import FunctionCallingAgentWorker, StructuredPlannerAgent
        from llama_index.llms.vertex import Vertex
        from llama_index.core import Settings
        from llama_index.embeddings.vertex import VertexTextEmbedding
        from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
        from llama_index.core.tools import QueryEngineTool

        embed_model = VertexTextEmbedding(
            client_email=...
            token_uri=...
            private_key=...
            private_key_id=...
        )

        llm = Vertex(model='gemini-1.5-pro-002',
                    max_tokens=8000,
                    temperature=0.1)

        Settings.embed_model = embed_model
        Settings.llm = llm

        # Load documents, create tools
        lyft_documents = SimpleDirectoryReader(
            input_files=["pipelines/lyft_2021.pdf"]
        ).load_data()
        uber_documents = SimpleDirectoryReader(
            input_files=["pipelines/uber_2021.pdf"]
        ).load_data()

        lyft_index = VectorStoreIndex.from_documents(lyft_documents)
        uber_index = VectorStoreIndex.from_documents(uber_documents)

        lyft_tool = QueryEngineTool.from_defaults(
            lyft_index.as_query_engine(llm = llm),
            name="lyft_2021",
            description="Useful for asking questions about Lyft's 2021 10-K filling.",
        )

        uber_tool = QueryEngineTool.from_defaults(
            uber_index.as_query_engine(llm = llm),
            name="uber_2021",
            description="Useful for asking questions about Uber's 2021 10-K filling.",
        )

        tools = [
            lyft_tool,
            uber_tool,

        ]

        agent_worker = FunctionCallingAgentWorker.from_tools(
            tools, llm=llm,
            verbose=True
        )

        agent = StructuredPlannerAgent(
            agent_worker, tools=tools, verbose=True, llm=llm,
        )

        message = "Summarize the key risk factors for Lyft and Uber in their 2021 10-K filings."

        plan_id = agent.create_plan(message)
        plan = agent.state.plan_dict[plan_id]

        yield f"\n{plan}"

        response = agent.chat(message)

        yield f"\n{response}"

Relevant Logs/Tracbacks
No complex plan predicted. Defaulting to a single task plan.
=== Initial plan ===
default:
Summarize the key risk factors for Lyft and Uber in their 2021 10-K filings. ->

@dosu I did try modifying the initial_plan_prompt and plan_refine_prompts
For example:
DEFAULT_INITIAL_PLAN_PROMPT = """
Think step-by-step. Given a task and a set of tools, create a comprehesive, end-to-end plan to accomplish the task.

The plan should end with a sub-task that satisfies the overall task.
The tools available are:
{tools_str}
Overall Task: {task}
"""
Is StructuredPlannerAgent prompt-based/output-parsing dependent like ReactAgent or does it leverage function calling ?
Is it possible the functioncalling api is putting the "plan" in the tool calls part of the response but that isn't parsed by the planner agent?

Vertex/Gemini could be just really bad at following prompt instructions?
If the LLM class doesn't implement FunctionCallingLLM base class, then it resorts to manual prompting/output parsing to parse structured outputs, which is very fragile. If it breaks, it defaults to a single task/plan
https://github.com/run-llama/llama_index/blob/227145fb94fcaa4da02d559fc81843fcb2af2b57/llama-index-core/llama_index/core/agent/runner/planner.py#L245
Looking at LLM base classes,

Gemini does not implement FunctionCallingLLM https://github.com/run-llama/llama_index/blob/227145fb94fcaa4da02d559fc81843fcb2af2b57/llama-index-integrations/llms/llama-index-llms-gemini/llama_index/llms/gemini/base.py#L52
Vertex does: https://github.com/run-llama/llama_index/blob/227145fb94fcaa4da02d559fc81843fcb2af2b57/llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/base.py#L50

but only if its a gemini model through vertex https://github.com/run-llama/llama_index/blob/227145fb94fcaa4da02d559fc81843fcb2af2b57/llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/base.py#L178
which seems like it relies on the model name starting with gemini: https://github.com/run-llama/llama_index/blob/227145fb94fcaa4da02d559fc81843fcb2af2b57/llama-index-integrations/llms/llama-index-llms-vertex/llama_index/llms/vertex/gemini_utils.py#L9

(If you don't have the latest vertex package too, you might want to update, pip install -U llama-index-llms-vertex

I am using the Vertex class and not Gemini, I only said Vertex/Gemini because I'm actually using Vertex('gemini-1.5-pro-002')
I am using llama_index-llms-vertex==0.3.7 which is the latest
I am not having any issues with function calling, I can see function calls/outputs going through just as the FunctionCallingAgent does by itself.
I also thought the model may be bad at following prompt instructions - that's why I tried to recreate the example from the documentation without having my own custom system prompts, etc... I mean I tend to agree vertex/gemini-1.5-pro isn't as good as chatgpt4 but this is a rather simple prompt with only 2 tools - so I don't think it's that bad... and just for funzies I tried doing a raw query through the vertex studio:
Think step-by-step. Given a task and a set of tools, create a comprehesive, end-to-end plan to accomplish the task.
Keep in mind not every task needs to be decomposed into multiple sub-tasks if it is simple enough.
The plan should end with a sub-task that satisfies the overall task.

The tools available are:
lyft_tool:"Useful for asking questions about Uber's 2021 10-K filling."
ubert_tool: "Useful for asking questions about Lyft's 2021 10-K filling."

Overall Task: Summarize the key risk factors for Lyft and Uber in their 2021 10-K filings.

vertex_ai/gemini-1.5-pro
Identify Lyft's risk factors: Use the lyft_tool to ask "What are the key risk factors mentioned in Lyft's 2021 10-K filing?"

Identify Uber's risk factors: Use the ubert_tool to ask "What are the key risk factors mentioned in Uber's 2021 10-K filing?"

Combine and summarize: Combine the information gathered from both tools, summarizing the key risk factors for both Lyft and Uber. This is the final step that completes the overall task.

So clearly the LLM does get the idea of what needs to be done... so maybe the output format is not being respected
Is there any easy way for me to log/monitor the exact payload sent to/received from the LLM?

So this is where it's "breaking" and defaulting to the single plan, there seems to this error 'Unknown field for Schema: $defs'

Seems like it needs a PR to update that function?

The full stack trace:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/agent/runner/planner.py", line 239, in create_plan
    plan = self.llm.structured_predict(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 307, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/llms/llm.py", line 371, in structured_predict
    result = program(llm_kwargs=llm_kwargs, **prompt_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 307, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/program/function_program.py", line 202, in __call__
    agent_response = self._llm.predict_and_call(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^\\
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 307, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/llms/function_calling.py", line 182, in predict_and_call
    response = self.chat_with_tools(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/llms/function_calling.py", line 48, in chat_with_tools
    response = self.chat(**chat_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 307, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py", line 173, in wrapped_llm_chat
    f_return_val = f(_self, messages, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/llms/vertex/base.py", line 240, in chat
    generation = completion_with_retry(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/llms/vertex/utils.py", line 113, in completion_with_retry
    return _completion_with_retry(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^\\
  File "/usr/local/lib/python3.11/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^\\
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^\\
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.11/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^\\
  File "/usr/local/lib/python3.11/site-packages/llama_index/llms/vertex/utils.py", line 92, in _completion_with_retry
    tools = to_gemini_tools(tools) if tools else []
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/llms/vertex/utils.py", line 60, in to_gemini_tools
    func_name = FunctionDeclaration(
                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 1657, in __init__
    raw_schema = aiplatform_types.Schema(gapic_schema_dict)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/proto/message.py", line 724, in __init__
    raise ValueError(
ValueError: Unknown field for Schema: $defs

I tried upgrading google-cloud-aiplatform but issue persists.
Based on my understanding and the error message 'Unknown field for Schema: $defs' it seems like $defs is not supposed to be part of 'parameters'**
class FunctionDeclaration:
    r"""A representation of a function declaration.

    Usage:
        Create function declaration and tool:
        '''
        get_current_weather_func = generative_models.FunctionDeclaration(
            name="get_current_weather",
            description="Get the current weather in a given location",
            parameters={
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": [
                            "celsius",
                            "fahrenheit",
                        ]
                    }
                },
                "required": [
                    "location"
                ]
            },
        )

Below is the output of tools[0].metadata.get_parameters_dict() which is producing this $defs
def get_parameters_dict(self) -> dict:
        if self.fn_schema is None:
            parameters = {
                "type": "object",
                "properties": {
                    "input": {"title": "input query string", "type": "string"},
                },
                "required": ["input"],
            }
        else:
            parameters = self.fn_schema.model_json_schema()
            parameters = {
                k: v
                for k, v in parameters.items()
                if k in ["type", "properties", "required", "definitions", "$defs"]
            }
        return parameters

{'$defs': {'SubTask': {...}}, 'properties': {'sub_tasks': {...}}, 'required': ['sub_tasks'], 'type': 'object'}
special variables:
function variables:
'$defs': {'SubTask': {'description': 'A single sub-task in a plan.', 'properties': {...}, 'required': [...], 'title': 'SubTask', 'type': 'object'}}
special variables:
function variables:
'SubTask': {'description': 'A single sub-task in a plan.', 'properties': {'name': {...}, 'input': {...}, 'expected_output': {...}, 'dependencies': {...}}, 'required': ['name', 'input', 'expected_output', 'dependencies'], 'title': 'SubTask', 'type': 'object'}
special variables:
function variables:
'description': 'A single sub-task in a plan.'
'properties': {'name': {'description': 'The name of the sub-task.', 'title': 'Name', 'type': 'string'}, 'input': {'description': 'The input prompt for the sub-task.', 'title': 'Input', 'type': 'string'}, 'expected_output': {'description': 'The expected output of the sub-task.', 'title': 'Expected Output', 'type': 'string'}, 'dependencies': {'description': 'The sub-task names that must be completed before this sub-task.', 'items': {...}, 'title': 'Dependencies', 'type': 'array'}}
special variables:
function variables:
'name': {'description': 'The name of the sub-task.', 'title': 'Name', 'type': 'string'}
special variables:
function variables:
'description': 'The name of the sub-task.'
'title': 'Name'
'type': 'string'
len(): 3
'input': {'description': 'The input prompt for the sub-task.', 'title': 'Input', 'type': 'string'}
special variables:
function variables:
'description': 'The input prompt for the sub-task.'
'title': 'Input'
'type': 'string'
len(): 3
'expected_output': {'description': 'The expected output of the sub-task.', 'title': 'Expected Output', 'type': 'string'}
special variables:
function variables:
'description': 'The expected output of the sub-task.'
'title': 'Expected Output'
'type': 'string'
len(): 3
'dependencies': {'description': 'The sub-task names that must be completed before this sub-task.', 'items': {'type': 'string'}, 'title': 'Dependencies', 'type': 'array'}
special variables:
function variables:
'description': 'The sub-task names that must be completed before this sub-task.'
'items': {'type': 'string'}
special variables:
function variables:
'type': 'string'
len(): 1
'title': 'Dependencies'
'type': 'array'
len(): 4
len(): 4
'required': ['name', 'input', 'expected_output', 'dependencies']
'title': 'SubTask'
'type': 'object'
len(): 5
len(): 1
'properties': {'sub_tasks': {'description': 'The sub-tasks in the plan.', 'items': {...}, 'title': 'Sub Tasks', 'type': 'array'}}
special variables:
function variables:
'sub_tasks': {'description': 'The sub-tasks in the plan.', 'items': {'$ref': '#/$defs/SubTask'}, 'title': 'Sub Tasks', 'type': 'array'}
special variables:
function variables:
'description': 'The sub-tasks in the plan.'
'items': {'$ref': '#/$defs/SubTask'}
'title': 'Sub Tasks'
'type': 'array'
len(): 4
len(): 1
'required': ['sub_tasks']
'type': 'object'
len(): 4

I'm not sure why $defs is here or what its purpose is... quick google searches aren't yielding anything (I thought maybe it's a thing for OpenAI function calling API)

I tried yolo just getting rid of $defs and it started throwing this error instead: ('Protocol message Schema has no "$ref" field.',) so it seems like it's not a useless field lol
I don't think I know enough about the dependencies and inner workings to fix this but I'm happy to help if you want me to test things with Vertex @logan-markewich

@dosu any suggestions for how I could fix this bug ?

I'm working on this and other issues with Vertex/Gemini - the update to google's generativeai 1.70.0 version breaks a lot of things in the integration and it is taking some time for me to get to them- but this is my day job for my company so I'm going to keep plugging away at it and eventually I'll have a version of the integration that works with the current Gemini/Vertex versions.
This particular problem is because the google library itself uses an out of date internal dependency - I've reached out to my Google support rep to see if I can get that issue resolved (or at least get a hotfix for it) but I only just hit this issue and it's late on a Friday so I don't really expect any movement today.

I tried yolo just getting rid of $defs and it started throwing this error instead: ('Protocol message Schema has no "$ref" field.',) so it seems like it's not a useless field lol or maybe I'm running into a separate issue?
I'm also confused why this would impact StructuredPlannerAgent and not FunctionCallingAgent, I would've thought the same/similar code is used for both
I don't think I know enough about the dependencies and inner workings to fix this but I'm happy to help if you want me to test things with Vertex @logan-markewich

the Json to Protobuf handler will treat all fields that start with $ as a protobuf extension so that's part of the issue.

After conversation with Google support, it is because of the way that pydantic serializes the Plan object. The internal schema object for google-cloud-aiplatform doesn't recognize the schema type $ref or $def so it throws this error. Pydantic does not provide a mechanism to rename the schema field - it provides a template value for the reference values but not the names Pydantic Documentation.
I've pointed out to Google that they need to add this support to their library because it isn't possible to make this change to pydantic.

After conversation with Google support, it is because of the way that pydantic serializes the Plan object. The internal schema object for google-cloud-aiplatform doesn't recognize the schema type $ref or $def so it throws this error. Pydantic does not provide a mechanism to rename the schema field - it provides a template value for the reference values but not the names Pydantic Documentation.
I've pointed out to Google that they need to add this support to their library because it isn't possible to make this change to pydantic.

so this is not something we can monkey-patch on our end ?
do you have any idea why this issue happens for the StructuredPlannerAgent but not the FunctionCallingAgent ?

I haven't looked at FunctionCallingAgent, but I don't think it serializes a Plan object into it's tool call - which is what serializes with the json schema in it's tool_parameter field.
I'm trying to see if I can override how the to_gemini_tool method creates the tool objects and perform a google-compatible serialization but finding a generic serialization that is compatible with both protobuf and pydantic is challenging - and I'm not familiar enough with either to just whip one up. I'll keep plugging away - both at google and at the problem to see if I can get this figured out and into a PR.

This is now a Google issue you can look follow the status at the link and star it so you get updates as it progresses.
I'll continue to plug away at custom serialization formats to make this work - I think I found a way; but only some experimentation will tell.

This is now a Google issue you can look follow the status at the link and star it so you get updates as it progresses.
I'll continue to plug away at custom serialization formats to make this work - I think I found a way; but only some experimentation will tell.

Many thanks @stfines-clgx
I saw you were able to adapt the multi-concierge to use vertex and fixed a related bug... did you have good success with that ?

I didn't have much luck with it - but I did get it to work; I also got the Planner agent in this defect to work. I'll push that to my open pr and fork tonight if you want to pull it down and test it out to give me feedback.

I didn't have much luck with it - but I did get it to work; I also got the Planner agent in this defect to work. I'll push that to my open pr and fork tonight if you want to pull it down and test it out to give me feedback.

Hey, I got the multi-concierge example working without your fork
llama-index==0.11.10
llama-index-core==0.11.10
llama_index-llms-vertex==0.3.4
llama-index-utils-workflow==0.3.0
google-cloud-aiplatform==1.56.0

I did only minor modifications:

Switch llm = OpenAI(model="gpt-4o", temperature=0.4) to llm = Vertex(model="gemini-1.5-flash-002",max_tokens=8000,temperature=0.4)

Add instructions to the prompts to make sure orchestrator kicked in from the sub agents properly

Put some slight validations in workflow.py to make it fail cleaner when "agent_name" is not in tool_call.tool_kwargs or if tool = tools_by_name.get(tool_call.tool_name) returns None

Change the get_account_id to not always return the same user's account id

But after that, I got a nice workflow running, here's a sample output:
Orchestrator taking over...

SYSTEM >> Transferring to agent Authentication Agent

SYSTEM >> Recording username
SYSTEM >> Tool store_username called with {'username': 'test_username'} returned None

AGENT >> Please provide your password.

USER >> Okay, my password is  'password123'.

SYSTEM >> Logging in user test_username
SYSTEM >> Tool login called with {'password': 'password123'} returned Logged in user test_username with session token<PHONE_NUMBER>. They have an account with id 123 and a balance of $1000.
SYSTEM >> Agent is requesting a transfer. Please h[...]
AGENT >> What can I help you with today?

USER >> Okay, I'd like to check my account balance first.  Then, I want to transfer some money.  Finally, I'd like to know the current price of Apple stock (AAPL).

Orchestrator taking over...

SYSTEM >> Transferring to agent Account Balance Agent

SYSTEM >> Checking if authenticated
SYSTEM >> Looking up account balance for 123
SYSTEM >> Tool get_account_balance called with {'account_id': '123'} returned Account 123 has a balance of $1000

SYSTEM >> Agent is requesting a transfer. Please hold.
Orchestrator taking over...

AGENT >> Okay, your balance is $1000.  Now, how much money would you like to transfer and to which account?

USER >> Okay, I want to transfer $200 to account number 456.  Then tell me the price of AAPL.

Orchestrator taking over...

AGENT >>

USER >> Okay, the transfer is complete. Now, what's the price of AAPL?

Orchestrator taking over...

SYSTEM >> Transferring to agent Stock Lookup Agent

SYSTEM >> Looking up stock price for AAPL
SYSTEM >> Tool lookup_stock_price called with {'stock_symbol': 'AAPL'} returned Symbol AAPL is currently trading at $100.00

AGENT >> Okay, the current price of AAPL is $100.00. Is there anything else I can help you with?

USER >> exit

Is this too good to be true? Is it really using the vertex function calling api ?


Job #8579 - Ensure that CME filters are correctly selected

Correct the navigation for Filters for Context Menu entries so that they are filtered correctly even for Specialism ''

I verified the the plugin.xml looks right after this change and that the Core System Level tests now pass.


Add support for Google groups

Specifically, I'm interested in https://groups.google.com/a/dartlang.org/g/announce

Seems impossible without scraping. Will try to work around by subscribing and using the Fastmail API


Rename jobs.groovy.j2 to jobs.groovy now it doesn't use jinja.

In preparation for dropping the jinja translation step
we rename jobs.groovy.j2 to jobs.groovy so that it would be
found by the setup_ClusterHQ-flocker job config.
A symlink with the old name is kept around so that there
doesn't have to be a co-ordinated change across all
branches with Jenkins.
Once jenkins has changed to not using jobs.groovy.j2 then
the symlink can be removed.

LGTM


Tutorial on generating synthetic data with point spread

I have been writing some code to evaluate the effect of point-spread on inferring connectivity. @Eric89GXL suggested that a tutorial on how to put synthetic signals into a source space and corrupt those signals with point-spread via a forward and inverse solution could be useful.
I created this issue to get some feedback on what a good minimal example would be. I need to clean up my current code a bit, but can include it in the issue soon.

Great! I would personally expect to make two scenarios: one with independent sources and one with sync sources then:

simulate two sources in sample source space
add 1/fsource noise
project them in sensor space with the fwd model
add 1/f sensor noise
inverse the data
apply connectivity analysis of interest

Thanks for the feedback.
I am shooting to keep this example simpler rather than demonstrate a full pipeline. There are already some examples of how to perform connectivity analysis with mne-python, and I am happy to include links to them, however, I want to focus on the basics of how to put a known signal into the mne-python data structures and corrupt them with point-spread. Thus, I want to avoid incorporating anything superfluous to this goal so that it's easy to follow. Users can then look to the other MNE examples to make it more sophisticated.

Closing since #3350 has been merged.


fix #30

It appears that when appleorange1 pull-requested this into pyelliptic, he messed up the indentation.

Reference to #30

Ah yes, sorry about that. This should definitely be merged.

Will you be able to merge this soon?

Sorry for the delay. I had a lot of work. I'm on vacation this week, I'll work on pyelliptic.

@appleorange1 @Atheros1 v1.5.6 is available


Support loading model from local storage

/kind feature
Describe the solution you'd like
kfserving should support onprem cluster, since there are some user cases which are privisioned for on-premise cluster, and the trained model is stored in local storgae. That's better to have a easy way to configure local storage such as PV/PVC and then allow modelUri point to local path that's mounted by PVC.

/assign

The subject seems to broad.. maybe update it to support loading from
mounted PVC or something like that?
On Wed, May 29, 2019, 5:56 PM Jin Chi He<EMAIL_ADDRESS>wrote:

/kind feature
Describe the solution you'd like
kfserving should support onprem cluster, since there are some user cases
which are privisioned for on-premise cluster, and the trained model is
stored in local storgae. That's better to have a easy way to configure
local storage such as PV/PVC and then allow modelUri point to local path
that's mounted by PVC.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kubeflow/kfserving/issues/129?email_source=notifications&email_token=AA2CTBIUPPTCEIXHMGLFJFDPX4Q4DA5CNFSM4HQZJYSKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GWTWEMA,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA2CTBNSLL4GWSNAZXB6BNLPX4Q4DANCNFSM4HQZJYSA
.

The subject seems to broad.. maybe update it to support loading from mounted PVC or something like that?

Agree, updated, thanks.

Seems the knative does not support mounts PVCs for container. See tickets https://github.com/knative/serving/issues/2025 and beflow logs for details.
{"level":"error","ts":1559532065.8891385,"logger":"kfserving-controller","msg":"Failed to reconcile default model spec","name":"flowers-sample-default","error":"Internal error occurred: admission webhook \"webhook.serving.knative.dev\" denied the request: mutation failed: expected exactly one, got neither: spec.revisionTemplate.spec.volumes[0].configMap, spec.revisionTemplate.spec.volumes[0].secret\nmust not set the field(s): spec.revisionTemplate.spec.volumes[0].persistentVolumeClaim"

Seems need knative support the cases firstly. Thanks.

Keep in mind we can always work around these with MutatingWebhooks. A Knative interface limitation is annoying but not a blocker for us.

@ellis-bigelow Thanks for your comments.  Yes, I think that's a work around solution, seems the kanative side has no action for this recently.  I'm going to have a investigation for the way and create RP then.
/cc @hougangliu @gyliu513

@jinchihe Would your scenario be solved by a combination of Minio and #148?

@rakelkar  We have a trying case that's used Minio for our trained mode that's stored locally on premise cluster. But I think we should support some easy way like PVC mounting to let kfservice get mode.
Personally we can useWebhook way to get our purpose, but seems it's impure :-) We can enhance later once knative` supported.

In general I prefer we first try to push for the right solution. From a KNative perspective mounting a PVC itself is impure and I doubt that they will ever support that directly.
I would advocate starting with something simple like the INIT container approach and then learn and try to incrementally optimize. I dont think it makes sense to build out this webhook workaround at this time.
@ellis-bigelow  thoughts?

@rakelkar Thanks. We have real Pipeline case for on-premise cluster, the model is trained in the upstream steps of Kubeflow Pipeline, and stored local storage (such as NFS), and just want using kfserving to start service. You know, not all user have policy for access clould storage.
I strongly suggest to implement this. I thinlk this is a common way to resolve similar problems.  And I had a trying webhook way, it works as expected. Will create a PR once finishing tests. Thanks.

@rakelkar
In our on-prem cluster, we have similar pipelines where serving component loads from mounted PVCs backed by NFS.  This is a likely scenario in on-prem KF environments(though underlying storage can be different)

I’m not sure I can join meeting for discussion, but I tried to summarize the pros/cons for supporting pvc, please discuss that. Great thanks!
Pros for pvc supporting:

PVC is generic storage for k8s, and import storage, especially for on prem cluster.
For Pipeline cases, downloading source and training model in the upstream steps of Pipeline, and start service using KFServing, that’s a easy way of use the PVC directly.
Kubeflow Fairing may use kfserving, and Fairing is going to support on-prem cluster, if so, pvc will be better way.
PVC performance is volume plugin related, saying if volume plugin is GCEPersistentDisk, PVC performance should keep same as gs.
May use minio to work around, personally seems extra walk for this, PVC is directly way. And how get model if use monio from storage? seems cannot skill pvc or local storage, and minio need one more pod from my understanding.

Cons for pvc supporting:

Knative does not support PVC (recheck knative ticket, I did not get real reasons), so if we support this, need use webhook method.
Not sure if have performance issue if mount multiple pods for autoscaling.

This is now supported! /close


Organize input folder

Hi all,
I discussed with @nichollsh about the issue of escape.toml being accidentally removed from the input folder previously. To address this and to avoid having too many initialization files in the input directory, we thought about organizing them into subfolders.
In this PR, I have re-added dummy.toml and escape.toml and organized the input folder into three subfolders: demos, planets, and trappist1.
Please let us know your thoughts on this approach and if you have any suggestions, ideas, or improvements.

The tests seem to be failing due to inconsistencies in the value of CH4_mol_atm.

DataFrame.iloc[:, 71] (column name="CH4_mol_atm") values are different (0.31447 %)

I have noticed this error prior to your PR, @EmmaPostolec, so it's most likely not a result of these changes. This quantity is very sensitive to temperature at the surface, so differences on the order of 0.3% are not unexpected. If you run the tests again, they will probably pass. We need to think about this some more.

Hi @nichollsh,
Thanks a lot for mentioning this! At first, I thought it might be related to the change in the path for the input files we are testing, but after digging deeper into the test script, I am not so sure.
I am re-running the test on GitHub right now let's hope it works! Alternatively, would it make sense to adjust the test criteria to 0.4% instead of 0.3% to ensure it passes?
Also, I was wondering if the tests are failing for the same reason in PR 301: https://github.com/FormingWorlds/PROTEUS/pull/301.
Let me know what you think is the best approach!

Let me know what you think is the best approach!

I think the best thing to do here is to re-run the tests until they pass, and then merge these PRs, since we know they are not causing the problem.
We should revisit the failing tests in a different issue. We could increase the tolerance, but if it's only CH4_mol_atm that ever causes problems then maybe we could exclude that particular  column from the tests. It is testing the same physics as CH4_bar, H2_mol_atm, etc, anyway.

Made an issue here: #302

Nice thanks for the comments @timlichtenberg and @nichollsh ! Can I merge it now ? :)

Yes!


Fix screenshot comparison

Objective

After merging #13248 the new upload job fails

Solution

Fix the file path
Instead of a pull_request_target workflow, keep the examples in the pull_request workflow and add another job that will run once its all completed on a workflow_run event to upload screenshots

Testing

Tested in a ubuntu docker container, running the exact same script
Manual result: https://pixel-eagle.com/project/B04F67C0-C054-4A6F-92EC-F599FEC2FD1D/run/5/compare/2
The CI on this job will still fail as its using the job from main

@BD103 I reverted the parts that used a pull_request_target to move the upload to a workflow_run event which should still have permissions to access secrets
It's still kinda hard to make sure everything will work fine as it uses the workflow definition from main, not from the PR


Portfolio website

Description
Added a cool and professional portfolio website.
Fixes #454

Type of change 📝
Added new project.

[x] Add a new project

Web development
Please delete options that are not relevant.And in order to tick the check box just but x inside them for example [x] like this

[x] Project using HTML, CSS & JS

Checklist ✅
Please delete options that are not relevant.And in order to tick the check box just but x inside them for example [x] like this

[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GSSoC'22 contributor

Screenshots

@srinjoy-26 bro are you copied this?
https://www.youtube.com/watch?v=tcskp-ncN0I

@srinjoy-26 bro are you copied this?
https://www.youtube.com/watch?v=tcskp-ncN0I

I had just followed the tutorial completely.


Code example with CUDA 12.3, Windows 11, Visual Studio 2022

Hi David,
Thank you for sharing this library.
I am trying to make the example code to work with CUDA 12.3 and Visual Studio Community 2022 version but there are many errors. I guess this problem is because I didn't config project properties properly.
May you please share the working example with Visual Studio Project file for reference?
Thank you.

Hi exterity!
Thanks for sharing your thoughts.
I replied the process step by step, and everyting runs fine.
Please, make sure to double check this particular things (because VStudio is sometimes a bit picky).

Check that you created a CUDA project
Check that you copied BigInteger.cu and BigInteger.h into the repo folder, also imported into the solution
Check that you marked the -rdc=true option under CUDA C/C++ > Common
Check that you defined the preprocessor macros CUDA_ENABLED=1; BI_STANDALONE=1  under  CUDA C/C++ > Host

I will also be attaching a folder that I hope you can copy/paste so you don't have to play around with all the preferences everytime.
Default Template.zip
When I have some time, I will update the main page to add a bit more of detail on how to make the first set-up.
Hope this solves your issue!

Just want to confirm with you that it also works with CUDA 12.3. Thank you.


Async tests with rethinkdb

First, thanks very much for sharing this package, it's awesome!
I am currently trying to test a tornado app with rethink, however there is no hook to call the rethinkdb.set_loop_type method before creating a connection.  It would be cool if you could add something.
If not, I will try to set aside some time in a couple of weeks and contribute a PR.  Any advice/pointers on implementation would be appreciated.

Hi there, good suggestion thanks!
There's a number of ways we could do this, but how I'd do it is something like:
Add a config variable in pytest_server_fixtures.init:
DEFAULT_SERVER_FIXTURES_RETHINK_LOOP_TYPE = None
CONFIG=FixtureConfig(
  ..
  rethink_loop_type==os.getenv('SERVER_FIXTURES_RETHINK_LOOP_TYPE', DEFAULT_SERVER_FIXTURES_RETHINK_LOOP_TYPE),
  ...
)

Add a constructor argument to the RethinkDBServer class:
class RethinkDBServer(TestServer):
    random_port = True

    def __init__(self, loop_type=CONFIG.rethink_loop_type, **kwargs):
        global rethinkdb
        try:
            import rethinkdb
        except ImportError:
            pytest.skip('rethinkdb not installed, skipping test')
        super(RethinkDBServer, self).__init__(**kwargs)
        if loop_type:
           rethinkdb.set_loop_type(loop_type)

Then, you could enable tornado by either setting the environment variable before running your test, setting the config options manually in the Python, or constructing your own fixture with the class directly.

@skolsuper Hi there, with the sad news recently of the demise of RethinkDB, do you think you will still be working on this issue? Otherwise I will close it, thanks.

Hi @eeaston, I apologize, I had forgotten all about this issue as I am not working with Python much any more :(.  It is a shame about rethinkdb the company, but I imagine the database might live on for a while.  Then again, if nobody else has had this issue maybe it's just not a common combination, and I doubt anybody is about to pick it up now, so I think you can safely just close this :(

Hi @eeaston, I apologize, I had forgotten all about this issue as I am not working with Python much any more :(. It is a shame about rethinkdb the company, but I imagine the database might live on for a while. Then again, if nobody else has had this issue maybe it's just not a common combination, and I doubt anybody is about to pick it up now, so I think you can safely just close this :(


Live view processing performance.

Hi,
How could we make this faster?
The ofLoadImage() call is the one demanding the most part of the processing time.
Would ofxCV be of help for this?
lock();
if(liveBufferMiddle.size() > 0) {
	// decoding the jpeg in the main thread allows the capture thread to run in a tighter loop.
	ofBuffer* middleFront = liveBufferMiddle.front();
	liveBufferFront.set(middleFront->getBinaryBuffer(), middleFront->size());
	liveBufferMiddle.pop();
	unlock();
	ofLoadImage(livePixels, liveBufferFront);
        livePixels.rotate90(rotateMode90);
	if(liveTexture.getWidth() != livePixels.getWidth() ||
		liveTexture.getHeight() != livePixels.getHeight()) {
		liveTexture.allocate(livePixels.getWidth(), livePixels.getHeight(), GL_RGB8);
	}
	liveTexture.loadData(livePixels);
	lock();
	liveDataReady = true;
	frameNew = true;
	unlock();
} else {
	unlock();
}

Thanks

ofLoadImage() is actually taking a compressed jpg, decompressing it, and loading it into pixels. that's going to be an intensive operation no matter what. i think OF is using libjpeg via FreeImage for this, you could try wrapping your own decoder like libjpeg-turbo or there might be a faster native solution for your platform, or there might be a library that decodes on the GPU.
it would be good to double-check that the livePixels are not being reallocated by ofLoadImage(). that would be a performance hit.

Yep exactly... decompressing and loading into pixels is heavy.
Thanks for the link to libjpeg-turbo though. Using ofxTurboJpeg I have been able to test this in minutes, and the perf are already better.

great! going to close this issue. please report the exact performance difference if you get a chance.

For sure, I will post here some more details when I will have that running in a simple example project in order to really see the difference.


Find memory leak in Realm

I don't know how to fix the memory leak.

@jinmin888 can you please share more information as outlined in our issue template? Posting a screenshot is not sufficient for us to identify what is happening here.

@jinmin888 just following up on my last request for information.

I'm facing the same problem.
Leak Instrument shows that each time when [RLMRealm defaultRealm] is called, there is a new leak show up in my app. And the last readable line is same as the first post.

Version of Realm and Tooling
Realm version: 0.98.6
Xcode version: Version 7.3 (7D175)
iOS/OSX version: 9.3

@tgoyne, this looks related to the CachedRealm stuff you introduced in the object store.

I am also seeing a bunch of realm leaks, and all of the call trees point to the default realm, and a set of these are inside of an async call like so:
`dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
// Get new realm and table since we are in a new thread
RLMRealm *realm = [RLMRealm defaultRealm];
[realm beginWriteTransaction];
    // Do realm createOrUpdateInDefaultRealmWithValue on objects

    [realm commitWriteTransaction];

});`


Cadence RPM seems to be half

Hi arpruss,
Your project is just what I need.
I have converted it to a platformio project in my fork here: https://github.com/MortenVinding/BLEBike.git
But it seems like the RPM counter is counting only half the revolutions.
Is this configurable on the display I haven't connected since I don't have a HD44780 display at hand?
 Is it normally high, with a low spike (how long?)? Or the other way around?

Also I only measured with a multimeter.
It’s normally high (4.29 volts if memory serves), and the low pulse seemed to be responsible long.
I will scoope that also later.
But it is precisely half of the build-in RPM display.
Changing the 60000 / in the code to 30000 / made it bang-on with the bikes computer.

Why don't you add a bit of code to the interrupt handler to toggle an LED reach time it detects a rotation, and then see what happens when you rotate the pedals slowly: do you miss every second one? You can also hook up a multimeter at the same time and make sure there was a signal that it missed.
It wouldn't surprise if there was some bug in my code. The ESP has trouble with triggering interrupts on a specific edge, and so I have to trigger on both edges and kludgily drop one.
Could be the same with the heart rate.

Why don't you add a bit of code to the interrupt handler to toggle an LED each time it detects a rotation, and then see what happens when you rotate the pedals slowly: do you miss every second one? You can also hook up a multimeter at the same time and make sure there was a signal that it missed.

Great idea. I will do that. And also hook up my scope.

It wouldn't surprise if there was some bug in my code. The ESP has trouble with triggering interrupts on a specific edge, and so I have to trigger on both edges and kludgily drop one.

Ahh that could also very well explain the half RPM number. So maybe for some weird reason, my ESP32 is only triggering on one of the edges.

Could be the same with the heart rate.

Might yes. But there must also be something else going on, because it's much more off. Around 2-3 BPM when the bike registers around 70 BPM.

On my bike, there are two different heart lines. There are the lines to the sensors, and you are unlikely to be able to catch anything on them without some circuitry to amplify the faint signals, and there are digitally processed signals -- nice square waves -- that should be easier to work with. But I never actually tried that heart code, because I ended up getting a chest band, and so now I integrate that data over bluetooth.

Ohh and don't worry about the voltages, I forgot to zero out the DSO138 (yes it's a cheap scope 😆)
They do go 0-3.3 volt (rotation) and 0-4.2 volt (heart).

I wouldn't expect serial print to work inside an interrupt handler. Interrupt handlers need to move fast.
Did you configure the LED in your setup() (i.e., pinMode(LED,OUTPUT))?  Then you need a bit of code like:
static uint8_t ledState = 0; digitalWrite(LED, ledState); ledState = !ledState;
in the ISR.
I suggest starting with putting that code right at the beginning of rotationISR();. You should only see a brief blink of the LED, because the width of the signal is narrow.

Try this. At the beginning of the ISR: digitalWrite(LED,digitalRead(rotationDetectPin)). This should exactly duplicate the rotation detect pin on the LED if both edges are being captured. If you can't tell from the LED what is going on, put the scope on the LED pin (or use a different output pin). Ideally you'd use a two-channel scope, one for the LED and the other for the rotation detect, and see if both channels are the same, but you can make-do with just one channel.

Thanks.
That seems to work reliably. The LED is on in a little more than 3/4 of the revolution and of in little less than 1/4.
So I guess it triggers the interrupt on both edges after all?
But if I change
uint32_t rpm;   uint32_t _power = _lastPower;   if (!rev || _lastRotationDuration == 0)     rpm = 0;   else if (_lastRotationDuration < t - prevRotationMarker) {     rpm = 60000 / (t - prevRotationMarker);     _power = calculatePower(t - prevRotationMarker);   }   else     rpm = 60000 / _lastRotationDuration;
to 120000 instead of 60000, the RPM number is exactly the same as the bike computers.

I would rather figure out what is wrong than just change the timing to make it work, since even if that works for you, it won't work for others (e.g., it won't work for me).
I have two more test sfor you. Comment out these lines:
if (t < lastBounce + rotationMarkerDebounceTime) {
    lastBounce = t;
    return;
  }

And see if now the timing is right. If it is, then maybe play a bit with the rotationMarkerDebounceTime and making it smaller and see if that solves your problem.
If this doesn't solve the problem, then move the LED code to right before the uint32_t t = millis(); line.

Absolutely agree.
It was only to explain that the RPM number is exactly half of the real.
I tried removing the (t < lastBounce + rotationMarkerDebounceTime) part but it did not change anything.
Haven't tried to put the LED code before uint32_t t = millis(); but I did have it in the very top of the rotationISR part before.
Btw: I use the ledPin you have already configured in the code (that is GPIO 2 and the internal LED on my board).
I noticed that lastBounce is used both in rotationISR and heartISR, don't know if that's a problem.
I triede to rename it in the rotationISR handler, but that also didn't change anything.
Thank you so much for your help.

The use of the same bounce timer is definitely an issue I should look into.

lastBounce is static, so it's local to the two functions. No problem there.
Add an LED toggler at the end of rotationISR().
static uint8_t ledState = 0; digitalWrite(LED, ledState); ledState = !ledState;
Then you should have the LED toggling once per full rotation. Do you?

Then you should have the LED toggling once per full rotation. Do you?

I already did that per your instructions before, but I tried to move it to the bottom of the rotationISR().
And yes it toggles once per revolution. Only once.
I think there is something strange going on with the ISR, because the heartbeat is way off, but if I swap the heart and rotation input, the rotation (now triggered by the heart sensor), is also exactly half of my heart rate (as displayed by the bike computer), and the heart rate (now triggerede by the rotation sensor) is showing the same strange behaviour as when the heart sensor was connected to it.

Toggling once per rotation is correct behavior. So the problem is somewhere else.

Toggling once per rotation is correct behavior. So the problem is somewhere else.

but I thought there was a problem with the ESP32 interrupts triggering on both the rising and falling edge?
don’t you divide by 2 in the code? (I can’t figure it out. I’m terrible programmer 🙄)

Yes, the interrupt handler does trigger on rise and fall. But the following code makes sure that the fall is ignored:
if (cleanRotationMarkerState != defaultRotationMarkerValue)
    return;
  // trigger on end of marker

Try disabling the heart code completely (comment out #define HEART_PIN) and see if the rotation works better.
Another thing to try. After the rpm = 60000 / _lastRotationDuration; line, print to serial both the rpm and _lastRotationDuration. Then using a stopwatch, rotate the pedals slowly, so that a rotation takes about three seconds. Compare the three seconds the _lastRotationDuration (which is in milliseconds).

Yes, the interrupt handler does trigger on rise and fall. But the following code makes sure that the fall is ignored:

Ahh of course!

Try disabling the heart code completely (comment out #define HEART_PIN) and see if the rotation works better.

Tried. No change :(

Another thing to try. After the rpm = 60000 / _lastRotationDuration; line, print to serial both the rpm and _lastRotationDuration. Then using a stopwatch, rotate the pedals slowly, so that a rotation takes about three seconds. Compare the three seconds the _lastRotationDuration (which is in milliseconds).

I added:
    Serial.print("RPM: ");
    Serial.println(rpm);
    Serial.print("_lastRotationDuration: ");
    Serial.println(_lastRotationDuration);

Pictures of output from it, from the bike computer and screenshot from Zwift.
Maybe I should try a different app?

So it looks like it IS correctly counting the rpm, but there is some problem in the transmission to Zwift.
The code sends data over multiple bluetooth services: cadence, power, heart and fitness. Based on your data, I think Zwift is picking up the fitness service data. Turn off the fitness service in the code (remove the FITNESS define at the top). See what happens.

Turn off the fitness service in the code (remove the FITNESS define at the top). See what happens.

Wheee that was IT!!
Now the RPM is spot on!
That is so cool 💪
Unfortunately the heart rate is still too low. I will try the same with the serial debug messages on that now.

There is still a bug in the fitness service, or at least a compatibility problem with Zwift.

Ohh okay sorry…

I should have read the specs more carefully. Cadence in FTMS is measured in half-units. Your 120,000 fix was basically right. I corrected this, and also improved the rounding.


Document the standard character limits with auto-generated random routes

As a user, I've the SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_ENABLE_RANDOM_APP_NAME_PREFIX set to true, so I'm able to generate random routes for all the apps. However, when the randomly generated route (with: host + domain) name length is beyond the standard 63 characters, the applications fail to stage/start.
Findings:
Cloud Controller returns an error when creating a route with host or domain name that is longer than specified in RFC 1035.
Acceptance:
Documentation to include the limitations when using the automatic random-route generation feature and the remedy to recover from this situation.

I would rather make sure we surface any error appropriately, rather than re-documenting what is itself a CloudFoundry / RFC limitation

Sounds good; I will update the description.

Closing this in favor of https://github.com/spring-cloud/spring-cloud-deployer-cloudfoundry/issues/89.


Prevent open_basedir restriction warnings

Prevent open_basedir restriction warnings (.. and . folders) when upload_dir is set to webserver root dir.

Thanks for your contribution @programmerg !
I'll accept those changes if you also add a check for strlen($file_name) > 0 to the front of the condition, for cases where the $file_name might be empty.

Thanks @programmerg ☺


Missing quick-start Java Android code for Voice Assistant in GitHub Speech SDK repo

Usually there is a link in these quick-start docs to get the code from the Speech SDK Git Hub repo. However this repo https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/quickstart/java/android is missing the above quick start Voice Assistant code for Java Android
Thanks,
Darren

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: eccff5b5-9038-3871-57ad-913d5739c417
Version Independent ID: 29139141-b9c1-1bf1-525c-38784cff23ac
Content: Quickstart: Create a custom voice assistant - Speech service - Azure Cognitive Services
Content Source: articles/cognitive-services/Speech-Service/quickstarts/voice-assistants.md
Service: cognitive-services
Sub-service: speech-service
GitHub Login: @trrwilson
Microsoft Alias: travisw

@dargilco
Thanks for the feedback! We are currently investigating and will update you shortly.

@dargilco
Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.


feat: add glsl_analyzer

This resolves https://github.com/williamboman/mason.nvim/issues/625

Closes #4443


Ballerina theme looks different

The theme in docs/ballerina-theme folder looks slightly different than the one currently hosted:

@madhuramendis Do you have any thoughts?

This has been fixed in https://github.com/ballerinalang/website.


DataUse & DataCategory Simplifications

The following proposed changes to fideslang follow months of review and feedback from users. Most of these changes are due to confusion when implementing causing hesitation.

Remove identifiable, nonidentifiable, derived, and provided subcategories

Remove the parent categories, combining the subcategories into one

Rename provide.system to provide.service
Move account to a subcategory of user

e.g. user.account.email, user.contact.email

Describe the solution you'd like
Implementing these changes will provide some simplicity when selecting a DataCategory and help reduce the barrier to entry.
Describe alternatives you've considered, if any
Maintaining the existing taxonomy as backwards compatible.
Additional context
Some decision made around this change:

Do not implement backwards compatibility for this breaking change
Some suggested enhancements have been pushed to a later minor release of fideslang

@mfbrown @NevilleS - @brentonmallen1 suggested adding another layer for address prior to the more detailed values (i.e. street
So instead of user.account.street we would have something like user.account.address.street
Any thoughts here? I can include these changes as well if we want to go that route 👍🏽
The only issue I can think of off the top of my head would be if there was a separate use for something like city or state that didn't align with address somehow?

A potential means to  abstract that out a bit further would be using
“location” like user.account.location.city  or
user.account.location.address along with user.account.location.city
On Tue, Jul 5, 2022 at 3:03 PM Steve Murphy @.***>
wrote:

@mfbrown https://github.com/mfbrown @NevilleS
https://github.com/NevilleS - @brentonmallen1
https://github.com/brentonmallen1 suggested adding another layer for
address prior to the more detailed values (i.e. street
So instead of user.account.street we would have something like
user.account.address.street
Any thoughts here? I can include these changes as well if we want to go
that route 👍🏽
The only issue I can think of off the top of my head would be if there was
a separate use for something like city or state that didn't align with
address somehow?
—
Reply to this email directly, view it on GitHub
https://github.com/ethyca/fideslang/issues/58#issuecomment-1175401259,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABZEH3XC65NPAONUFQ5TJWDVSSBGJANCNFSM52FWUY5A
.
You are receiving this because you were mentioned.Message ID:
@.***>

I like that suggestion, to add user.account.address and user.contact.address as subcategories, so we have those for city/street/etc.

Let's do it. I think it's a good improvement and all those address fields (city/street/postal code/etc) were really looking for a home

As part of a large(ish) dataset labeling exercise I've come across the same questions in the last 24 hours.
To list my observations and comments on what's said here:

contact today encapsulates address labels as well as email and phone_number
The first issue with this is that if you have a field that is an entire address i.e. street, city, state, zip, country, you can’t label it correctly - contact isn’t semantically appropriate and it also encapsulates email and phone.
We should introduce user.contact.address and move phone and email up to user.contact
We are also missing building, suite or apartment # equivalent. So basically some version of unit_number or unit (not very intuitive?)
On your point of the edge cases of city and state not part of an address. I would say they should be labeled user.location , however our current use of location, in this context is actually for lat/long data so not sure this is right. Perhaps it should be location with optional sub categories for location.city, location.state and location.coordinates?
A final question though, what's the thinking in having user.account.address and user.contact.address - do we not risk complicating the understanding for manual labeling?

Yes, I think the wrinkle is probably from moving account --> user.account, as in the prior structure account represented something not owned by a user directly (pretty distinct) and I'm wondering if nesting these is now creating a doubling of these branches unnecessarily. I'm going to go back and look at the ISO spec and get back on this later today...

just for some perspective on how these choice can impact things in a practical example, here's a label mapping I'm attempting to do for a separate effort.
https://docs.google.com/spreadsheets/d/1myvSpNPXT5U78B5XZb5p9mFAu1sIJqwsMs6vW36xkE8/edit#gid=0

All good to me @NevilleS, it certainly feels like this better achieves the goal of reducing hesitancy when annotating a dataset while providing more value as described by @brentonmallen1  - thanks for driving this to a conclusion and for the thoughts and feedback from everyone!

ship it!

would location be a better parent than address?

would location be a more generic parent than address?

I feel like your original suggestion of address is pretty comprehensive for the purposes here (street, city, etc.) I think I remember from a live meeting someone mentioning location was generally more in line with point or coordinate data as well

I forgot there was already a user.location


Drop Mambaforge Python distribution, add warning regarding Anaconda

Mambaforge will be discontinued, so don't recommend it in our install docs anymore:
https://conda-forge.org/news/2024/07/29/sunsetting-mambaforge/
Also added a warning that Anaconda's lawyers might come after your if you use their packages in a commercial environment without paying a license fee.

Rendered docs:
https://output.circle-artifacts.com/output/job/4e3f160b-7cc6-4e1f-8a23-7bec690d28d1/artifacts/0/html/install/manual_install_python.html

looks like the CI errors here should be resolved by #12928
(too bad we don't have merge queues enabled)


How do the mildest areas of my server?

tittle =)

Can you make that more clear? That made no sense at all.

How I can make softer spaces?

I don't know what you are referring to.
What do you want to have the "softer spaces"?  What is "mildest areas" and "softer spaces"?  You're saying the object without telling us the subject of the topic.

If it's in the design of the website, edit your main_out.js. Backgrounds, how cells are formed and shaped are not controlled by MultiOgar.


Improve DisplayServer.window_set_current_screen

Prevent unnecessary fullscreen toggle animation on macOS, when screen is not changed
Fix window position on Linux/X11

Thanks!


Fix issue #30

Probably a typo - args list has only one element in this
case.

This will be fixed with the next update. Thanks for the contribution anyway!


feat: Changed username regex to allow email-like usernames

Hey there! First off, a big thanks to everyone for maintaining Casdoor
The motivation of this PR is to make the validation process more generic, allowing email-like usernames. This update ensures that the username validation aligns more closely with real-world usage where dots are frequently used in usernames.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Angel Guidet seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@ajguidet there's no need to change this. If you want to use Email to signup, you can either:

Replace "Username" with "Email" in signup items table of the application edit page
Use "Verification code" instead of "Password" for signin methods table of the application edit page

For username, we don't want to mess it with Email. Username is username, Email is Email. Mixing them will bring trouble for Casdoor, like Alice's username is Bob's Email. So the login via username/Email/Phone will also be fragile. And there's no gain for this.


Fdroid release?

Have you considered releasing this amazing app on F-Droid?
Thanks!

Hey @Seb3thehacker! Thanks for the comments. I am currently working on a bit of a design overhaul and afterwards I will be releasing the application on preferably all the app stores. It's a good idea to include F-Droid as well. Should be somewhere in the coming months.

Love it! Take your time! ❤

What is the current status?

Currently working on getting 2.0 out to the App Store. Google Play will be afterwards, then F-droid. Since the build will be built by F-droid I will need to overhaul the build system a teeny bit. Making good progress, but it'll still take some time.

If you don't (yet) want to deal with F-droid inclusion, there are third-party repos where inclusion is easier than the main repo.
For example IzzyDroid uses your published APK, see details and requirements for app inclusion
Thank you for your work, I still have some more testing to do, but your app already looks very polished for my use case.

I'm well underway in getting Fintunes on F-Droid and I would prefer to go for F-Droid directly. You can track the repository where I'm working on inclusion here: https://gitlab.com/leinelissen/fdroiddata. The process has been quite finicky though because of the large amount of native dependencies. If anyone has any experience with getting stuff onto F-Droid, any help is appreciated.

Fintunes is now available on F-Droid! Find it here: https://f-droid.org/en/packages/nl.moeilijkedingen.jellyfinaudioplayer/


Remove unnecessary fullName argument in api.auth.register

api.auth.provider.registerUser(login, password) accept only two arguments, so fullName is not required.

fullName has been removed.

I believe that, for illustrating the registration method, it would be beneficial to include parameters such as fullName, especially since this is an example project. Although the current code doesn't utilize the fullName field, I suggest considering its inclusion in the database model. This way, it can be sent to api.provider.registerUser during user registration, providing a more comprehensive example for this project. Addressing this issue would enhance the demonstration of the registration process in a practical example

@timursevimli  By the way,  It would be nice to then add fields such as phone number, email, address, and possibly user roles – basically, fields that are present in the standard authentication process for applications to make the example more realistic.


Can someone send or make a livery template for the 747-8 I want to make a Lufthansa and Atlas Air

anyone cant make liveries for B747-8 cause Ls doesnt have LRX licence

oh

I’m sure u can find it in the console or inspection tab.

 But Xavier is probably going to replace all the lrx planes. Like the 777-300ER replaced the LRX one. Plus LRX planes have so many bugs. I contacted and they said we would not fix it. Cargolux is nice but using it for already 4 years is boring.


Partner application link is outdated

File: docker-hub/publish/publisher_faq.md
The URL is just pasted into the start of the FAQ's. Should be updated to this URL
https://goto.docker.com/Partner-Program-Technology.html

Issue moved to docker/docs-private #1146 via ZenHub


Add functionality to move files across folders

Description of the feature
To be able to organise the uploaded datasets, we need to provide our submitters with the necessary functionality to group related files/folders and ensure that the project > study > dataset hierarchy exists
Implementation steps or Task

[ ] Add new folder
[ ] Delete folder
[ ] Move files / folders
[ ] Flag or unFlag folder as study
[ ] Rename folder / file

All of these need to be implemented at the s3 storage level and also the file system objects.

Duplicate of #290


Write Telemetry / Google Analytics product brief

Product brief missing
Proof of concept: #2531

I thought we could use #2372 as the epic for this 🤔

You're right, I'll make this more specifically about writing the product brief then.

Some thoughts from #2671 that might help inform the product spec:

Tracking should be explicitly opt-in and per-user
The opt-in CTA links to the privacy policy
Let's say a big news publisher uses the Web Stories editor - should there be a way for the publisher to disable tracking for all their users?
Let's say the plugin is installed on WordPress multisite, i.e. a network of WordPress sites like WordPress.com with a shared user list - should the opt-in be per-user, per-site? Or if they opt-in somewhere (e.g. site1.wordpress.com), it means they opt-in everywhere (also on site2.wordpress.com, site3.wordpress.com, etc.)?

Are we including JS error reporting in this feature? Tangent: what about in-app "send feedback"?

@choumx JS error reporting is currently listed at https://docs.google.com/spreadsheets/d/17yjdZwVPMiwH-JKNq1smcdiu8iFZpiChYbUCnU_sDxc/edit, so tentatively yes. Happy to add you to the existing email thread about telemetry if you want to know more


Reset function to smart pointers

Right now, I didn't encounter the need for Reset function for Smart pointers anywhere in my code yet. If it's needed I will add in the future.


Depends on window

Any particular reason for this library to access the window object?
Sadly it makes it impossible to use on a NodeJS backend.

Hi @hwaterke this PR should already fix that https://github.com/MozillaReality/ecsy/pull/132
I'll have that in mind in the future changes of the library. Thanks for repoting


Disable Metadata API v1

Hi there,
we got an AWS Security Best Practices finding that the Instance Metadata Service Version 1 (IMDSv1) should be disabled in favor of only supporting Instance Metadata Service Version 2 (IMDSv2). It turns out that this is configured during VM creating, or in this case of bosh stemcells baked into the AMI. Currently it looks like this isn't configured in the stemcell at all so it defaults to supporting both API versions.
Are there any plans for disabling IMDSv1 for the AWS stemcells in the future?

Was this already done here? https://github.com/cloudfoundry/bosh-aws-cpi-release/commit/3876adcd41f947264e7dc1b2eb2ecc2e7d8a893a

@scult could you please check whether the linked commit above works for you? If yes, we are happy to review a docs pr for it because it seems that the documentation was missed.

That worked, thank you very much for your help

For anybody following up on this issue later, I believe the configuration needed to require IMDSv2 is configuring the aws.metadata_options property of the CPI with { 'http_tokens' => 'required' }


Add --date-from-filename to osxphotos import

A common problem when importing photos is setting the correct date (see #840). osxphotos import can set date from EXIF data but not from the filename.  #842 would allow a custom python function to do this but it would also be useful to be able to directly pull the date from the filename.
Use case:
Imported files have the correct date [and time] embedded in the filename but EXIF data is not set or is not correct.  For example, consider the following filenames:

IMG1234_2022_11_17.jpeg
IMG2134_20221117_0931.jpeg

In the first example, the date is 2022-11-17 and in the second, the date/time is 2022-11-17T09:31:00.  Many other variations are of course possible.  The python strptime method of python's datetime can parse date strings using various format codes (see full list at strftime.org).  --date-from-filename would use these patterns to parse the date [and time] from the filename.  For example:
--date-from-filename "_%Y_%m_%d" would match the first example above:
_: match underscore
%Y: match 4-digit year
_: match underscore
%m: match 2-digit month
_: match underscore
%d: match 2 digit day
Multiple formats could be provided separated by |:
--date-from-filename "_%Y_%m_%d|_%Y%m%d_%H%M"
would match either example above.
This would work by first attempting to match the first pattern then the second, and so on.
There would need to be a way to prevent false matches, for example, %Y prevented from matching the first 4 digits if filename was in format IMG_1234_2022_11_17.jpeg.  Possible solutions include a way to anchor the pattern, e.g. _%Y_%m_%d. to match the period in the filename or %Y_%m_%d$ to anchor to end of file stem.  Other possibilities include full regex support but that would be harder to implement and mesh with the strptime format codes.
This could be implemented by creating a dynamically generated regex from the format codes which would help limit false matches.  Something like this:
--date-from-filename "_%Y_%m_%d"
would become
r"\_(\d{4})\_(\d{2})_(\d{2})"

Would be nice if datetime.datetime.strptime() could do this directly but it raises error if string doesn't match the pattern exactly.  However, _strptime.py from python source should be able to do this as it builds the regexes based on locale. I can use that code to build the dynamic regex:
https://github.com/python/cpython/blob/bbac9a8bcc4d7c0692e8b4f62a955f7ca107b496/Lib/_strptime.py#L170

I've implemented my own strptime replacement at strpdatetime that will be a nice way to implement this feature.  For example, if you had filenames with dates embedded in different formats such as:
IMG_1234_2022_11_20.jpg
IMG_4567_2022_11_19.jpg
5678_20221101.jpg

You could parse these with:
--date-from-file "^IMG_*_%Y_%m_%d|*_%Y%m%d"

Fixed in v0.54.4


Make a Rshiny webserver

It would be cool to allow folks to run their own GMEA tests online.

all done


Feature ac 53

I put all of the modifications for this branch, because it's causing some problems in the auto tests.


Display as a client

Implement a way for a display to connect to a WebSocket server as a client so that two embedded displays can be connected.

This feature requires a new library.
If this functionality is desired in the future, it might be a good idea to rename the current "RemoteMicroGUI" to "RemoteMicroGUI-server" and add "RemoteMicroGUI-client" or something similar to this. Keep these pieces of code separate in some way.


can't find an item by keyword

hi,
doing search: http://npmsearch.com/query?q=keywords:brackets-extension&fields=name
returns a number of items, but at least one (https://github.com/zaggino/brackets-eslint/issues/34) is missing: https://www.npmjs.com/package/brackets-eslint
you can see on the npm website that "brackets-extension" keyword is there and this was actually working for a long time (it was being found and returned)

I believe this was an issue with the indexer stalling, closing for now


Doris-spark connector examples

Proposed changes
Provides some examples for doris spark connector using java and scala language.
Types of changes
What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)
[ ] Code refactor (Modify the code structure, format the code, etc...)
i. Two code files has been added.DorisSparkConnectionExampleJava.java and DorisSparkConnectionExampleScala.scala
[ ] Optimization. Including functional usability improvements and performance improvements.
[ ] Dependency. Such as changes related to third-party components.
[ ] Other.

Add a usage documentation


feat(base): generate new image with version 18.6.0

Splitting #712

@admah Don't hesitate to tell me if anything is missing. I took the liberty to clean some docs.

@ivangabriele it looks like there are some failures in testing unrelated to this PR, but that we will need to get sorted out before this is merged. Just wanted to give you a heads up.

@admah Thanks for the heads up ^^. I'll rebase (or merge if you prefer) this branch once master is ready.

@ivangabriele master is good to go. Update this branch and I'll take a look. Thanks!

@admah Sorry for the late update! I just rebased ;). I'm not sure why the CI don't run the new generated job, I guess it's because of Github branch protection rules ^^.

I made a new PR since it's easier than resolving conflicts: https://github.com/cypress-io/cypress-docker-images/pull/738

Closing this now that #738 has been merged.


Keybindings editor shifts up on row select

Open keybindings editor
Select a row
The page shifts up a few pixels

@roblourens I do not see it.. can you please help me in finding it?

I'm not seeing it now, but I do remember seeing it recently. I'll try to see what's going on next time I see it.

Thanks, Please reopen it when you see it again


This action doesn't seem to respect the current working directory.

Describe the bug
In github actions, you can set the default folder from which all actions should be run.
example:
name: CI
on: [push, pull_request]
defaults:
  run:
    working-directory: ./rust-app
jobs:
  cargo-deny:
    runs-on: ubuntu-22.04
    steps:
    - uses: actions/checkout@v3
    - uses: EmbarkStudios/cargo-deny-action@v1

However, this action does not respect the default folder and instead runs from the root directory of the repo.
here is an example repo / workfow:
https://github.com/ctriley/cargo-deny-example/actions/runs/7349611506/job/20009854868

Same issue here...

The docs for github actions (somewhat) clearly state that working-directory only applies to .run actions, the docker run command is not under control of this action (AFAIK) and you can see /usr/bin/docker run --name eaa03abb2882f8a99948049e12e8fbacd21fc4_1aecb8 --label eaa03a --workdir /github/workspace where the working directory for the container is always the repo root. I've added the manifest-path key to the arguments if you want to use that, or you can also specify arguments: --manifest-path <path-to-Cargo.toml> to select whichever Cargo.toml manifest you want.

@Jake-Shadle Thanks a lot!
- name: Cargo Deny
        uses: EmbarkStudios/cargo-deny-action@v1
        with:
          manifest-path: extensions/oidc/Cargo.toml

worked for me.
When putting
- name: Cargo Deny
        uses: EmbarkStudios/cargo-deny-action@v1
        with:
           arguments: --manifest-path: extensions/oidc/Cargo.toml

it says that i cannot define manifest path multiple times.. Or am i misunderstanding your previous comment?

You need to remove the : after --manifest-path

Got it. Thanks again!


Allow reading PEM's with an AsymmetricCipherKeyPair. Allow access to Pkcs10CertificationRequest

A few minor changes which fix blocking issues using Certes to generate certificates using files from 'Crypt-LE'. This will:

Allow KeyAlgorithmProvider.GetKey(string pem) to work if the PemReader.ReadObject() returns an AsymmetricKeyParameter instead of a AsymmetricCipherKeyPair.
Enable public access to the actual Pkcs10CertificationRequest which CertificationRequestBuilder uses internally.

This project will die over that bot closing issues unattended.

This project will still die over that bot closing issues unattended.

Boo bot

Hey, @ericwj  are you using this change in production within an existing tool?

p.s. why Pkcs10CertificationRequest is being exposed as part of this PR? How does certes relate to Crypt-LE?

I shouldn't have mentioned Crypt-LE, sorry about that. Maybe I got an AsymmetricKeyParameter because I used some other tool with LetsEncrypt before of which I have files on disk, if Certes isn't able to encounter them normally.
In the latest production code I use, I use the equivalent of CertificationRequestBuilder.GeneratePkcs10().GetDerEncoded().
Anyhow Certes used to be the best thing around in C# before it stopped being maintained and the bot started auto-closing issues. I'm well aware this PR exposes types publicly from BouncyCastle and might not be the preferred way of fixing my issue, but sure let's talk about it.

@ericwj regarding CertificationRequestBuilder.GeneratePkcs10().GetDerEncoded() I think you can call Generate() as that will in turn run GeneratePkcs10() then return the result of GetDerEncoded(). If you can exclude that from this PR and we can determine that exiting tests cover this change then I think we'd be ok to merge it.
This isn't my library but I do use it, and I suspect @fszlin doesn't currently have a lot of time for it (or has no current use for it), which is exactly what's great about open source - projects can continue even if the author loses interest. I have one or two PRs of my own to submit shortly.

Sure I am using this PR already from my own fork, but that has all sorts of caveats and annoyances associated with it. Do you have the required permissions such that you are able to accept PR's into this repo and actually publish version updates to the NuGet packages?
Regarding Pkcs10CertificationRequest, an older project still uses GeneratePkcs10 and passes the result to Org.BouncyCastle.OpenSsl.PemWriter. I guess it would suffice what this project needs to add an API to export to PEM instead of exposing GeneratePkcs10.

@ericwj it would appear that I can merge PRs (I don't remember being given permission) but I can't update nuget. Regarding PemWriter, as mentioned you can get the Der encoded cert via .Generate(), so you should be able to use that to pass to pemwriter?

I don't think so, I'd get a byte array. How would PemWriter know what it represents and how to write that properly?
Plus it defeats not having BouncyCastle stuff in the public API, if I then still need to mess with that myself. WriteObject is a total mystery to begin with. It takes object, not even Asn1Encodable, System.Object. It doesn't make sense.

Weird that it fails on MacOS only. Anyway the build SDKs are out of date.

Re-ran and it passed. Good enough for me :)

ty


Data is not present after PVC restore from snapshot

Describe the bug
I follow the guide of https://github.com/ceph/ceph-csi/blob/master/examples/README.md#how-to-test-rbd-snapshot-feature  to create and restore a snapshot but failed.
Did I do something wrong?
Environment details

Image/version of Ceph CSI driver :  cephcsi canary, master branch
Kubernetes cluster version: v1.14.3
external-snapshotter: v1.2.2
external-provisioner: v1.4.0
Logs

Steps to reproduce
Steps to reproduce the behavior:
follow the guidance of the Md.
https://github.com/ceph/ceph-csi/blob/master/examples/README.md#how-to-test-rbd-snapshot-feature

kubectl create -f snapshotclass.yaml
kubectl create -f snapshot.yaml
After this, I ran kubectl get volumesnapshot and got nothing.
Instead, I got volumesnapshot when I ran this cmd kubectl get volumesnapshots.snapshot.storage.k8s.io

root@master1:~/wz/ceph-csi-2.0.0-canary/example/rbd# kubectl get volumesnapshot -n test
No resources found.
root@master1:~/wz/ceph-csi-2.0.0-canary/example/rbd# kubectl get volumesnapshots.snapshot.storage.k8s.io -n test
NAME                      AGE
rbd-pvc-snapshot-canary   3h55m

kubectl create -f pvc-restore.yaml
kubectl create -f pod-restore.yaml

Actual results
The pvc created is just like a brand new pvc, with no data of snapshot in it.
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
The log of external-provisioner showed that the VolumeContentSource is nil in CreateVolumeRequest.
I0116 07:57:57.685447       1 controller.go:1199] provision "test/rbd-pvc-restore-canary" class "csi-rbd": started
I0116 07:57:57.690409       1 event.go:255] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"test", Name:"rbd-pvc-restore-canary", UID:"e8b0a447-3835-11ea-8827-eeeeeeeeeeee", APIVersion:"v1", ResourceVersion:"85042625", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "test/rbd-pvc-restore-canary"
I0116 07:57:57.690294       1 controller.go:494] CreateVolumeRequest {Name:pvc-e8b0a447-3835-11ea-8827-eeeeeeeeeeee CapacityRange:required_bytes:11811160064  VolumeCapabilities:[mount:<fs_type:"ext4" mount_flags:"discard" > access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[clusterID:12734c43-4005-4aef-96e7-dfff3fcadd93 csi.storage.k8s.io/controller-expand-secret-name:ceph-secret-csi csi.storage.k8s.io/controller-expand-secret-namespace:default csi.storage.k8s.io/fstype:ext4 csi.storage.k8s.io/node-stage-secret-name:ceph-secret-csi csi.storage.k8s.io/node-stage-secret-namespace:default csi.storage.k8s.io/provisioner-secret-name:ceph-secret-csi csi.storage.k8s.io/provisioner-secret-namespace:default imageFeatures:layering imageFormat:2 pool:rbdpool] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0116 07:57:57.695164       1 connection.go:180] GRPC call: /csi.v1.Controller/CreateVolume
I0116 07:57:57.695204       1 connection.go:181] GRPC request: {"capacity_range":{"required_bytes":11811160064},"name":"pvc-e8b0a447-3835-11ea-8827-eeeeeeeeeeee","parameters":{"clusterID":"12734c43-4005-4aef-96e7-dfff3fcadd93","imageFeatures":"layering","imageFormat":"2","pool":"rbdpool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4","mount_flags":["discard"]}},"access_mode":{"mode":1}}]}
I0116 07:58:02.177380       1 leaderelection.go:282] successfully renewed lease test/rbd-csi

root@master1:~/wz/ceph-csi-2.0.0-canary/example/rbd# cat snapshot.yaml
---
apiVersion: snapshot.storage.k8s.io/v1alpha1
kind: VolumeSnapshot
metadata:
  name: rbd-pvc-snapshot-canary
spec:
  snapshotClassName: csi-rbdplugin-snapclass
  source:
    name: rbd-pvc-canary
    kind: PersistentVolumeClaim

root@master1:~/wz/ceph-csi-2.0.0-canary/example/rbd# cat pvc-restore.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: rbd-pvc-restore-canary
spec:
  storageClassName: csi-rbd
  dataSource:
    name: rbd-pvc-snapshot-canary
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 11Gi

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: rbd-csi
  creationTimestamp: "2020-01-16T07:57:57Z"
  finalizers:
  - kubernetes.io/pvc-protection
  name: rbd-pvc-restore-canary
  namespace: test
  resourceVersion: "85042990"
  selfLink: /api/v1/namespaces/test/persistentvolumeclaims/rbd-pvc-restore-canary
  uid: e8b0a447-3835-11ea-8827-eeeeeeeeeeee
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 11Gi
  storageClassName: csi-rbd
  volumeMode: Filesystem
  volumeName: pvc-e8b0a447-3835-11ea-8827-eeeeeeeeeeee
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 11Gi
  phase: Bound

In this issue, I used k8s cluster of version 1.14.3 and ceph cluster of version 12.2.11 (may be too old). So today I have another try of rbd snapshot using rook-ceph (rook 1.2, and ceph 14.2.6) and cephcsi(relase2.0.0) in a new k8s (version1.16.2) cluster, and got the same result of this issue.
So I opened a new issue https://github.com/rook/rook/issues/4828  @Madhu-1

I provided the detailed steps in the new issue. Please check, thank you! @Madhu-1

@windayski I spend a whole day struggling with same issue and finally realized VolumeSnapshotDataSource feature gate is disabled. Is it your case?

@reith Thanks for your reply. I enabled VolumeSnapshotDataSource feature gate for both kube-controller-manager and kubelet, it's not working. Did I miss anything else?

@reith Thanks for your reply. I enabled VolumeSnapshotDataSource feature gate for both kube-controller-manager and kubelet, it's not working. Did I miss anything else?

you also need to add it to api-server also

@Madhu-1  It works. Thanks. Close issue.


Feat: Torus solana wallet integration

Following things were added-

Torus solana wallet integration (https://docs.tor.us/solana-wallet/api-reference/installation)
Added detected parameters in the wallets
Updated /packages/ui/react-ui components
Updated /pacakages/starter/react-ui-starter
Updated wallet flow with easier signup process.

As a user of this package, I would recommend against tying support with Torus until they fix things like this issue which makes it hard to this package into your app.

@vpontis The PR has been merged and released in 1.1.8.

This PR seems a bit more broad than it needs to be. It contains

changes to the Torus adapter
a new detected property added to the Wallet interface
changes to UI components

Could we limit the scope of this PR to just the Torus changes? This would help us review and merge quickly.
I can see the value of the detected change, and I think we should evaluate it in a separate PR. The adapters have a ready property which

serves effectively the same purpose
is expected to be updated asynchronously and fires events

This is important because wallet extensions/injections aren't always loaded immediately. Relying on this synchronously in the UI isn't very safe, so the UI changes that require the detected property should be a followup once we get the design for detecting wallets right. This may mean removing the ready API we have now and decoupling the ready event from the adapter.
@rathishubham7 what do you think?

Yes. The ready property could be used for wallet detection. I'll limit the scope of this PR to torus related changes only.
Closing this.


Is it possible to use ComBat in native space?

Hello,
We know that ComBat requires the images to be registered to a common template space. For some of our analyses, we want to get the value of subjects in native space. Is it possible to use ComBat in native space? Or can we firstly use ComBat for all in a template space, then project the subjects back to native space? Any risk for this procedure (i.e. introduce the bias back again)?
Thanks!

ComBat requires the voxels to be aligned across subjects, therefore the need of a registration to template. As you suggested, back transforming ComBat-normalized values to subject space is a viable solution. We have not investigated the performance of ComBat across different registration algorithms, but we do not think this introduces a bias of any sort.

Thanks. It really helps.


[afreecatv] Support catchstory

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

Support afreecaTv catch-story(similar to YouTube Shorts)
Closes #10112
Template

Before submitting a pull request make sure you have:

[X] At least skimmed through contributing guidelines including yt-dlp coding conventions
[X] Searched the bugtracker for similar pull requests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

[X] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Fix or improvement to an extractor (Make sure to add/update tests)
[X] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

Thanks

I'm sorry I bothered you.
I have a question. AfreecaTv is scheduled to change to sooplive.co.kr around October.
Then, can I create a PR to change the existing extractor name? Or am I wondering if I should keep the existing extractor and create a new extractor.

I'm sorry I bothered you.

No bother at all. It's all good.

I have a question. AfreecaTv is scheduled to change to sooplive.co.kr around October.
Then, can I create a PR to change the existing extractor name? Or am I wondering if I should keep the existing extractor and create a new extractor.

We typically try to avoid changing the extractor class names. We could change the IE_NAME value(s), but that would only be appropriate if the video ids are going to carry over from afreecatv to sooplive. If not, then we should just write new extractor(s)


typeid error when enabling GPU offloading in the compiler

I posted this question in the general forum at first with no answer so far *, but it is probably better placed here. I can't successfully compile code that includes offloading from OpenMP to Vega20, since aomp seems not to support typeid requests anymore when gpu offloading is enabled in the compiler. Does anybody know how to solve this problem?
I am using rocm 2.4 on Ubuntu 18.04 with aomp 0.6-2 and OpenMP. The code works fine as long as I offload to the CPUs. However, once I offload to my GPU it only compiles and works when I remove all typeid requests. But I need these requests to pass the results and data from and to Rcpp.
I provide a minimal example by simply adding a single typeid request and the required header to one of the working examples from aomp, namely reduction.c:
aomp example
`#include <stdio.h>
#define N   1000000ll
#define SUM (N * (N-1)/2)
int main (void)
{
long long a, i;
//std::cout << typeid(a).name() << std::endl;
#pragma omp target parallel shared(a) private(i)
{
#pragma omp master
a = 0;
#pragma omp barrier

#pragma omp for reduction(+:a)
for (i = 0; i < N; i++) {
    a += i;
}

// The Sum shall be sum:[0:N]
#pragma omp single
{
  if (a != SUM)
    printf ("Incorrect result = %lld, expected = %lld!\n", a, SUM);
  else
    printf ("The result is correct = %lld!\n", a);
}

}
return 0;
}`
This works fine when compiled unchanged with the following options:
$ /opt/rocm/aomp/bin/clang++ -frtti -target x86_64-pc-linux-gnu -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx906 reduction.c
However, when I uncomment the single typeid request in the code above and add the required headers at the very top:
#include <typeinfo> #include <iostream>
the compilation fails with the error:
reduction.c:12:14: error: use of typeid requires -frtti
which is however already used. Once I compile with CPU options only, it works again:
$ /opt/rocm/aomp/bin/clang++ -target x86_64-pc-linux-gnu -fopenmp  reduction.c
Does anybody know the reason for this problem and how it can be solved? Any hint would be greatly appreciated!

Hi D-Dirk,
Thanks for cross posting here in AOMP, where we on the AOMP team pay more attention.
So we have some code in clang that explictily adds -fno-rtti flag , regardless of whether the user asked for -frtti
As an experiment, I added the following test to see if the user explicitly added -frtti and then honor it
which seems to allow your minimal test case to compile and run.
To use this of course, you need to build our AOMP product.
This change has not been reviewed by anyone, so its not necessarily correct.
I will ask my peers to look at the change, and if they are ok with it, it could get in our imminent 0.6-3 release (not sure the exact date).
diff --git a/lib/Driver/ToolChains/Clang.cpp b/lib/Driver/ToolChains/Clang.cpp
index bc7a8e6..3e7b4f9 100644
--- a/lib/Driver/ToolChains/Clang.cpp
+++ b/lib/Driver/ToolChains/Clang.cpp
@@ -4658,7 +4658,8 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
if (KernelOrKext || (types::isCXX(InputType) &&
(IsCuda || IsHIP || IsOpenMPDevice ||
(RTTIMode == ToolChain::RM_Disabled))))

CmdArgs.push_back("-fno-rtti");
+i   if (!Args.hasArg(options::OPT_frtti))

 CmdArgs.push_back("-fno-rtti");

// -fshort-enums=0 is default for all architectures except Hexagon.
if (Args.hasFlag(options::OPT_fshort_enums, options::OPT_fno_short_enums,

Ron.

I created a clang PR on our development branch.
https://github.com/ROCm-Developer-Tools/clang/pull/7

Great, thank you very much! Can you let us know as soon as  this change appears in some development or experimental version? I am only a beginner and started last week, but I would be happy to try it! It would really be great to have it in 0.6-3, since I am afraid it is required to use GPUs and aomp directly from Rcpp. This feature would however be fantastic. Everything else seems to work already with Rcpp, so this might be the final step.

Waiting to hear back from our project lead as to when we can spin out a release.
i am curious , what company/organization are you affliliated with?
if you want to answer offline, ron.lieberman @ amd '.' com

This is for academic research and will hopefully be used to speed up MCMC simulations. I think many R users will be happy to get GPU support in this way with essentially no additional programming effort. I don't even have a technical background but Rcpp and aomp do a great job of making these things very easy to access. Once it works, I can post it to the Rcpp mailing list so probably others will get interested and test aomp in their applications. Thank you very much for your help!

@D-Dirk we should have the release 0.6-3 out today with a resolution for your issue.

looks like 0.6-3 is released now
https://github.com/ROCm-Developer-Tools/aomp/releases/tag/rel_0.6-3
closing this issue


Fix enterexit coloring for rects

you know, that thing matt was supposed to do but somehow missed...

matt did it


Added vectorized implementation for mat4.{scale, transpose, translate}

Vectorized implementation for mat4.scale, mat4.transpose and mat4.translate

Looks good, but there's a merge conflict. Please rebase and I'll pull it in!

tests!

@nickdesaulniers: This should be automatically tested with the current mat4 test structure.
Unfortunately I'm still seeing conflicts. Sorry! Don't know what the issue is.

This should be automatically tested with the current mat4 test structure.

This seems to only be the case where TEST_PATH is only being used for multiply in the tests, not other functions like the ones being added in this PR.
I'll work with @sajjadt on resolving conflicts.


improve the footer of contact us page

And It is also not responsive, @codervivek5 could you please assign me to this issue. So, I can fix the responsive footer. Thank You.


[Question]: Unable to install on Mac

What is your question?
I followed the instructions but unable to install fabric on Mac.
"
Fatal error from pip prevented installation. Full pip output in file:
/Users/.local/pipx/logs/cmd_----_pip_errors.log
pip failed to build package:
chroma-hnswlib
Some possibly relevant errors from pip install:
error: subprocess-exited-with-error
clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=14'
RuntimeError: Unsupported compiler -- at least C++11 support is needed!
Error installing fabric from spec '/Users/git/fabric'.
"
I checked C++ compiler and it is installed on my machine.
Any suggestions ? thanks.

@AhsanT Im on macos 14.5 and just installed it yesterday.
Which os version are you on, and have you installed xcode cli tools with xcode-select --install and approved xcode eula?

Updated the Xcode and it worked, thanks for your help :)


fix libqglviewer-qt4 and libqglviewer-qt4-dev debian jessie rules

Follow up of https://github.com/ros/rosdistro/pull/14424/
libqglviewer-qt4-dev: https://packages.debian.org/stretch/libqglviewer-dev-qt4
libqglviewer-qt4: https://packages.debian.org/stretch/libqglviewer2-qt4

@ahornung FYI

@ahornung Now that the rosdep rule is fixed you will need to rerun bloom to take the new rule into account and get octovis passing for all platforms (except armhf where I blacklisted octovis because the libqglviewer package doesn't exist for this platform)
Thanks!

Thanks for taking care, @mikaelarguedas. Will rerun right now.