[Bug] [dashboard]  'ClickhouseConnector' object has no attribute 'session'

Search before asking

[X] I had searched in the issues and found no similar issues.

Operating system information
Linux
Python version information
3.10
DB-GPT version
main
Related scenes

[ ] Chat Data
[ ] Chat Excel
[ ] Chat DB
[ ] Chat Knowledge
[ ] Model Management
[X] Dashboard
[ ] Plugins

Installation Information

[X] Installation From Source

[ ] Docker Installation

[ ] Docker Compose Installation

[ ] Cluster Installation

[ ] AutoDL Image

[ ] Other

Device information
local openai api
Models information
llam3
What happened
Use the dashboard to query clickhouse
clean prompt response: [{     "sql": "SELECT country, port, bandwidth, outmax FROM traffic_data ORDER BY outmax DESC LIMIT 3",     "title": "Top 3 Highest Traffic Ports by Country",     "showcase": "BarChart",     "thoughts": "This SQL query selects the country, port, bandwidth, and the maximum output traffic for the top three ports with the highest output traffic from the 'traffic_data' table. A BarChart is chosen to visually compare the traffic across different countries and ports." }]
2024-05-07 15:00:38 pnma-ck dbgpt.datasource.rdbms.base[26170] INFO Query[SELECT country, port, bandwidth, outmax FROM traffic_data ORDER BY outmax DESC LIMIT 3]
'ClickhouseConnector' object has no attribute 'session'
What you expected to happen
'ClickhouseConnector' object has no attribute 'session'
How to reproduce
Use the dashboard to query clickhouse
ClickHouse client version <IP_ADDRESS> (official build).
DBGPT 5.5
Additional context
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

I installed the new clickhouse test (<IP_ADDRESS>8)，used the agent query, and after a few automatic retries, it returned a normal result.
2024-05-07 15:37:08 pnma-ck dbgpt.agent.expand.data_scientist_agent[38979] ERROR DataScientist check exception！'ClickhouseConnector'object has no attribute 'session'
Traceback (most recent call last):
File "/app/DB-GPT/dbgpt/agent/expand/data_scientist_agent.py", line 110, in correctness_check
columns, values = await resource_db_client.query(
File "/app/DB-GPT/dbgpt/serve/agent/resource_loader/datasource_load_client.py", line 62, in query
return conn.query_ex(sql)
File "/app/DB-GPT/dbgpt/datasource/rdbms/base.py", line 356, in query_ex
cursor = self.session.execute(text(query))
AttributeError: 'ClickhouseConnector' object has no attribute 'session'

Edgar (to PlanManager)-[chatgpt_proxyllm]:
"{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}"

Edgar Review info:
Pass(None)
Edgar Action report:
execution succeeded,
{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL��句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}

2024-05-07 15:37:08 pnma-ck dbgpt.datasource.manages.connect_config_db[38979] INFO Result: <sqlalchemy.engine.cursor.CursorResult object at 0x7f22783f0dc0>

PlanManager (to Edgar)-[]:
"SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'"

2024-05-07 15:37:08 pnma-ck dbgpt.agent.core.base_agent[38979] INFO _a_select_llm_model:None
2024-05-07 15:37:08 pnma-ck dbgpt.agent.core.llm.llm_client[38979] INFO Request:
{'model': 'chatgpt_proxyllm', 'prompt': None, 'messages': [{'content': 'You are a DataScientist,named Edgar,your goal is Use correct clickhouse SQL to analyze and solve tasks based on the data structure information of the database given in the resource.\nPlease thinkstep by step to achieve the goal. You can use the resources given below. At the same time, please strictly abide by the constraints and specifications in IMPORTANT REMINDER.\n\nRESOURCES:clickhouse  --['bgp_lax2_old_csv(name-dashboard-rowname-country-panelname-device-port-type-isp-bandwidth-highspeed-inid-outid-clock-inmax-outmax-outavg-inavg-inpercent-outpercent)']\n\n\n\n*** IMPORTANT REMINDER ***\nPlease answer in simplified Chinese.\n1. Please check the generated SQL carefully. Please strictly abide by the data structure definition given. It is prohibited to use non-existent fields and data values. Do not use fields from table A to table B. You can perform multi-table related queries.\n2. If the data and fields that need to be analyzed in the target are in different tables, it is recommended to use multi-table correlation queries first, and pay attention to the correlation between multiple table structures.\n3. It is forbidden to construct data by yourself as a query condition. If you want to query a specific field, if the value of the field is provided, then you can perform a group statistical query on the field.\n4. Please select an appropriate one from the supported display methods for data display. If no suitable display type is found, table display is used by default. Supported display types: \n response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable forexploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of timeseries data, large-scale data sets, distribution of classified data, etc.\n\n\nPlease response in the following json format:\n    {\n  "display_type": "The chart rendering method selected for SQL. If you don’t know what to output, just output 'response_table' uniformly.",\n  "sql": "Executable sql generated for the current target/problem",\n  "thought": "Summary of thoughts to the user"\n}\n        Make sure the response is correct json and can be parsed by Python json.loads.\n        ', 'role': 'system'}, {'content': "执行SQL查询，选取'port'列中的最大三个值。", 'context': {'plan_task': "执行SQL查询，选取'port'列中���最大三个值。", 'plan_task_num': 1}, 'role': 'human'}, {'content': '{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"根据要求，我编写了一个SQL查询，该查询从'bgp_lax2_old_csv'表中选择'port'列，并按照降序排列，然后选取前三个最大值。结果将以表格形式展示。"}', 'context': {'display_type': 'response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.', 'dialect': 'clickhouse'}, 'role': 'ai'}, {'content': "SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'", 'context': {'display_type': 'response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time seriesdata, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.', 'dialect': 'clickhouse'}, 'role': 'human'}, {'content': '{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}', 'context': {'display_type': 'response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time seriesdata, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.', 'dialect': 'clickhouse'}, 'role': 'ai'}, {'content': "SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'", 'role': 'human'}], 'temperature': 0.5, 'max_new_tokens': 1024, 'echo': False}
2024-05-07 15:37:08 pnma-ck dbgpt.model.adapter.base[38979] INFO Message version is v2
2024-05-07 15:37:08 pnma-ck dbgpt.model.cluster.worker.default_worker[38979] INFO current generate stream function is asynchronous stream function
llm_adapter:
model prompt:
system: You are a DataScientist,named Edgar,your goal is Use correct clickhouse SQL to analyze and solve tasks based on the data structure information of the database given in the resource.
Please think step by step to achieve the goal. You can use the resources given below. At the same time, please strictly abide by the constraints and specifications in IMPORTANT REMINDER.
RESOURCES:clickhouse  --['bgp_lax2_old_csv(name-dashboard-rowname-country-panelname-device-port-type-isp-bandwidth-highspeed-inid-outid-clock-inmax-outmax-outavg-inavg-inpercent-outpercent)']
*** IMPORTANT REMINDER ***
Please answer in simplified Chinese.

Please check the generated SQL carefully. Please strictly abide by the data structure definition given. It is prohibited to use non-existent fields and data values. Do not use fields from table A to table B. You can perform multi-table related queries.
If the data and fields that need to be analyzed in the target are in different tables, it is recommended to use multi-table correlation queries first, and pay attention to the correlation between multiple table structures.
It is forbidden to construct data by yourself as a query condition. If you want to query a specific field, if the value of the field is provided, then you can perform a group statistical query on the field.
Please select an appropriate one from the supported display methods for data display. If no suitable display type is found, table display is used by default. Supported display types:
response_line_chart:used to display comparative trend analysis data
response_pie_chart:suitable for scenarios such as proportion and distribution statistics
response_table:suitable for display with many display columns or non-numeric columns
response_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.
response_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.
response_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories,etc.
response_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.
response_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.

Please response in the following json format:
{
"display_type": "The chart rendering method selected for SQL. If you don’t know what to output, just output 'response_table' uniformly.",
"sql": "Executable sql generated for the current target/problem",
"thought": "Summary of thoughts to the user"
}
Make sure the response is correct json and can be parsed by Python json.loads.
human: 执行SQL查询，选取'port'列中的最大三个值。
ai: {"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"根据要求，我编写了一个SQL查询，该查询从'bgp_lax2_old_csv'表中选择'port'列，并按照降序排列，然后选取前三个最大值。结果将以表格形式展示。"}
human: SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'
ai: {"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}
human: SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'
async stream output:
2024-05-07 15:37:08 pnma-ck dbgpt.model.proxy.llms.chatgpt[38979] INFO Send request to openai(1.25.2), payload: {'stream': True, 'model': 'mixtral-8x7b', 'temperature': 0.5, 'max_tokens': 1024}
messages:
[{'role': 'system', 'content': 'You are a DataScientist,named Edgar,your goal is Use correct clickhouse SQL to analyze and solve tasks based on the data structure information of the database given in the resource.\nPlease think step by step to achieve the goal. You can use the resources given below. At the same time, please strictly abide by the constraints and specifications in IMPORTANT REMINDER.\n\nRESOURCES:clickhouse  --['bgp_lax2_old_csv(name-dashboard-rowname-country-panelname-device-port-type-isp-bandwidth-highspeed-inid-outid-clock-inmax-outmax-outavg-inavg-inpercent-outpercent)']\n\n\n\n*** IMPORTANT REMINDER ***\nPlease answer in simplified Chinese.\n1. Please check the generated SQL carefully. Please strictly abide by the data structure definition given. It is prohibited to use non-existent fields and data values. Do not use fields from table A to table B. You can perform multi-table related queries.\n2. If the data and fields that need to be analyzed in the target are in different tables, it is recommended to use multi-table correlation queries first, and pay attention to the correlation between multiple table structures.\n3. It is forbidden to construct data by yourself as a query condition. If you want to query a specific field, if the value of the field is provided, then you can perform a group statistical query on the field.\n4. Please select an appropriate one from the supported display methods for data display. If no suitable display type is found, table display is used by default. Supported display types: \n response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.\n\n\nPlease response in the following json format:\n            {\n  "display_type": "The chart rendering method selected for SQL. If you don’t know what to output, just output 'response_table' uniformly.",\n  "sql": "Executable sql generated for the current target/problem",\n  "thought": "Summary of thoughts to the user"\n}\n        Make sure the response is correct json and can be parsed by Python json.loads.\n        '}, {'role': 'user', 'content': "执行SQL查询，选取'port'列中的最大三个值。"}, {'role': 'assistant', 'content': '{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"根据要求，我编写了一个SQL查询，该查询从'bgp_lax2_old_csv'表中选择'port'列，并按照降序排列，然后选取前三个最大值。结果将以表格形式展示。"}'}, {'role': 'user', 'content': "SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'"}, {'role': 'assistant', 'content': '{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或���法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}'}, {'role': 'user', 'content': "SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'"}]
{"2024-05-07 15:37:09 pnma-ck dbgpt.model.cluster.worker.default_worker[38979] INFO is_first_generate, usage: None
display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保使用正确的库方法或上下文执行此SQL。"}
full stream output:
{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保使用正确的库方法或上下文执行此SQL。"}
model generate_stream params:
{'model': 'chatgpt_proxyllm', 'messages': [ModelMessage(role='system', content='You are a DataScientist,named Edgar,your goal is Use correct clickhouse SQL to analyze and solve tasks based on the data structure information of the database given in the resource.\nPlease think step by step to achieve the goal. You can use the resources given below. At the same time, please strictly abide by the constraints and specifications in IMPORTANT REMINDER.\n\nRESOURCES:clickhouse  --['bgp_lax2_old_csv(name-dashboard-rowname-country-panelname-device-port-type-isp-bandwidth-highspeed-inid-outid-clock-inmax-outmax-outavg-inavg-inpercent-outpercent)']\n\n\n\n*** IMPORTANT REMINDER \nPlease answer in simplified Chinese.\n1. Please check the generated SQL carefully. Please strictly abide by the data structure definition given. It is prohibited to use non-existent fields and data values. Do not use fields from table A to table B. You can perform multi-table related queries.\n2. If the data and fields that need to be analyzed in the target are in different tables, it is recommended to use multi-table correlation queries first, and pay attention to the correlation between multiple table structures.\n3. It is forbidden to construct data by yourself as a query condition. If you want to query a specific field, if the value of the fieldis provided, then you can perform a group statistical query on the field.\n4. Please select an appropriate one from the supported display methods for data display. If no suitable display type is found, table display is used by default. Supported display types: \n response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships betweenmultiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportion display and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysisof time series data, large-scale data sets, distribution of classified data, etc.\n\n\nPlease response in the following json format:\n            {\n  "display_type": "The chart rendering method selected for SQL. If you don’t know what to output, just output 'response_table' uniformly.",\n  "sql": "Executable sql generated for the current target/problem",\n  "thought": "Summary of thoughts to theuser"\n}\n        Make sure the response is correct json and can be parsed by Python json.loads.\n        ', round_index=0), ModelMessage(role='human', content="执行SQL查询，选取'port'列中的最大三个值。", round_index=0), ModelMessage(role='ai', content='{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"根据要求，我编写了一个SQL查询，该查询从'bgp_lax2_old_csv'表中选择'port'列，并按照降序排列，然后选取前三个最大值。结果将以表格形式展示。"}', round_index=0), ModelMessage(role='human', content="SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'", round_index=0), ModelMessage(role='ai', content='{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}', round_index=0), ModelMessage(role='human', content="SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'", round_index=0)], 'temperature': 0.5, 'max_new_tokens': 1024, 'echo': False, 'span_id': '85b3817a-68bf-4f0c-b4a4-1ba853e0c15e:c3747402-06f1-47fa-ac7c-2f718615e842', 'context': {'stream': False, 'cache_enable': False, 'user_name': None, 'sys_code': None, 'conv_uid': None, 'span_id': None, 'chat_mode': None, 'chat_param': None, 'extra': {}, 'request_id': None}, 'convert_to_compatible_format': False, 'string_prompt': 'system: You are a DataScientist,named Edgar,your goal is Use correct clickhouse SQL to analyze and solve tasks based on the data structure information of the database given in the resource.\nPlease think step by step to achieve the goal. You can use the resources given below. At the same time, please strictly abide by the constraints and specifications in IMPORTANT REMINDER.\n\nRESOURCES:clickhouse  --['bgp_lax2_old_csv(name-dashboard-rowname-country-panelname-device-port-type-isp-bandwidth-highspeed-inid-outid-clock-inmax-outmax-outavg-inavg-inpercent-outpercent)']\n\n\n\n IMPORTANT REMINDER ***\nPlease answer in simplified Chinese.\n1. Please check the generated SQL carefully. Please strictly abide by the data structure definition given. It is prohibited to use non-existent fields and data values. Do not use fields from table A to table B. You can perform multi-table related queries.\n2. If the data and fields that need to be analyzed in the target are in different tables, it is recommended to use multi-table correlation queries first, and pay attention to the correlation between multiple table structures.\n3. It is forbidden to constructdata by yourself as a query condition. If you want to query a specific field, if the value of the field is provided, then you can perform a group statistical query on the field.\n4. Please select an appropriate one from the supported display methods for data display.If no suitable display type is found, table display is used by default. Supported display types: \n response_line_chart:used to display comparative trend analysis data\nresponse_pie_chart:suitable for scenarios such as proportion and distribution statistics\nresponse_table:suitable for display with many display columns or non-numeric columns\nresponse_scatter_plot:Suitable for exploring relationships between variables, detecting outliers, etc.\nresponse_bubble_chart:Suitable for relationships between multiple variables, highlighting outliers or special situations, etc.\nresponse_donut_chart:Suitable for hierarchical structure representation, category proportiondisplay and highlighting key categories, etc.\nresponse_area_chart:Suitable for visualization of time series data, comparison of multiple groups of data, analysis of data change trends, etc.\nresponse_heatmap:Suitable for visual analysis of time series data, large-scale data sets, distribution of classified data, etc.\n\n\nPlease response in the following json format:\n            {\n  "display_type": "The chart rendering method selected for SQL. If you don’t know what to output, just output 'response_table' uniformly.",\n  "sql": "Executable sql generated for the current target/problem",\n  "thought": "Summary of thoughts to the user"\n}\n        Make sure the response is correct json and can be parsed by Python json.loads.\n        \nhuman: 执行SQL查询，选取'port'列中的最大三个值。\nai:{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"根据要求，我编写了一个SQL查询，该查询从'bgp_lax2_old_csv'表中选择'port'列，并按照降序排列，然后选取前三个最大值。结果将以表格形式展示。"}\nhuman: SQLexecution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session'\nai: {"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"错误信息与SQL语句本身无关，它指出的是执行环境或库的问题，即'ClickhouseConnector'对象缺少执行SQL的'session'属性。确保使用正确的上下文或方法来执行SQL。此SQL语句正确，旨在从'bgp_lax2_old_csv'表中获取'port'列的前三大的值，并以表格形式展示。"}\nhuman: SQL execution error, please re-read the historical information to fix this SQL. The error message is as follows:'ClickhouseConnector' object has no attribute 'session''}
un_stream ai response: {"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保使用正确的库方法或上下文执行此SQL。"}
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.manages.connect_config_db[38979] INFO Result: <sqlalchemy.engine.cursor.CursorResult object at 0x7f225dc2f040>
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.rdbms.conn_clickhouse[38979] INFO SQL:SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.rdbms.conn_clickhouse[38979] INFO SQL:SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3, ttype:Token.Keyword.DML, sql_type:SELECT, table:None
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.rdbms.conn_clickhouse[38979] INFO Query[SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3]
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.manages.connect_config_db[38979] INFO Result: <sqlalchemy.engine.cursor.CursorResult object at 0x7f228df8e920>
2024-05-07 15:37:15 pnma-ck dbgpt.datasource.rdbms.base[38979] INFO Query[SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3]
2024-05-07 15:37:15 pnma-ck dbgpt.agent.expand.data_scientist_agent[38979] ERROR DataScientist check exception！'ClickhouseConnector'object has no attribute 'session'
Traceback (most recent call last):
File "/app/DB-GPT/dbgpt/agent/expand/data_scientist_agent.py", line 110, in correctness_check
columns, values = await resource_db_client.query(
File "/app/DB-GPT/dbgpt/serve/agent/resource_loader/datasource_load_client.py", line 62, in query
return conn.query_ex(sql)
File "/app/DB-GPT/dbgpt/datasource/rdbms/base.py", line 356, in query_ex
cursor = self.session.execute(text(query))
AttributeError: 'ClickhouseConnector' object has no attribute 'session'

Edgar (to PlanManager)-[chatgpt_proxyllm]:
"{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保使用正确的库方法或上下文执行此SQL。"}"

Edgar Review info:
Pass(None)
Edgar Action report:
execution succeeded,
{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保��用正确的库方法或上下文执行此SQL。"}

PlanManager (to User)-[]:
"查询最大的三个端口"

PlanManager Review info:
Pass(None)
PlanManager Action report:
execution failed,
{"display_type":"response_table","sql":"SELECT port FROM bgp_lax2_old_csv ORDER BY port DESC LIMIT 3","thought":"尽管有提到'ClickhouseConnector'对象缺少'session'属性的错误，这实际上是一个执行环境问题而非SQL语法问题。SQL本身是正确的，旨在查询'bgp_lax2_old_csv'表中'port'列的前三个最大值。请确保使用正确的库方法或上下文执行此SQL。"}

2024-05-07 15:37:15 pnma-ck dbgpt.serve.agent.agents.controller[38979] INFO save agent chat info！82c5c98e-0c44-11ef-8715-a0369fa9eae0

it looks the __init__ function in ClickhouseConnector didn't set the session attribute


implement parse SignaturePolicyEnvelope to human readable expression

fix #127

@SamYuan1990 fix commited，please make a review

@SamYuan1990  lint problem fixed，please make a review

There is still a lint failure. To save time, it would be worth running make lint locally before pushing a PR update.

tks for the advice,learned a lot.

@SamYuan1990 has tested with golang-lint, please make a review


not an issue per se, but is there any way to support you?

i use cactbot all the time and id love to support you through patreon or something just as a thanks

No patreon, sorry!  I really appreciate the kind words, though.  :heart:  Glad it's been useful for you.


Sequencce Definitions Accessor Propertyt Is Broken in LV 2015

The accessors for the property have slightly different names and LV 2015 cares about this even though 2014 didn't.

a54e2bc


Support in-app-update for installing required updates

Checklist

[x] Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
Tested Manually as automation tests require a lot of changes and supporting injection.

[x] If this PR comes from a fork, please Allow edits from maintainers

[x] Set a speaking title. Format: {task_name} (closes #{issue_number}). For example: Use logger (closes # 41)

[ ] Link your Pull Request to an issue (if applicable)

[ ] Create Work In Progress [WIP] pull requests only if you need clarification or an explicit review before you can continue your work item.

[x] Make sure that your PR is not introducing unnecessary reformatting (e.g., introduced by on-save hooks in your IDE)

[x] Make sure that your PR does not contain changes in text files, therefore the PR must not contain changes in values/strings/* and / or assets/* (see issue #332 for further information)

[x] Make sure that your PR does not contain compiled sources (already set by the default .gitignore) and / or binary files

Description
Integrate with  in-app updates to make app updates inside the App instead of navigating the user to Google Play store.
In case of failure for any reason, we fallback to the normal dialog that asks the user to update the App on Google Play.

While I think this could be a useful addition, it is impossible to merge as there doesn't seem to be a prior discussion where this was green-lit. The scope of the CWA and the impact that changes can have are too big to just accept PRs.
I think the contributors template was only changed to better highlight this after this PR was already created.
If I'm mistaken, and this was green-lit in a prior discussion, please let me know.


[Bug] Exception in converter encoding doesn't propagate as expected

What are you really trying to do?
I am writing a temporal workflow and I want to handle the exceptions that happen during the execution, I want the workflow to catch any unexpected error and fail the entire workflow in a non-retriable way.
Describe the bug
I am catching all the exceptions during workflow run and re-raise them as non-retriable ApplicationErrors and everything works as expected except for the case where the exception is thrown during the encoding of data to an activity.
Running a workflow that sends unserializable obj to an activity results in the following history (which doesn't contain any failure in it except for the timeout [2s]):

and the following warning in the logs:
WARN temporal_sdk_core::worker::workflow: Error while completing workflow activation error=status: InvalidArgument, message: "invalid TaskQueue on ScheduleActivityTaskCommand: missing task queue name. ActivityId=1 ActivityType=some_activity", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} }

I would expect the workflow to fail because of the exception (and log it in the history) before the timeout occurs, in the same way it happens if exception is thrown in the middle of the workflow execution.
Minimal Reproduction
import asyncio
import datetime
import uuid

import temporalio.activity
import temporalio.client
import temporalio.common
import temporalio.exceptions
import temporalio.worker
import temporalio.workflow

#### activities
class SomeObj:  # non serializable by default so senfding it to activities should fail
    def __init__(self, value: str):
        self.value = value

@temporalio.activity.defn
async def some_activity(obj: SomeObj) -> str:
    return obj.value

#### workflow
@temporalio.workflow.defn
class FailingWorkflowExceptionDuringActivitySend:
    @temporalio.workflow.run
    async def run(self) -> str:
        try:
            obj = SomeObj("value")
            value = await temporalio.workflow.execute_activity(
                some_activity,
                obj,
                start_to_close_timeout=datetime.timedelta(seconds=10),
            )
            return value
        except Exception as e:
            raise temporalio.exceptions.ApplicationError(
                "Failed during activity send as expected", non_retryable=True
            ) from e

####

async def _main():
    client = await temporalio.client.Client.connect(
        target_host="localhost:7233",
    )

    task_queue = "task-queue-name"
    workflow = FailingWorkflowExceptionDuringActivitySend

    async with temporalio.worker.Worker(
        client,
        task_queue=task_queue,
        workflows=[workflow],
        activities=[some_activity],
    ):
        try:
            handle = await client.start_workflow(
                workflow.run,
                id=f"{workflow.__name__}-{uuid.uuid4().hex}",
                task_queue=task_queue,
                run_timeout=datetime.timedelta(seconds=2),
                execution_timeout=datetime.timedelta(seconds=2),
                retry_policy=temporalio.common.RetryPolicy(
                    maximum_attempts=1,
                ),
            )

            result = await handle.result()
            print(f"Result: {result}")
        except Exception as e:
            print(f"Workflow: {workflow.__name__} failed:{e.__cause__}")

if __name__ == "__main__":
    asyncio.run(_main())

Environment/Versions

OS and processor: Linux
Temporal Version: sdk version: temporalio==1.6.0, temporal cli (temporal --version) temporal version 0.12.0 (server 1.23.0) (ui 2.26.2)
Are you using Docker or Kubernetes or building Temporal from source? No

I would expect the workflow to fail because of the exception (and log it in the history) before the timeout occurs, in the same way it happens if exception is thrown in the middle of the workflow execution.

Common exceptions do not fail the workflow by default, they "suspend" it. This is the same way as if an exception is thrown in the middle of workflow execution. See https://github.com/temporalio/sdk-python?tab=readme-ov-file#exceptions. You can customize this behavior and/or customize the converter to throw certain exceptions.

I would expect the workflow to fail because of the exception (and log it in the history) before the timeout occurs, in the same way it happens if exception is thrown in the middle of the workflow execution.

Common exceptions do not fail the workflow by default, they "suspend" it. This is the same way as if an exception is thrown in the middle of workflow execution. See https://github.com/temporalio/sdk-python?tab=readme-ov-file#exceptions. You can customize this behavior and/or customize the converter to throw certain exceptions.

I know, that is why I catch the exception and raise it again as non-retriable ApplicationError.

Right, so the exception raised by the data converter is similar to raising the exception in the workflow, but it occurs on a different path (it occurs in the payload converter not the workflow code). It's a bit advanced, but you can customize the converter the same way you customize the workflow to catch and re-raise in a certain way. Alternatively to both, you can just put failure_exception_types=[Exception] or similar in the @workflow.defn and it should fail the workflow with any exception instead of failing the task (i.e. suspending pending code fix).

Right, so the exception raised by the data converter is similar to raising the exception in the workflow, but it occurs on a different path (it occurs in the payload converter not the workflow code). It's a bit advanced, but you can customize the converter the same way you customize the workflow to catch and re-raise in a certain way. Alternatively to both, you can just put failure_exception_types=[Exception] or similar in the @workflow.defn and it should fail the workflow with any exception instead of failing the task (i.e. suspending pending code fix).

Thanks!
But I am able to catch the exception in the try except of the workflow so I think everything is as I expected.
I think the problem is related to the warning log I attached to the issue:
WARN temporal_sdk_core::worker::workflow: Error while completing workflow activation error=status: InvalidArgument, message: "invalid TaskQueue on ScheduleActivityTaskCommand: missing task queue name. ActivityId=1 ActivityType=some_activity", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} }

I was able to (maybe) fix that locally if in (temporalio.worker._workflow_instance._ActivityHandle._apply_schedule_command)
https://github.com/temporalio/sdk-python/blob/365ceadb0732f36b66bf1ae2970131fc7944b565/temporalio/worker/_workflow_instance.py#L2145-L2215
I moved the part that handles the payloads:
https://github.com/temporalio/sdk-python/blob/365ceadb0732f36b66bf1ae2970131fc7944b565/temporalio/worker/_workflow_instance.py#L2167-L2170
to the end of the function (_apply_schedule_command)
I am not too familiar with the code, but it seems like in that context the queue name is only initialized after the payload conversion, so if I had an exception during the payload conversion it could not be sent to the temporal server.

I was able to (maybe) fix that locally if in (temporalio.worker._workflow_instance._ActivityHandle._apply_schedule_command) to the end of the function (_apply_schedule_command)

I see the issue. The issue is that we create the command before we try to serialize the contents. So if this does not fail the workflow task, the command is sent off incomplete (because it threw an exception during building). I have opened https://github.com/temporalio/sdk-python/issues/564 to track (it is easier as a separate issue to state the problem clearly), but will keep this issue open.


Inline modules

From this discord conversation.
I have a bunch of related tasks:
    db-generate-migration
    db-migrate
    db-nuke
    db-reset
    db-seed
    db-up
    db-destroy

What I'd like to do is have a more clear separator (like rake):
    db:generate-migration
    db:migrate

This is possible with modules, but I cannot create multiple modules in a single Justfile (creating separate Justfiles would be messy in my case).
Two ideas I can think of to solve this problem:

Would it be possible to support :, . in the name grammar?
Could we define multiple modules within a single Justfile?

Thanks for opening this issue!
I would like to eventually support inline modules, which would allow submodules to be defined in a single justfile.
We could allow :: in recipe names, but modules do more than just provide namespaces for recipes, since they can have their own settings, variables, and submodules, so I'm more keen to just do full blown inline modules, which would support all of that.
The syntax I've had in mind is:
bar::
  foo:
    echo 'recipe foo inside module bar'

But it could also be:
mod bar::
  foo:
    echo 'recipe foo inside module bar'

just uses indentation to delimit recipe bodies, since recipe bodies are arbitrary text, and just is descended from make, which does the same. So it would be natural to also use indentation to delimit modules bodies, as in the examples above.
However, I do personally find indentation to be annoying and fiddly, so we could consider using {…} for modules, which would allow for indenting the contents:
bar {
  foo:
    echo 'recipe foo inside module bar'
}

Or not, depending on preference:
bar {
foo:
  echo 'recipe foo inside module bar'
}

I really like the :: indentation structure:
bar::
  foo:
    echo 'recipe foo inside module bar'

Feels really natural and easy to read.
Nested indentation isn't awesome, but { syntax wouldn't seem to align with the rest of the Justfile grammar IMHO. However, this is a departure from Makefile syntax in a bigger way so maybe it makes sense to do something very different?
Either way sounds great to me! I love this idea.

Yah, honestly I like it to, especially in terms of being easy to read. My preference for avoiding indentation-based syntax is mostly when it comes to writing and editing. For example, copy-pasting a recipe from an inline module, where it's indented, to a dedicated module file, where it won't be indented, would require fixing the indentation manually.
But it's probably important that the syntax all feels consistent, and not randomly have {…} delimited constructs when everything else is based on indentation.

But it's probably important that the syntax all feels consistent

I'd agree here!
Looking forward to kicking the tires on this whenever it's implemented.
Thanks for the excellent tool.

Seconding. I was going to file this same issue.
Is the intent for these to be just logical groupings or more like clap sub commands?
I would lean more towards the subcmd option, though that is certainly more complicated:

Can modules be nested recursively? If you have the machinery to do it at a single level, I would expect that to be most of the complexity.
Can each module define it's own "default:" like a top level justfile
Can a parent module have variables that are available in a child scope

msrv := "1.65.0"

ci opts=""::
  default: core
  core:
    cargo test {{opts}}
  msrv:
    cargo +{{msrv}} test {{opts}}
  features:
    cargo test --all-features {{opts}}
  all: features msrv

> just ci
> just ci --verbose
> just ci::features --verbose
> just ci::all --no-run

After tinkering around with an ever-growing Justfile, I'm going to put in my vote for:

Allowing a single : in the recipe name grammar. This would still be helpful/cleaner when not using modules.
Keeping :: for modules

Allowing : in the recipe name grammar would lead to some issues.
For example, should this:
foo:bar:baz

Be interpreted as recipe foo with dependency bar:baz, or recipe foo:barwith dependencybaz`. You could work around this by whitespace to disambiguate it, but that feels kinda bad.

Ah I see, that makes sense. Any workarounds would make the syntax ugly.
Another idea here is allowing dot-syntax. Mix does this and it works pretty well.
command.with.dots: other.command

Yah, similar issues with . syntax. . is used so common for property access, and we might want that later, i.e. object.method().

Yup yup makes sense especially if you aim to expand the language and syntax over time. Love how thoughtful you've been with the tool!


Update dependencies

[ ] Update all dependencies

Closed with https://github.com/AmplicaLabs/reconnection-service/pull/134


Limits ?

I have a script which add users
I have 35 users and only 27 are added
If I rerun of course some of them already exists ...
But the the are never added ...
Any clues welcome, here is the script
Try {
Import-Csv -Path "EYTC.csv" | foreach {
Add-TeamUser -GroupId 3337d494-dde7-4496-ba18-1bc392203ef7 -user $.Name
Write-Output $.Name
}
}

Catch {}

Finally {
Get-TeamUSer -GroupId 3337d494-dde7-4496-ba18-1bc392203ef7
}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: da0c53f2-1c62-14f6-6371-f54bf86b0c24
Version Independent ID: aac162db-a9c0-fbfb-dd40-526e94d3bef8
Content: Add-TeamUser (teams)
Content Source: teams/teams-ps/teams/Add-TeamUser.md
Service: teams-powershell
GitHub Login: @kenwith
Microsoft Alias: kenwith

@officedocsbot assign @yogkumgit

Hello @Chewbee how are you? 👍
Here, we can handle issues that impact changes in documentation.
If the cmdlet, ran outside your script, works as the documentation says, I can not identify any change needed to the documentation.
Did you try running the Add-TeamUser individually for those users that you see aren't being added?

@get-itips Thank you very much for the contribution and sharing this explanation. @Chewbee
Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


UI update suggestions

Hey, your app idea is cool. I have ran it and it works great. But i think few changes will be good. I think it will be better to disable the exit application button when user starts a session and enable it again during break that way user will not slack off after starting a session. Also i think you should turn off the maximise button , currently when it is maximised it awkwardly gets stuck at the corner of the screen.

Oh, thanks for the suggestion!
They sound really good but the thing is that I don't know how I can do that.

Is it okay if i add those changes and create a pull request ? I will describe and comment out the changes as well for better understanding.

That would be great.
Thanks

@AndrewAung11  hey i have created the pull request for this and described it on the pull request . Please check it.

Thanks for the help!


NERDTree command :NERDTreeFind errors out.

NERDTree command for NERDTreeFind errors out with the following error.
"E492: Not an editor command: NERDTreeFind"
Info:
This is on macVim.
The file manager is working alright.
Posted on stackoverflow as well.
http://stackoverflow.com/questions/27727723/nerdtreefind-errors-out-with-e492-not-an-editor-command-nerdtreefind

Noticed the below two files from an older NERDTree install.
~/.vim/doc/NERD_tree.txt
~/.vim/plugin/NERD_tree.vim
Removed the above files and reinstalled the plugin.

Noticed the below two files from an older NERDTree install.
~/.vim/doc/NERD_tree.txt
~/.vim/plugin/NERD_tree.vim
Removed the above files and reinstalled the plugin.


[Doc Issue] [Prerequisites list on step 1 of tutorial lists required installations out of order]

Docs Build status updates of commit 6f180c5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/sbs-gs-javascript.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/sbs-gs-javascript.yml
91
40
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.


Create Flasher UI for supplying the session key

Currently the magical session key is supplied to the Flasher via command line.
Work out and create a simple UI that asks the session key in a popup window upon starting the Flasher daemon.

Fixed with 311d436cade51cb4617ff53c6c20cade35afb770.


Allow register actions to "add to catalog" button

Fixes #2488

Of course this broke integration tests... will fix asap.

Integration tests should now pass 🤞

Integration tests should now pass 🤞

They did not 😿

Integration tests should now pass 🤞

They did not 😿

Integration tests should now pass 🤞 🤞


Add missing license header on one file

variables-parameters-constants.lsp was missing a license header
and (at the time of this commit) had only ever been touched by
the one first commit in 2013, so I added the samed header as in
the other files, with the same copyright year (2013).

Thanks!


Unable to download JSON metadata

D:\p>youtube-dl.exe --verbose --cookies D:\p\Plural\cookies.txt -o "D:\p\Plural%(playlist_index)s - %(title)s.%(ext)s" --min-sleep-interval 31 --max-sleep-interval 32 https://app.pluralsight.com/library/courses/aws-security-fundamentals/table-of-contents --playlist-start 1
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies', 'D:\p\Plural\cookies.txt', '-o', 'D:\p\Plural\%(playlist_index)s - %(title)s.%(ext)s', '--min-sleep-interval', '31', '--max-sleep-interval', '32', 'https://app.pluralsight.com/library/courses/aws-security-fundamentals/table-of-contents', '--playlist-start', '1']
[debug] Encodings: locale cp1256, fs mbcs, out cp720, pref cp1256
[debug] youtube-dl version 2018.03.20
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] aws-security-fundamentals: Downloading course JSON
[download] Downloading playlist: AWS Security Fundamentals
[pluralsight:course] playlist AWS Security Fundamentals: Collected 32 video ids (downloading 32 of them)
[download] Downloading video 1 of 32
[pluralsight] aws-security-fundamentals-m0-0: Downloading course JSON
[pluralsight] aws-security-fundamentals-m0-0: Downloading mp4-high-widescreen viewclip JSON
[pluralsight] aws-security-fundamentals-m0-0: Waiting for 5 seconds to avoid throttling
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'http://vid20.pluralsight.com/clips/resolution/4650ae6d-5526-4a3e-b3b1-c1a759aa2f48/current/mp4/1280x720.mp4?vd3RpDl_hubVbW9_oZdtjTU4-WuMEf3wIHur8XK_y1cpNdvMlZmb4W0nC3pUZJFLXSDjUVLM2rVsSjUPRS0OdB-qgkY0m1WNKAiu0EG5Cp4TpZf36iU9f5QcmS4I0GypYL7TGv0pi9L53vrjG5bR76LMKyy55FDvmAo5qxkaRecfwrCZOWMFqYWYpdJJ'
[download] D:\p\Plural\01 - Course Overview.mp4 has already been downloaded
[download] 100% of 3.26MiB
[download] Downloading video 2 of 32
[pluralsight] aws-security-fundamentals-m1-0: Downloading course JSON
[pluralsight] aws-security-fundamentals-m1-0: Downloading mp4-high-widescreen viewclip JSON
WARNING: Unable to download JSON metadata: HTTP Error 403: Forbidden
[pluralsight] aws-security-fundamentals-m1-0: Waiting for 3 seconds to avoid throttling
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\YoutubeDL.py", line 785, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py", line 440, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\pluralsight.py", line 326, in _real_extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py", line 1083, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
I've tried to reinstall it, export cookies from the beginning, clear cache and cookies .. but still same issue

Just remove the table-content part and you'll be fine.
e.g. https://app.pluralsight.com/library/courses/aws-security-fundamentals

Same issue !
D:\p>youtube-dl.exe --verbose --cookies D:\p\Plural\cookies.txt -o "D:\p\Plural%(playlist_index)s - %(title)s.%(ext)s" --min-sleep-interval 31 --max-sleep-interval 32 https://app.pluralsight.com/library/courses/aws-developer-introduction-aws-lambda/ --playlist-start 1
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies', 'D:\p\Plural\cookies.txt', '-o', 'D:\p\Plural\%(playlist_index)s - %(title)s.%(ext)s', '--min-sleep-interval', '31', '--max-sleep-interval', '32', 'https://app.pluralsight.com/library/courses/aws-developer-introduction-aws-lambda/', '--playlist-start', '1']
[debug] Encodings: locale cp1256, fs mbcs, out cp720, pref cp1256
[debug] youtube-dl version 2018.03.20
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] aws-developer-introduction-aws-lambda: Downloading course JSON
[download] Downloading playlist: AWS Developer - An Introduction to AWS Lambda
[pluralsight:course] playlist AWS Developer - An Introduction to AWS Lambda: Collected 25 video ids (downloading 25 of them)
[download] Downloading video 1 of 25
[pluralsight] aws-developer-introduction-aws-lambda-m0-0: Downloading course JSON
[pluralsight] aws-developer-introduction-aws-lambda-m0-0: Downloading mp4-high-widescreen viewclip JSON
WARNING: Unable to download JSON metadata: HTTP Error 403: Forbidden
[pluralsight] aws-developer-introduction-aws-lambda-m0-0: Waiting for 2 seconds to avoid throttling
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\YoutubeDL.py", line 785, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py", line 440, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\pluralsight.py", line 326, in _real_extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py", line 1083, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

This course is not free. Can you watch it in browser in the first place?

of course, I tried to download courses that I already downloaded for
testing but still same issue
and I've premium subscription
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:38 AM, Sergey M<EMAIL_ADDRESS>wrote:

This course is not free. Can you watch it in browser in the first place?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375860937,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o66N2-G7sZQjeX6OMHYZ3y-lkoL1ks5thhQxgaJpZM4SzawM
.

Does it work if you explicitly provide account credentials?

Sure, no problem just send me your email
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:44 AM, Sergey M<EMAIL_ADDRESS>wrote:

Does it work if you explicitly provide account credentials?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375861591,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o8A16hikwxJKV7QtKqxt6oDOu4w3ks5thhWBgaJpZM4SzawM
.

I mean if you provide it to youtube-dl instead of cookies.

I'm not sure If I fully understand but can you send me the code for testing
?
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:49 AM, Sergey M<EMAIL_ADDRESS>wrote:

I mean if you provide it to youtube-dl instead of cookies.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375862243,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o_Q79XzX_hT26kQ33NoxWNvyBIXWks5thhbFgaJpZM4SzawM
.

https://github.com/rg3/youtube-dl#authentication-options

so the code should be like that
youtube-dl.exe -u username -p password --verbose --cookies
D:\p\Plural\cookies.txt -o "D:\p\Plural%(playlist_index)s -
%(title)s.%(ext)s" --min-sleep-interval 31 --max-sleep-interval 32
https://app.pluralsight.com/library/courses/aws-developer-introduction-aws-lambda/
--playlist-start 1
and of course I'll replace username & pass
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:44 AM, Sergey M<EMAIL_ADDRESS>wrote:

Does it work if you explicitly provide account credentials?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375861591,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o8A16hikwxJKV7QtKqxt6oDOu4w3ks5thhWBgaJpZM4SzawM
.

It's working, so the problem with the cookies from the beginning ?, so will
u fix the cookies issue soon ?
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:54 AM, Express Plus Company <
<EMAIL_ADDRESS>wrote:

so the code should be like that
youtube-dl.exe -u username -p password --verbose --cookies
D:\p\Plural\cookies.txt -o "D:\p\Plural%(playlist_index)s -
%(title)s.%(ext)s" --min-sleep-interval 31 --max-sleep-interval 32
https://app.pluralsight.com/library/courses/aws-developer-
introduction-aws-lambda/ --playlist-start 1
and of course I'll replace username & pass
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:44 AM, Sergey M<EMAIL_ADDRESS>wrote:

Does it work if you explicitly provide account credentials?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375861591,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o8A16hikwxJKV7QtKqxt6oDOu4w3ks5thhWBgaJpZM4SzawM
.

Most likely your cookies are expired.

stopped again :(
[download] Destination: D:\p\Plural\04 - Module Overview.mp4
[download] 100% of 8.84MiB in 00:19
[download] Downloading video 5 of 32
[pluralsight] aws-security-fundamentals-m1-3: Downloading course JSON
[pluralsight] aws-security-fundamentals-m1-3: Downloading JSON metadata
ERROR: Unable to download JSON metadata: <urlopen error [Errno 11001]
getaddrinfo failed> (caused by URLError(gaierror(11001, 'getaddrinfo
failed'),))
File "C:\Users\dst\AppData\Roaming\Build
archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\extractor\common.py",
line 519, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build
archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\YoutubeDL.py", line
2199, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
File "C:\Python\Python34\lib\urllib\request.py", line 482, in _open
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Users\dst\AppData\Roaming\Build
archive\youtube-dl\rg3\tmpbt_jbnlh\build\youtube_dl\utils.py", line 1101,
in https_open
File "C:\Python\Python34\lib\urllib\request.py", line 1185, in do_open
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:57 AM, Express Plus Company <
<EMAIL_ADDRESS>wrote:

It's working, so the problem with the cookies from the beginning ?, so
will u fix the cookies issue soon ?
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:54 AM, Express Plus Company <
<EMAIL_ADDRESS>wrote:

so the code should be like that
youtube-dl.exe -u username -p password --verbose --cookies
D:\p\Plural\cookies.txt -o "D:\p\Plural%(playlist_index)s -
%(title)s.%(ext)s" --min-sleep-interval 31 --max-sleep-interval 32
https://app.pluralsight.com/library/courses/aws-developer-in
troduction-aws-lambda/ --playlist-start 1
and of course I'll replace username & pass
--------
With best wishes,
Website: explus4host.com http://WwW.Explus4Host.CoM
Mobile:<PHONE_NUMBER>2347
FB: https://www.fb.com/ExpCompany https://www.facebook.com/ExpCompany
Twitter: https://twitter.com/ExpCompany https://twitter.com/ExpCompany
On Sat, Mar 24, 2018 at 11:44 AM, Sergey M<EMAIL_ADDRESS>wrote:

Does it work if you explicitly provide account credentials?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rg3/youtube-dl/issues/15943#issuecomment-375861591,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALM_o8A16hikwxJKV7QtKqxt6oDOu4w3ks5thhWBgaJpZM4SzawM
.

The last one is due to DNS issues on your side.


Explore reimplementing ornaments in Compose / Material3 in a separate module, using camera state

see discussion in: https://github.com/sargunv/maplibre-compose/issues/13
ideally in a separate module so users can opt-in to it (built in ornaments will still be supported)
compass should:

use material3 style
have all the same behavior as the native compass
also reset pitch on tap by default (like google maps compass), configurable

eventually scale bar too, requirements tbd

For my app, I already have a widget for the compass, it's here:
https://github.com/streetcomplete/StreetComplete/blob/master/app/src/main/java/de/westnordost/streetcomplete/screens/main/controls/CompassButton.kt
https://github.com/streetcomplete/StreetComplete/blob/master/app/src/main/java/de/westnordost/streetcomplete/screens/main/controls/MapButton.kt
As you see, it's not exactly a lot of code, hehe, but anyways, I hereby release the two linked files under the terms of the BSD-3-Clause license, maybe it can be helpful ¯\_(ツ)_/¯

scale bar tbd but #153 creates the new module; it'll be included in the next release


Capable of deleting old games

Did it for Dixit but not the others. Will likely need to overhaul the others at some point?


4.3 csharp files are changed on editor save causing issues in IDEs - regression?

Tested versions
Reproducible in: 4.3 dev4 and 4.3[bfdac951e3efe87c14a18ca33b16bb901687a43f]
Not reproducible: 4.2.1
System information
Windows 11 - Vulkan - Nvidia RTX 4070 - intel i5 13600KF
Issue description
Video showcasing the issue in 4.3 and comparison with 4.2.1: https://youtu.be/LFMKV9npR8U
In 4.3 for some reason saving editor when any scene is open also modifies c# files which were last modified for some reason, this causes discontinuity in how IDEs see the file since the IDEs now see that the file was modified, one such issue is in Rider when a popup dialog appears asking you if you want to reload from disk when you try to undo every time. This is increadibly annoying.
From the file itself codewise nothing changes, so maybe godot started changing some meta data or some kind of format?
I've been checking modified status in windows file explorer (also the issue with undo happening in my IDE constantly which doesn't seem to happen in 4.2.1).
Steps to reproduce
(Follow the video above for clarity or steps below)

Create a C# godot project
Build the project.
Add any node and attach a C# script (or open the attached script through the editor in MRP).
Make any changes to the file, for example add couple empty lines. (this step is necessary the IDE first needs totake the current state of the file into account).
Open a file explorer and check the script file's modified status, write down/remember the date + time.
Wait for a minute or more (depends on what is the minimum change your system can display, in this test I recommend a minute to make it simpler to notice the change clearly).
Switch to Godot editor and pres CTRL + S (save scene/all scenes) while the scene is opened (if no scene is opened then nothing happens, if you change script A but have a scene B with a script B open then the issue still happens in the script A even though you are saving scene B).
Go to the file explorer, refresh the window if you haven't (typically F5 shortcut), and observe the changed modified status.
Open the script file in your IDE and observe that all the code is still the same.
(10. This can cause issues in IDE syncing with files/undo history and I don't know what else, in my case it's undo issues in Jetbrains Rider)

Minimal reproduction project (MRP)
Test43master.zip

Hello. Can you share the following setting values for your editor?

text_editor/external/use_external_editor
text_editor/external/exec_path
text_editor/external/exec_flags
dotnet/editor/external_editor
dotnet/editor/custom_exec_path
dotnet/editor/custom_exec_path_args
dotnet/editor/editor_path_optional

I personally cannot reproduce on my end (Windows 10, Godot 4.3.dev [9b94c80e9aff2a4f363ae6d8e2bbe837aa5876bc], Rider 2023.3.3).

Hi, yeah definitely here it is in the same order as yours list:

I tried none as well as rider
I tried none as well as rider
{file} <- this is default
Jetbrains RIder and Fleet
nothing
{file}
E:\Program Files\JetBrains\JetBrains Rider 2023.2.2\bin\rider64.exe

I honestly haven't touched these settings since forever so I assume it has always been like this since I set it up.
I recorded the whole thing on a video to showcase what happens, how and comparison to 4.2.1 where it does not happen here: https://youtu.be/LFMKV9npR8U

Thank you for the additional details 👍

No thanks to you, I pulled the changes https://github.com/godotengine/godot/pull/89261 and it seems to be ok from a quick check.

Removing topic:dotnet, as it is not specific to C# scripts.

I having this same issue in Linux, 4.3dev-4
https://github.com/godotengine/godot/issues/89029#issuecomment-1974147333


Installation failing using npm

I am trying to install typedoc and i am getting the following error. Please suggest me how to fix it.
npm ERR! path \node_modules\highlight.js\tools\build.js
npm ERR! code ENOENT
npm ERR! errno -4058
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '\node_modules\highlight.js\tools\build.js'

For us it's a blocking issue ...

Same here on 0.14.2
npm ERR! path \node_modules\typedoc\node_modules\highlight.js\tools\build.js
npm ERR! code ENOENT
npm ERR! errno -4058
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '\node_modules\typedoc\node_modules\highlight.js\tools\build.js'
npm ERR! enoent This is related to npm not being able to find a file.

same here:
node 10.15.1
npm 6.8.0

Same here on version 0.14.2.
It looks the highlight.js 1.15.1 breaks the backward compatibility. My workaround is to install the highlight.js to specific version "9.13.1", which is same as the one used by typedoc

Same!
I installed<EMAIL_ADDRESS>manually just now, and re-install again. It works for me, you can try it.

https://github.com/highlightjs/highlight.js/issues/1984

I have the same issue, but I can't good solution for resolving

Same issue

SAME

It seems like this is mainly an issue if you have another dependency which depends on<EMAIL_ADDRESS>One workaround would be to use some of the new features in<EMAIL_ADDRESS>and use the time machine.
I plan on updating all dependencies when I push the next release.

It seems like this is mainly an issue if you have another dependency which depends on<EMAIL_ADDRESS>One workaround would be to use some of the new features in<EMAIL_ADDRESS>and use the time machine.
I plan on updating all dependencies when I push the next release.

I get the same error when I try npm install typedoc in a brand new directory, with no other dependencies.
I'm using npm version 6.8.0. Here is the actual result:
npm install typedoc
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated<EMAIL_ADDRESS>Use uuid module instead
npm WARN deprecated<EMAIL_ADDRESS>The major version is no longer supported. Please update to 4.x or newer
npm WARN enoent ENOENT: no such file or directory, open '/.../package.json'
npm WARN temp No description
npm WARN temp No repository field.
npm WARN temp No README data
npm WARN temp No license field.

npm ERR! path /.../node_modules/highlight.js/tools/build.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/.../node_modules/highlight.js/tools/build.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /.../_logs/2019-02-25T15_19_56_137Z-debug.log

Also, it appears that the new features that @aciccarello is referring to are in the next release of npm (which is currently in pre-release).
https://npm.community/t/release-npm-6-9-0-next-0/5553?u=zkat
For the short term, publishing a new version of typedoc with updated dependencies will help us a lot!

#977 should temporarily work around this issue until highlightjs/highlight.js#1984 is fixed. Could project maintainers take a look and make additional commits as necessary?

Highlight.js v9.15.5 has been released so this issue should be resolved.


WIP: Minor fixes and update to new ink! version

I am currently working through the contracts workshop as part of my onboarding. I am just fixing some minor oversights that I notice while doing that.
The update to the new ink! version is not yet included and will compromise the two changes discussed in riot which are necessary to fix example code against the current ink! version:

changing ink_lang2 to ink_lang
moving the using statement of in_core::storage into the contract module

Thank you for the fixes. However, I guess there are many more things that need to be fixed for our ink! workshop.

One step at a time I guess. Right now, I only intend on fixing things that somehow blocked or confused me while doing the workshop.

We can merge as soon as you press "Ready for review". xD

I guess I will put the coming changes into another PRs then. I just removed the commit regarding the overstate vs understate and squashed the fixup. Yea just merge this micro pr when you are ready.

Thank you for your help in keeping this updated!


Make project wiki or Google doc

Purpose
We may want to consider having a project wiki or Google doc to reference any important discussion points, etc. that might get lost on Discord.  See this discussion for reference.
Additional Information
If we decide to go with a wiki, this issue will need to wait until the GitHub organization is in place and this repo is transferred over.
Criteria

[ ] Come up with a way to document non-trivial group discussions for those who can't keep up with Discord on a daily basis.
[ ] Make it both creative and organized

Are we in an org now?

Yup, so we should be good now

Alright I got big boss to enable wiki so I'll get working on this

I'm thinking on moving some of the more tutorial-is parts of our CONTRIBUTING.md file to the wiki. That will slim down the file. Everything under "Typical Git workflow" will go to the wiki. We keep Initial Setup and reporting issues

That's fine, but I think they should still be linked to from CONTRIBUTING.md.
So in CONTRIBUTING, something like
Typical Git Workflow
We use a typical Git branching workflow.  See more at our wiki (link to wiki)

And then maybe in the wiki we can also throw in this link for further reading
https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows


Bug when minimal number of atoms has a mass that is over the molecule mass

If you give a range of
C0-10As1-10
For a target mass of 24
The system will propose C2 !!! Which is false !!!!

Fixed


Don't select notebooks.content to improve response time

The HTML tables of notebooks don't need content so we could not select it.
PageController could pass an argument select with the columns required — without content.
Then, Notebooks.list_notebooks/1 should consider the option select.

Duplicated https://github.com/notesclub/notesclub/issues/72


HTTP Cache invalidates complete config's instead of 1 key

Hello,
I am having problems with the HTTP Cache configuration.
Every time a change is detected that is applicable for one of the cache configs, then that complete config is invalidated in the memory cache.
Our structure is a bit like this:
/content/site/language/year/month/day/page.html
Case:
I have 2 pages in my HTTP cache.

/content/site/language.html
/content/site/language/year/month/day/page.html

when i edit a component on 1 of those 2 pages, they both get invalidated when they are in the same cache configuration.
This is a sample config that looks like the one we use.

We could use multiple configs so they are invalidates separately, but this would result in a lot of configurations since our content tree is date based.

@dimio111 could you throw logging at DEBUG got com.adobe.acs.commons.httpcache and run it again?
I think i see the problem - it appears the Group extension builds off the actual RequestURI (complete w selectors, extensions) where-as the ResourceType extension builds off the resource path.
Need to stew on in for a bit and see if there is a simple, safe way to adjust so INVALIDATIONS can match configs on resource path, but match cache REQUESTS on full URIs.
/cc @Sivaramvt

I'm experiencing the same problem!

I was looking at this some more, and i think this going to have to follow a convention of invalidating at the hierarchy node level ... for example

If i request /content/some/page.html ... it will be cached
If i modify /content/some/page/jcr:content/root/my-component, trying to match on resource path's wont work:

/content/some/page/jcr:content/root/my-component != content/some/page

I think we'll have to invalidate changed resources via the node above the jcr:content node ..  for mounted resources, this will have sweeping effects.
thoughts?

Fixed in latest version; it will now invalidate based on the closest hierarchyNode, so changes to
/content/page/jcr:content/foo's invalidation key is /content/page


Producing events (such as playing audio) at constant intervals is jittery.

Godot version:
3.1.1
OS/device including version:
Arch Linux, Surface Pro 6
Issue description:
Producing events (such as playing audio) at constant/equal intervals is jittery.
For instance, it is impossible to make a "decent" metronome in Godot with just a few lines of code as one would expect.
Steps to reproduce:
Just tell the engine to reproduce a short audio file at high frequency (eg, every 5 delta).
Instead of hearing a constant pulse of 16*5 ms, you will hear jittery things.
Minimal reproduction project:
https://github.com/aleksfadini/godot-flawed-metronome
The code is just these 5 lines:
var counter = 0
func _physics_process(delta):
	counter += delta
	while counter >= 0.1:
		counter = 0
		$AudioStreamPlayer.play()

To start with, your code introduces jitter itself. A slightly better version would be this:
var counter = 0
func _physics_process(delta):
	counter += delta
	while counter >= 0.1:
		counter -= 0.1
		$AudioStreamPlayer.play()

With your existing code, every time your counter goes past 0.1, it loses synchronization by ignoring the remaining time.
In addition, timing doesn't work quite the way you may think. At e.g. physics 60 ticks per second, it will attempt to call physics on average 60 times per second. However those ticks aren't guaranteed to occur at 1/60 second intervals. If a particular frame has a long duration, the physics will perform a lot of ticks to catch up, one immediately after the other.

Why are you setting Engine.iterations_per_second=600?

I'm looking at your example. I changed your code to:
extends Node2D
var counter = 0

func _physics_process(delta):
	counter += delta

	if counter > 0.1:
		$AudioStreamPlayer.play(0)
		counter = 0

I also can't reproduce the jitter from your previous code.
@lawnjelly I don't think those are issues in this case, cause the counter is increased by a delta. It's happening every 0.1s

In this case another problem is the 600 ticks per second as girng says. (I didn't download the project initially, just looked at your posted code).
The max number of physics ticks / frame is capped at 8, to prevent runaway physics. So you are probably getting random fluctuations due to this as well (you are losing time). If you change the tick rate to 60, you'll notice you'll get a high frequency of sounds.

Eh, I take my previous statement back. You can actually tell the difference between each sound and the delay. It's so subtle and small though. I am wrong, this is reproducible.

You will also get jitter due to the audio buffers. In general this type of approach with high frequency sounds is likely to give less than perfect results.

Also could be something with .wav, going to try another format

Although it is reproducible, I'm not sure any of this constitutes a 'bug', it's more a feature (that you will find in most game engines). Playing sounds in such a manner is confounding a cascade of different sources of jitter on top of each other. It would be more remarkable if the sounds didn't have significant jitter (something would have been really wrong)! :smile:
Jitter is something that needs to be actively dealt with right down to the innards of the rendering, you have to expect it will happen, and build strategies to deal with it.
In general I think if any of these is to be investigated further they need to be isolated to provide an example of something that could be considered a bug. In this issue there are several things to consider including:

Bug in original gdscript
Misunderstanding of non-realtime nature of physics ticks
Too high tick rate giving tick capping behaviour
Audio buffer delays (probably non constant)
Possible delays in playing compressed sounds

If the objective is to have regular game events, then it is better to investigate with non-audio methods. If you want to investigate audio, then I would suggest isolating that.
If the objective is to have closely timed audio (say a music app), then other approaches are far more appropriate, such as editing the raw audio.

I've been thinking about that a few times, but would there be a way to "schedule" a sound to play at a specific time in the future (handled in the audio server since that's where samples get fed to the driver at constant 44100Hz) so that it actually starts playing exactly when it should?

I've been thinking about that a few times, but would there be a way to "schedule" a sound to play at a specific time in the future (handled in the audio server since that's where samples get fed to the driver at 44100Hz, which is much more precise than physics or render frames) so that it actually starts playing exactly when it should? This way jitter in the scheduling code would almost never matter.

Probably. But then again, the use-case for a working metronome to play smoothly at 0.1s.. is something reduz will most likely have a hard time accepting IMO

I've been thinking about that a few times, but would there be a way to "schedule" a sound to play at a specific time in the future (handled in the audio server since that's where samples get fed to the driver at 44100Hz, which is much more precise than physics or render frames) so that it actually starts playing exactly when it should? This way jitter in the scheduling code would almost never matter.

Going a little off topic but I believe so, but it would depend on support for this in the Godot audio code. You could take account of latency from the audio buffers (which might be constant or variable with a tiled approach).
Realtime music apps have to deal with similar kinds of problem, trading off the latency with the danger of starving the audio player with realtime input from e.g. a midi keyboard, or scheduling notes in advance from sequencer. But typically in games for sound effects you can put up with something being 'good enough', whereas in music, or synchronizing a movie with audio more care is usually taken.
With such a system you could schedule sound effects like a sword clang to happen at the exact moment within an animation. But whether most users would notice I don't know.
It reminds me of an old trick back in the 80s, music samplers (think drums) were unable to play lots of notes at once without a delay between each (so they came out like fast machine gun fire). One of the solutions was to add varying amounts of silence to the start of each sample, and have them play in advance, so they all hit the start of the actual sound simultaneously. :smiley:

Note that AudioStreamPlayer::play has an optional parameter, from_position. While you cannot give that parameter a negative value, maybe you could try giving it the leftover time, so that it plays from the right position?
Also, moving the code to _process should help with getting the code to execute closer to "realtime".
Finally, using AudioStreamGenerator to drive the audio should be better, though I am not sure if it would be possible to read the stream which has to be repeated.

Note that AudioStreamPlayer::play has an optional parameter, from_position. While you cannot give that parameter a negative value, maybe you could try giving it the leftover time, so that it plays from the right position?

There, you go, with that and an exact timer you could have a go at scheduling something reasonably accurately into the future with some silence tacked on the front. You'd still have to deal with audio buffer issues, but you could get around the vagaries of when the process or physics_process got called.

@bojidar-bg I thought that parameter was to start the sound from a different offset rather than just delaying it? (i.e playing a clip from its 42th sample rather than starting from 0th).

@Zylann Reread. I said the same?

@bojidar-bg sorry it's quite ambiguous... so let's say I have a 3 second sound with 3 clicks in it. I thought that if I set from_position to 1 second, my sound would only play 2 clicks. Is that correct?

Also could be something with .wav? Going to try .ogg
edit: Here is the .ogg. The delay between sounds is much less and seems more consistent and less "jittery". But I mean, I don't think it's possible to have perfect timing when playing sound at such a high interval. However, objectively, the OP's issue is technically reproducible lol

It also gets progressively worse if the cpu is busy or frames go even slightly below 60.

Note that AudioStreamPlayer::play has an optional parameter, from_position. While you cannot give that parameter a negative value, maybe you could try giving it the leftover time, so that it plays from the right position?

There, you go, with that and an exact timer you could have a go at scheduling something reasonably accurately into the future with some silence tacked on the front. You'd still have to deal with audio buffer issues, but you could get around the vagaries of when the process or physics_process got called.

This should be marked as a work around, if you can add a buffer of silence to each audio file (let’s say a constant second) and then before playing them, check what is the random offset of the processing frame, let’s call it delay (in secs), and use $Sound.play(1-delay).
This exactly what you suggested above.
However is a bit clunky and can get complicated when different pulses are supposed to verso and meet at a certain time, because in that case even a small offset will be noticeable and perceived as a “non simultaneous “ event.
So I guess it would be nice if the engine would take care of it, as it is a potential problem for games that include rhythm, depending how the game mechanics work.
Also, it is not the expected behavior of the engine.
Another way to put it is: if you set up a timer mode for 0.25 secs and it plays a sound at each loop, that sounds very non constant, whenever FPS wobble even a bit.
So a component of this is definitely more of a big than a suggested feature in my opinion.

Why are you setting Engine.iterations_per_second=600?

Yeah, sorry about that, it should not be in the minimum viable project to show the bug. I tried different things to see if it improved the jitter, and forgot to delete that line. As you can see, increasing the cycles doesnt help.

Note that AudioStreamPlayer::play has an optional parameter, from_position. While you cannot give that parameter a negative value, maybe you could try giving it the leftover time, so that it plays from the right position?

There, you go, with that and an exact timer you could have a go at scheduling something reasonably accurately into the future with some silence tacked on the front. You'd still have to deal with audio buffer issues, but you could get around the vagaries of when the process or physics_process got called.

By the way, here is a version with "buffers" of silence detected at each frame rate, yet it does not seem to sound constant if one taps a moderately high bpm (like 200). Especially after a while, things drift and become non regular. Anyone knows why?
https://github.com/aleksfadini/metrollnome

The hack with the silence at the start seems to be working reasonably well, it sounds pretty in sync at least at the starting tempo.
I haven't really examined the logic in your source (and there seemed to be a double note sounding at low bpms on mine) but one possibility for the weirdness at higher BPMs is actually bugs in the godot audio player, I reported one here #22016. You could try the workaround I found in that issue and see if it helps, but clearly it isn't bug free. Also you might want to try the latest 3.2 alpha version, it is possible some of these bugs could have been fixed.
Generally though for this kind of thing, ideally I would personally would be aiming to write directly to an audio stream (primary buffer) in c++, that way you can get timing bang on. But it looks as though you are getting reasonably close with these methods playing through godot API, this might be good enough for your purposes.
What might be nice for apps of this type is native ability to schedule a sound to be played in the future, i.e. allow the _from_position parameter to be a negative value. It is possible to use the silence trick with drums, but if you were playing anything at variable sample rates (e.g. instrument notes at different pitches) it becomes more complex to calculate the delay. You could try adding this to godot proposals, as it may be quite simple to implement, however you would need broad support and it is quite a niche use (unless you could argue using it for game sound effects too).

I haven't really examined the logic in your source (and there seemed to be a double note sounding at low bpms on mine) but one possibility for the weirdness at higher BPMs is actually bugs in the godot audio player, I reported one here #22016. You could try the workaround I found in that issue and see if it helps, but clearly it isn't bug free. Also you might want to try the latest 3.2 alpha version, it is possible some of these bugs could have been fixed.

Things fall apart when I tap even only at 200 bpm. For some reason they sound fine at 120 bpm.
I tried to use your workaround, but I get this error in the editor:
Method StreamFromSType is not declared in the current class.
I was trying to add your code in my main scene script (Node2d.gd), but I get the same error even when using it as an AudioStream script. Any ideas? I updated the github repository if you want to see where things are.
My hope was to incorporate your code in my function "play_with_delay". It is also unclear to me if I can use "from position" in your work around.

Ah yes it's a while since I wrote that, it contains some project specific stuff (the stype was the sound type), but the general gist is that instead of reusing an AudioStreamPlayer you create a new one each time you want to play a sound, attach it as a node to the scene graph, and make it so that when the sound has finished playing it destroys itself. I actually got the idea from a suggestion by Calinou in another forum. This is gross, but it worked around the bug that was causing glitches, and that same bug could conceivably cause timing abnormalities too.
That is just a wild hunch though, there could well be something else causing your issue at 200 bpm.

You can implement @lawnjelly's suggestion by using this autoload: https://github.com/Calinou/escape-space/blob/master/autoload/sound.gd

Thank you everyone.
I did implement it in that way, instancing an audiostream and queuing it free. It did improve things ... a little. If you launch the project now, at 350BPM, you can clearly here something "new": the jitteriness is clear but is periodic. You can see how it correlates to the cyclical "discrepancies" in the latency buffer I have created, in relation to how the frames fall across the timeline.
I wonder if I could expand my code to include the latency of the audioserver through:
AudioServer.get_time_to_next_mix() and AudioServer.get_output_latency()
in order to finally achieve a decent result.
for your convenience, this is my repo again:
https://github.com/aleksfadini/metrollnome
Thank you for chiming in, it would be nice to have a solid work around and perhaps we could upload a metronome example to the Godot's assets store if we manage to get it working as it should.

AudioServer.get_time_to_next_mix() and AudioServer.get_output_latency()

Ah, that sounds perfect for your use, new in 3.2 it looks like. :smile: get_time_to_next_mix() should give you the random fluctuation due to audio tiles I was mentioning, so if you can compensate for that, you should be able to get much better timing.
https://docs.godotengine.org/en/latest/tutorials/audio/sync_with_audio.html

I've faced this same issue and investigating a little bit I came to more or less the same conclusions (need a precise clock and ahead-of-time scheduling). It would be nice if this is implemented on the engine side.
While investigating I found this paper written by Aaron Patterson about the implementation of the SonicPI clock (a music livecoding environment). Perhaps has some interesting ideas https://www.cs.kent.ac.uk/people/staff/dao7/publ/farm14-aaron.pdf
There's also the "tale of 2 clocks" https://www.html5rocks.com/en/tutorials/audio/scheduling/ which doesn't talk about implementation, but usage.
Hope it helps and we can have precise audio scheduling in godot soon!

Isn't it possible to avoid all these issues by writing a small wrapper around AudioStreamGenerator, which internally holds e.g. AudioStreamSample data with additional "time scheduling" information, and renders the raw samples exactly at the desired position into the audio output buffer. Basically how real sequencers/synths work.

@bluenote10 If I understand it correctly from the documentation you linked this would indeed be a viable solution for scheduled playback of audio (and possibly solving the OPs problem?). But doesn't it also mean that one needs to re-implement things like i.e. AudioStreamPlayer2D if for example spatial audio is required?

@kerskuchen if implemented with a stream, it is decoupled from spatialized playback, so I don't think you'd have to recode that.

Ah @Zylann thanks for the clarification! So the AudioStreamPlayer2D could take a AudioStreamGenerator as its "input" stream because AudioStreamGenerator is itself an AudioStream. Thats nice and convenient :)

I believe issue 38215 is part of the problem.
TLDR: The mixer can never mix at intervals below 1024 frames, so assuming your sample rate is 44100 then 1024 / 44100 = ~0.02321. This means sounds can only be timed at multiples of approximately 0.02321 seconds, which is not enough for precise musical timing.

I believe issue 38215 is part of the problem.
TLDR: The mixer can never mix at intervals below 1024 frames, so assuming your sample rate is 44100 then 1024 / 44100 = ~0.02321. This means sounds can only be timed at multiples of approximately 0.02321 seconds, which is not enough for precise musical timing.

Quite interesting. Not sure how this could be fixed.

I believe issue 38215 is part of the problem.
TLDR: The mixer can never mix at intervals below 1024 frames, so assuming your sample rate is 44100 then 1024 / 44100 = ~0.02321. This means sounds can only be timed at multiples of approximately 0.02321 seconds, which is not enough for precise musical timing.

Quite interesting. Not sure how this could be fixed.

Fixed in PR 38280. I have a version of 3.2 with this fix backported for a music game (among other audio improvements)

Ah @Zylann thanks for the clarification! So the AudioStreamPlayer2D could take a AudioStreamGenerator as its "input" stream because AudioStreamGenerator is itself an AudioStream. Thats nice and convenient :)

Hi, so that's the solution to play a perfect loop of multiple audio files?

Depending on what you mean, there might be no solution. If you by perfect loop you mean that they keep themselves in sync over time.

Closing in favor of https://github.com/godotengine/godot-proposals/issues/1151, as this requires new engine features to be implemented.
See also https://github.com/godotengine/godot/pull/63265, which may be useful in some situations.


Add new journals/proceedings to publications.html.

[x] Springer proceedings
[ ] Jurnal Agribisnis Indonesia (FEM IPB) *DOAJ
[ ] Jurnal Manajemen dan Agribisnis (SB IPB) *Terakreditasi
[ ] Indonesian Journal of Business and Entrepreneurship (SB IPB) *Terakreditasi

Also find a way to include indexing and accreditation in the description.