Self cal stats

What type of PR is this?

[ ] API-change
[ ] BUG
[ ] Improvement
[ ] Documentation
[ ] Feature
[ ] Test and CI
[ ] Code Refactoring

Which issue(s) this PR fixes:
issue #
What this PR does / why we need it:
memorytime = Memory_Bytes*duration_ns 结果会溢出

建议：非聚合的保留为 Memory_Bytes;
聚合的取值为：( Memory_Bytes*duration_ns ) / duration. (duration = aggrWindow)
Memory_Bytes: unint64 (40 TB)
duration_ns: 1e9 （9）  byte*s
JSON_EXTRACT 目前仅支持小数掉后 3位
注：最后 CU最后只展示 0.01 两位小数，保证三位小数结果

精度验证

F = acpu + bmemdur + cS3Input + d*S30utput
F / e = result = 3072.12
result 误差小于 ± 0.01 ==> 因为 四舍五入的进位规则
==> result + delta ~= result ~= result - delta
==> | delta | < 0.005
假设 delte 计算全部来自 mem的计算
==> | F / e | < 0.005
==> | (b * (mem * dur))  *  e | < 0.005
==> | (mem * dur) | < 0.005 * e / b =  (0.01 * e / b) / 2 = (1.38 * 10^15) / 2 = 6.9 * 10^14
即 | mem * dur | < 6.9 * 10^14
假设 dur =    5 * 10^9, 则 | mem 误差 | < 1.38 * 10^5,
假设 dur = 10^4 * 10^9, 则 | mem 误差 | < 69.

@xzxiong Thanks for your contributions!
Here are review comments for file pkg/util/trace/impl/motrace/Aggr.go:
Review:
Title: Self cal stats
Body:

The body of the pull request is incomplete. It does not provide any information about what the PR does or why it is needed. This makes it difficult to understand the purpose of the changes.

Changes:

The changes made in the pull request modify the file pkg/util/trace/impl/motrace/Aggr.go.
A new constant DurationKey of type key is added.
The NewAggregator function is modified to add the windowSize value to the context using context.WithValue.

Suggestions:

The title of the pull request should provide a clear and concise summary of the changes made. It should accurately describe the purpose of the changes.
The body of the pull request should provide a detailed explanation of what the PR does and why it is needed. It should include information about any issues that are being fixed or any improvements being made.
The changes made in the pull request should be clearly explained in the body. Each change should be described in detail, including the reasoning behind it.
It would be helpful to include code comments explaining the purpose and usage of the new constant DurationKey.
It would be beneficial to include code comments explaining the purpose and usage of the modified NewAggregator function, particularly the addition of the windowSize value to the context.

Optimizations:

The changes made in the pull request seem to be minimal and focused on adding a new constant and modifying a function. It is difficult to suggest optimizations without a clear understanding of the purpose of the changes.

Here are review comments for file pkg/util/trace/impl/motrace/Aggr_test.go:
Title: Self cal stats
Body: This pull request adds self-calculating statistics functionality to the Aggregator in the Aggr_test.go file. It also includes some updates and improvements to the existing code.
Changes made in pkg/util/trace/impl/motrace/Aggr_test.go:

Added import statement for github.com/stretchr/testify/require
Added test cases and assertions for the self-calculating statistics functionality
Added a new instance of the Aggregator with updated parameters
Added more test cases and assertions for the updated Aggregator

Overall, the changes seem to be aimed at enhancing the functionality and test coverage of the Aggregator in the Aggr_test.go file.
Upon reviewing the pull request, I have identified the following problems that need addressing:

Inconsistent naming convention: The file Aggr_test.go does not follow a consistent naming convention. It should be renamed to aggr_test.go to match the naming convention used for test files.

Lack of description in the pull request body: The pull request body does not provide a clear description of the changes made and the purpose of the self-calculating statistics functionality. It would be helpful to provide a brief explanation of the motivation behind these changes and how they improve the codebase.

Missing issue reference: The pull request body mentions fixing an issue, but does not provide a reference to the specific issue number. It would be beneficial to include the issue number for better tracking and context.

Lack of comments in the code: The added code lacks comments to explain the purpose and functionality of the new test cases and assertions. It would be helpful to add comments to improve code readability and maintainability.

To optimize the changes made in the pull request, I suggest the following:

Provide a more detailed and descriptive pull request body: Include a clear explanation of the changes made, the motivation behind them, and how they improve the codebase. This will help reviewers and future maintainers understand the purpose and impact of the changes.

Add comments to the code: Include comments above the new test cases and assertions to explain their purpose and functionality. This will make the code easier to understand and maintain.

Ensure consistent naming conventions: Rename the file Aggr_test.go to aggr_test.go to follow the established naming convention for test files. This will improve consistency and readability in the codebase.

By addressing these problems and implementing the suggested optimizations, the quality and clarity of the codebase will be improved.
Here are review comments for file pkg/util/trace/impl/motrace/report_statement.go:
Title: Self cal stats
Body: The pull request adds self-calculating statistics functionality to the StatementInfo struct in the report_statement.go file. It includes changes to calculate and update the aggregated memory usage and duration of a statement.
Changes Made:

Added import statements for types and zap packages.
Added constants for Decimal128Width and Decimal128Scale.
Added functions convertFloat64ToDecimal128 and mustDecimal128 to convert float64 to Decimal128 type.
Modified the StatementInfoNew function to set the Duration field to the windowSize value from the context and calculate the aggregated memory usage using the convertFloat64ToDecimal128 and mustDecimal128 functions.
Modified the StatementInfoUpdate function to remove the addition of Duration field.
Added AggrMemoryByte field of type types.Decimal128 to the StatementInfo struct.
Modified the FillRow function to calculate the aggregated memory usage and update the statsArray accordingly.
Added the mergeStats function to merge the statistics of two StatementInfo instances.
Modified the ExecPlan2Stats function to initialize the statsArray if it is nil and return the JSON representation of the statsArray.
Added the MarkResponseAt function to set the ResponseAt field and calculate the Duration field if it is not already set.
Modified the EndStatement function to call the MarkResponseAt function.

Review:

Inconsistent naming convention: The function StatementInfoNew should be renamed to NewStatementInfo to follow the Go naming convention for constructors.

Unused import: The strings package is imported but not used in the file. It should be removed.

Magic numbers: The constants Decimal128Width and Decimal128Scale are used without any explanation of their significance. It would be helpful to add comments explaining their purpose.

Error handling: The function convertFloat64ToDecimal128 returns an error, but the error is not handled or returned. It would be better to handle the error and return it if necessary.

Redundant code: The line windowSize, _ := ctx.Value(DurationKey).(time.Duration) in the StatementInfoNew function assigns the value of DurationKey from the context to windowSize, but it is not used anywhere else in the function. This line can be removed.

Inefficient calculation: The calculation of the aggregated memory usage in the FillRow function can be optimized. Instead of converting the AggrMemoryByte to a float64, multiplying it by the duration, and then converting it back to Decimal128, the calculation can be simplified by directly multiplying the AggrMemoryByte by the duration using the Multiply method provided by the Decimal128 type.

Inconsistent use of error handling: The mergeStats function returns an error, but the error is not handled or returned. It would be better to handle the error and return it if necessary.

Inconsistent use of comments: Some functions have comments explaining their purpose and behavior, while others do not. It would be helpful to add comments to all functions to improve code readability and maintainability.

Inconsistent use of braces: The if statement in the ExecPlan2Stats function does not have braces, while the else statement does. It would be better to use consistent brace placement for readability.

Inconsistent naming of variables: The variables e and n in the StatementInfoUpdate function could be better named to improve code readability and maintainability.

Suggestions:

Rename the function StatementInfoNew to NewStatementInfo to follow the Go naming convention for constructors.

Remove the unused import statement for the strings package.

Add comments to explain the purpose and significance of the Decimal128Width and Decimal128Scale constants.

Handle the error returned by the convertFloat64ToDecimal128 function and return it if necessary.

Remove the line windowSize, _ := ctx.Value(DurationKey).(time.Duration) from the StatementInfoNew function.

Optimize the calculation of the aggregated memory usage in the FillRow function by directly multiplying the AggrMemoryByte by the duration using the Multiply method provided by the Decimal128 type.

Handle the error returned by the mergeStats function and return it if necessary.

Add comments to all functions to explain their purpose and behavior.

Use consistent brace placement in the ExecPlan2Stats function.

Rename the variables e and n in the StatementInfoUpdate function to improve code readability and maintainability.

Here are review comments for file test/distributed/cases/database/system_variables.result:
Title: Self cal stats
Body: This pull request makes changes to the file test/distributed/cases/database/system_variables.result. The changes involve removing some lines of code related to @@foreign_key_checks and adding a line to set @@sql_mode to default.
Changes made in test/distributed/cases/database/system_variables.result:

Lines 480-484: Removed code related to @@foreign_key_checks.
Line 486: Added code to set @@sql_mode to default.

Overall, the changes in this pull request seem to be focused on modifying the system variables in the test case. However, there are a few issues that need to be addressed:

Lack of context in the title and body: The title and body of the pull request do not provide enough context about the purpose of the changes. It would be helpful to include a brief explanation of why these changes are necessary and what problem they are addressing.

Suggestion: Provide a clear and concise explanation of the purpose of the changes in the title and body. For example, "Fix issue with system variable handling in test case" or "Improve test case by removing unnecessary code related to foreign key checks".

Inconsistent formatting: The changes made in the file test/distributed/cases/database/system_variables.result have inconsistent formatting. The removed lines have been commented out, but the added line does not follow the same formatting.

Suggestion: Ensure consistent formatting throughout the file. Either comment out the added line or remove the commented lines.

Lack of explanation for removed code: The removed lines of code related to @@foreign_key_checks are not explained in the pull request. It is unclear why these lines were removed and what impact it may have on the test case.

Suggestion: Provide a brief explanation in the pull request body or comments about why the code related to @@foreign_key_checks was removed and what impact it has on the test case.
Optimization suggestions:

Consider providing more detailed commit messages for each change made in the pull request. This will help future developers understand the purpose and impact of each change.
If possible, provide additional test cases or scenarios to ensure the changes have been thoroughly tested and do not introduce any regressions.

Overall, addressing the issues mentioned above and providing more context and explanation for the changes will improve the quality and clarity of this pull request.
Here are review comments for file test/distributed/cases/database/system_variables.sql:
Title: Self cal stats
Body: This pull request aims to make changes to the file test/distributed/cases/database/system_variables.sql. The changes include adding a test case for the interactive_timeout variable and removing some unnecessary code related to foreign_key_checks.
Changes made in test/distributed/cases/database/system_variables.sql:

Added a test case for the interactive_timeout variable with a reference to issue #10473.
Removed code related to foreign_key_checks which was not needed.

Review:

The title of the pull request is not descriptive enough. It should provide a clear indication of what the changes are about. Consider changing the title to something like "Add test case for interactive_timeout variable and remove unnecessary code".

The body of the pull request is incomplete. It does not provide enough information about the changes made and why they are necessary. Consider providing a more detailed explanation of the changes and their purpose.

The changes made in the system_variables.sql file are not clearly explained. It is unclear why the test case for interactive_timeout was added and why the code related to foreign_key_checks was removed. Consider providing clear explanations for each change made.

The code changes themselves seem fine. However, it would be helpful to include comments explaining the purpose of each test case and the reason for removing the code related to foreign_key_checks.

Suggestions:

Update the title of the pull request to provide a clear indication of the changes made.

Expand the body of the pull request to provide a detailed explanation of the changes and their purpose.

Add comments to the code changes in system_variables.sql to explain the purpose of each test case and the reason for removing the code related to foreign_key_checks.

Consider optimizing the changes by organizing the code in a more structured and readable manner.

Here are review comments for file test/distributed/cases/dml/insert/insert.result:
Title: Self cal stats
Body: This pull request aims to address issue # and make changes to the file test/distributed/cases/dml/insert/insert.result. The changes made in this file include modifying the expected results for certain queries.
Changes made in test/distributed/cases/dml/insert/insert.result:

Line 361: The expected result for the query select count(*) from t; has been changed from 200000 to 400000.
Line 364: The expected result for the query select count(*) from t; has been changed from 200000 to 400000.

Overall, the changes in this pull request seem to be related to fixing an issue and updating the expected results for certain queries.
However, there are a few issues that need to be addressed:

Lack of clear explanation: The pull request title and body do not provide a clear explanation of what the changes are intended to achieve. It would be helpful to include a brief description of the issue being fixed and why the changes are necessary.

Missing issue number: The pull request body mentions that it addresses an issue, but it does not specify the issue number. It is important to include the issue number so that it can be easily tracked and linked to the pull request.

To optimize the changes made in this pull request, the following suggestions can be considered:

Provide a more detailed explanation: In the pull request body, provide a clear and detailed explanation of the issue being fixed and why the changes are necessary. This will help reviewers and future readers understand the context and purpose of the changes.

Include the issue number: Specify the issue number in the pull request body to provide a clear reference to the problem being addressed. This will make it easier for reviewers and future readers to understand the context and track the progress of the issue.

By addressing these issues and providing a more detailed explanation, the pull request will be more informative and easier to review.
Here are review comments for file test/distributed/cases/dml/insert/insert.test:
Title: Self cal stats
Body: This pull request aims to make some changes to the insert.test file in the test/distributed/cases/dml/insert directory. The changes include adding and removing some comments and tags.
 However, the use of comments and tags seems inconsistent. It would be helpful to have a clear and consistent approach to using comments and tags throughout the file.

Suggestion: Review the purpose and guidelines for using comments and tags in the test files and ensure consistent usage.

Lack of explanation for changes: The pull request does not provide any explanation for the changes made in the insert.test file. It would be beneficial to include a brief explanation of why these changes are necessary or what problem they aim to solve.

Suggestion: Add a brief explanation in the pull request body to provide context for the changes made in the insert.test file.
Optimization:

Consolidate changes: The changes made in this pull request involve adding and removing comments and tags in multiple places in the insert.test file. It would be more efficient to consolidate these changes into a single location to make the code easier to read and review.

Suggestion: Combine the changes into a single commit or make the changes in a single location in the insert.test file.
Overall, this pull request needs some improvements in terms of consistent usage of comments and tags, providing explanations for changes, and optimizing the changes made.
Here are review comments for file test/distributed/cases/dml/insert/insert_with_function.sql:
Title: Self cal stats
Body: This pull request aims to make some changes to the file test/distributed/cases/dml/insert/insert_with_function.sql. The changes involve adding a newline at the end of the file.
Changes made in test/distributed/cases/dml/insert/insert_with_function.sql:

Line 94: Added a newline at the end of the file.

Review:

The title of the pull request is not descriptive enough. It should provide more information about the changes being made. For example, a more appropriate title could be "Add newline at the end of insert_with_function.sql file".

The body of the pull request is incomplete. It does not provide any context or explanation for why the changes are being made. It would be helpful to include a brief description of the problem being addressed or the goal of the changes.

The changes made in the file test/distributed/cases/dml/insert/insert_with_function.sql seem unnecessary. Adding a newline at the end of the file does not appear to have any functional impact on the code. It would be more useful to focus on changes that improve the code's functionality, readability, or performance.

Suggestions:

Update the title of the pull request to provide a clear and concise description of the changes being made.

Expand the body of the pull request to include a brief explanation of the problem being addressed or the goal of the changes. This will help reviewers understand the context and importance of the changes.

Consider making more meaningful changes to the codebase that improve functionality, readability, or performance. Adding a newline at the end of a file does not provide any significant benefit and can be considered a trivial change.

If there are other changes made in addition to the newline addition, please include them in the pull request description. This will help reviewers understand the full scope of the changes being made.

Here are review comments for file test/distributed/cases/dml/show/database_statistics.sql:
Review:
Title: Self cal stats
The title of the pull request is not very clear and does not provide much information about the changes being made.
Body:
The body of the pull request is incomplete and does not provide any information about the changes being made or the purpose of the pull request. It only contains a checklist for the type of PR and a placeholder for the issue number.
Changes:
The changes made in the pull request are in the file test/distributed/cases/dml/show/database_statistics.sql. The changes include adding a skip directive -- @skip:issue#10473 and modifying the SQL query.
Suggestions:

Title: Provide a more descriptive title for the pull request that clearly explains the purpose of the changes.

Body: Provide a detailed description of the changes being made and the reason for making them. This will help reviewers understand the context and purpose of the pull request.

Changes: It is unclear why the skip directive -- @skip:issue#10473 is being added. If there is a specific issue that needs to be skipped, it should be mentioned in the body of the pull request. Additionally, the modified SQL query should be explained in the body of the pull request.

Optimize changes: It is difficult to provide optimization suggestions without a clear understanding of the purpose of the changes. However, it is always a good practice to ensure that the changes made are necessary and do not introduce any unnecessary complexity or performance issues.

Overall, the pull request needs more information and context to be properly reviewed.
Here are review comments for file test/distributed/cases/dml/show/show.result:
Title: Self cal stats
Body: This pull request aims to make a small change to the test/distributed/cases/dml/show/show.result file. The change involves updating the expected result for a specific test case. The current expected result is 13, but it should be 14.
Changes made in test/distributed/cases/dml/show/show.result:
diff --git a/test/distributed/cases/dml/show/show.result b/test/distributed/cases/dml/show/show.result
@@ -238,7 +238,7 @@ mo_columns
 mo_tables
 show table_number from mo_catalog;
 Number of tables in mo_catalog
-13
+14
 show column_number from mo_database;
 Number of columns in mo_database
 9

Review:

The title of the pull request is not descriptive enough. It should provide more information about the specific change being made, such as "Update expected result for show table_number test case in show.result file".

Suggestions:

Update the title to be more descriptive and specific.

The body of the pull request is incomplete and lacks important information. It should include details about the issue being fixed, the reason for the change, and any potential impact or risks associated with the change.

Suggestions:

Provide information about the issue being fixed, such as the issue number or a brief description.
Explain why this change is necessary and what problem it solves.
Mention any potential impact or risks associated with the change.

The changes made in the show.result file seem to be correct and aligned with the purpose of the pull request.

Optimization suggestions:

None.

Overall, this pull request needs improvements in the title and body to provide a clear understanding of the changes being made and the reasons behind them.
Here are review comments for file test/distributed/cases/function/func_coalesce.test:
Title: Self cal stats
Body: This pull request aims to make changes to the file test/distributed/cases/function/func_coalesce.test. The changes include adding two new test cases and modifying an existing test case. The purpose of these changes is to improve the coverage and accuracy of the test cases for the COALESCE function.
Changes made in test/distributed/cases/function/func_coalesce.test:

Line 136: Added a new test case that casts the result of COALESCE(a, a) to a DECIMAL(25,3) data type. This test case is tagged with -- @bvt:issue#10473.
Line 139: Added a new test case that casts the result of COALESCE(a, a) to a DATETIME(6) data type. This test case is tagged with -- @bvt:issue.
Line 207: Modified an existing test case that performs a UNION ALL operation on the tables t1 and t2. No specific details about the modification are provided.
Line 210: Removed the line that had no newline at the end of the file.

Overall, the changes made in this pull request aim to enhance the test coverage for the COALESCE function by adding new test cases and modifying an existing one.
Review:

The title of the pull request is not descriptive enough. It should provide a clear indication of the purpose of the changes made in the pull request. Consider updating the title to something like "Enhance test coverage for COALESCE function in func_coalesce.test".

The body of the pull request is incomplete. It lacks information about the specific issues being addressed and the reasoning behind the changes. It would be helpful to provide more context and explanation in the body of the pull request.

The changes made in the func_coalesce.test file are not well-documented. It is unclear why the new test cases were added and what specific issues they are addressing. Providing more detailed comments or explanations in the code would improve the clarity and maintainability of the test cases.

The modification made to the existing test case is not explained. It would be beneficial to provide a brief description of the modification and the reason behind it.

Suggestions:

Update the title of the pull request to provide a clear indication of the purpose of the changes made.

Expand the body of the pull request to include information about the specific issues being addressed and the reasoning behind the changes.

Add comments or explanations in the code to provide more context and clarity for the new test cases.

Provide a brief description of the modification made to the existing test case and the reason behind it.

Consider optimizing the changes by ensuring that the test cases cover a wide range of scenarios and edge cases for the COALESCE function. This will help improve the overall quality and reliability of the test suite.

Here are review comments for file test/distributed/cases/function/func_datetime_utc_timestamp.result:
Title: Self cal stats
Body: The pull request is categorized as an "Improvement". It does not reference any specific issue that it fixes. The purpose of this pull request is not clearly stated.
Changes made in the pull request:

In the file test/distributed/cases/function/func_datetime_utc_timestamp.result, there are changes made to the expected results of the SQL queries.
In the first query, the year value has been changed from 2022 to 2023.
In the second query, the year value has been changed from 2022 to 2023.

Problems:

Lack of context: The body of the pull request does not provide any information about what the changes are intended to achieve or why they are necessary. It would be helpful to include a brief explanation of the purpose of the changes.

Suggestions:

Provide a clear explanation of the purpose of the changes in the body of the pull request. This will help reviewers understand the motivation behind the changes and provide more informed feedback.

Optimization:

The changes made in the pull request seem to be straightforward updates to the expected results of the SQL queries. There doesn't appear to be any optimization needed for these changes. However, it's important to ensure that the changes are accurate and reflect the expected behavior of the system.

Here are review comments for file test/distributed/cases/function/func_datetime_utc_timestamp.test:
Title: Self cal stats
Body: This pull request aims to add self-calculating statistics functionality.
Changes made in the file test/distributed/cases/function/func_datetime_utc_timestamp.test:

Added a test case for self-calculating statistics.
Added a comment to indicate the purpose of the test case.

Overall, the changes seem to be focused on adding a new test case for self-calculating statistics. However, there are a few issues that need to be addressed:

The title of the pull request is not descriptive enough. It should provide more information about the specific functionality being added or modified.

Suggestion: Update the title to something like "Add test case for self-calculating statistics".

The body of the pull request is incomplete. It does not provide any explanation of the changes made or the purpose of the test case.

Suggestion: Provide a detailed description of the changes made and the reason for adding the test case. This will help reviewers understand the context and purpose of the changes.

The changes made in the test file are not optimized. The added comment does not provide any useful information and can be removed.

Suggestion: Remove the comment as it does not add any value to the test case.

The test case itself is not clearly explained. It is unclear what the expected output of the test case should be.

Suggestion: Add a comment or description to explain the expected output of the test case. This will make it easier for reviewers to understand the purpose of the test case and verify its correctness.
By addressing these issues, the pull request will be more informative and easier to review.
Here are review comments for file test/distributed/cases/function/func_string_replace.result:
Title: Self cal stats
Body: The pull request does not provide any information about the type of PR or the issue it fixes.
Changes made in the pull request:

In the file test/distributed/cases/function/func_string_replace.result, lines 443 to 448 were removed.

Problems:

Lack of information: The pull request does not provide any information about the type of PR or the issue it fixes. This makes it difficult to understand the purpose and context of the changes.

Suggestions:

Provide information: The pull request should clearly state the type of PR (e.g., bug fix, feature, improvement) and reference the issue it fixes (if applicable). This will help reviewers understand the purpose and context of the changes.

Optimization:
No optimization suggestions can be made based on the provided information.
Here are review comments for file test/distributed/cases/function/func_string_replace.test:
Title: Self cal stats
Body: This pull request aims to make changes to the file test/distributed/cases/function/func_string_replace.test. The changes involve removing a line of code that selects the UNIX_TIMESTAMP from the REPLACE function in the replace_04 table.
Changes made in test/distributed/cases/function/func_string_replace.test:

Line 201: Removed the line that selects UNIX_TIMESTAMP from the REPLACE function in the replace_04 table.

Overall, the changes made in this pull request seem to be straightforward and focused on removing a specific line of code. However, there are a few issues that need addressing:

Lack of context in the body: The body of the pull request does not provide enough context or explanation for why the change is necessary. It would be helpful to include a brief description of the problem being addressed or the reason for removing the line of code.

To address this, the body of the pull request should be updated to provide a clear explanation of why the line of code is being removed. For example, if the line of code is causing a bug or is no longer needed, that information should be included in the body.

Missing reference to the issue: The pull request body includes a section for referencing the issue(s) that the pull request fixes, but it is left blank. It is important to reference the relevant issue(s) so that the pull request can be easily tracked and linked to the corresponding problem or feature.

To address this, the pull request should be updated to include the issue number(s) that this pull request fixes. This will help provide a clear connection between the pull request and the issue(s) it addresses.
In terms of optimizing the changes made in the pull request, it seems that the removal of the line of code is a straightforward and appropriate change. However, it is always a good practice to thoroughly test the code after making changes to ensure that it does not introduce any new issues or break existing functionality. Additionally, it may be helpful to include a brief comment in the code explaining the reason for the removal of the line, especially if it is not immediately obvious.
Here are review comments for file test/distributed/cases/function/table_func_metadata_scan.sql:
Title: Self cal stats
Body: This pull request makes changes to the file test/distributed/cases/function/table_func_metadata_scan.sql. It adds two new queries to the file and also includes a comment referencing two issues.
Changes made in test/distributed/cases/function/table_func_metadata_scan.sql:

Line 24: Added a new query to select col_name, rows_cnt, null_cnt, and origin_size from the metadata_scan function with the table name table_func_metadata_scan.t and column name a.
Line 25: Added a new query to select col_name, rows_cnt, null_cnt, and origin_size from the metadata_scan function with the table name table_func_metadata_scan.t and all columns.
Line 26: Added a new query to select the sum of origin_size from the metadata_scan function with the table name table_func_metadata_scan.t and all columns.
Line 28: Added a comment referencing an issue with the tag @bvt:issue#10473.
Line 29: Added a comment referencing an issue with the tag @bvt:issue.
Line 31: Added a query to drop the table t if it exists.

Problems:

The title of the pull request is not descriptive enough. It should provide more information about the changes made, such as "Added new queries to table_func_metadata_scan.sql".
The body of the pull request is incomplete. It does not provide any explanation of why the changes were made or what problem they are addressing.
The changes made in the file test/distributed/cases/function/table_func_metadata_scan.sql are not well-documented. There should be comments explaining the purpose of the new queries and why they are necessary.
The comments referencing the issues in the file (@bvt:issue#10473 and @bvt:issue) are not clear. They should provide more context about the issues they are addressing.

Suggestions:

Update the title of the pull request to provide a clear and concise description of the changes made.
Provide a detailed explanation in the body of the pull request about why the changes were made and what problem they are solving.
Add comments above the new queries in test/distributed/cases/function/table_func_metadata_scan.sql to explain their purpose and why they are needed.
Improve the comments referencing the issues in the file by providing more context and information about the issues being addressed.

Here are review comments for file test/distributed/cases/procedure/procedure.sql:
Title: Self cal stats
Body: This pull request makes changes to the file test/distributed/cases/procedure/procedure.sql. It adds a new test case for the test_repeat() procedure and includes a reference to an issue (#10477) in the comments.
Changes made in test/distributed/cases/procedure/procedure.sql:

Line 80: Adds a new test case for the test_repeat() procedure.
Line 83: Adds a comment referencing issue #10477.
Line 85: Calls the test_repeat() procedure.
Line 87: Adds a comment indicating the end of the test case.

Overall, the changes seem to be adding a new test case for a procedure and referencing an issue in the comments.
However, there are a few issues that need addressing:

The title of the pull request is not descriptive enough. It should provide a clear and concise summary of the changes made. For example, a more descriptive title could be "Add test case for test_repeat() procedure and reference issue #10477".

The body of the pull request is incomplete. It should provide more information about the purpose of the changes and why they are needed. Additionally, it should mention any potential impact or risks associated with the changes.

The changes made in the file test/distributed/cases/procedure/procedure.sql are not well-documented. It would be helpful to include comments explaining the purpose of the new test case and any specific scenarios it is testing.

To optimize the changes made in the pull request, consider the following suggestions:

Provide a more descriptive title that clearly summarizes the changes made in the pull request.

Expand the body of the pull request to provide more context and explain the purpose of the changes. Mention any potential impact or risks associated with the changes.

Add comments in the procedure.sql file to explain the purpose of the new test case and any specific scenarios it is testing. This will make it easier for future developers to understand the intent of the test case.

By addressing these issues and optimizing the changes, the pull request will be more informative and easier to review and merge.
Here are review comments for file test/distributed/cases/zz_accesscontrol/create_user.result:
Title: Self cal stats
Body: This pull request makes changes to the file test/distributed/cases/zz_accesscontrol/create_user.result. The changes include modifying the count of users and fixing an internal error related to a user. Additionally, there is a change to the global setting enable_privilege_cache.
Changes in test/distributed/cases/zz_accesscontrol/create_user.result:

Line 53: The count of users is changed from 24 to 13.
Line 58: An internal error related to the user user3 is fixed.
Line 61: The count of users is changed from 24 to 13.
Line 138: The global setting enable_privilege_cache is modified.

Review:

The title of the pull request is not descriptive enough. It should provide more information about the changes being made.
The body of the pull request is incomplete. It does not provide any explanation of why the changes are necessary or what problem they are addressing.
The changes made in the file test/distributed/cases/zz_accesscontrol/create_user.result seem to be unrelated to the title and body of the pull request. It is unclear how these changes are related to "Self cal stats".
The changes in the file test/distributed/cases/zz_accesscontrol/create_user.result should be explained in more detail. It is unclear why the count of users is being changed and what impact it has on the system.
The pull request does not reference any specific issue that it fixes. It would be helpful to provide a reference to the relevant issue(s) for better tracking and context.
The pull request template includes checkboxes for different types of PRs, but they are not filled out. It would be better to remove these checkboxes if they are not being used.

Suggestions:

Update the title of the pull request to provide a clear and concise description of the changes being made.
Expand the body of the pull request to explain the purpose and motivation behind the changes. Include information about any issues or bugs that are being addressed.
Provide more detailed explanations for the changes made in the file test/distributed/cases/zz_accesscontrol/create_user.result. Explain why the count of users is being modified and what impact it has on the system. Also, provide more context for the internal error fix and the modification of the global setting.
Reference the relevant issue(s) that the pull request fixes.
Remove the unused checkboxes from the pull request template.

Here are review comments for file test/distributed/cases/zz_accesscontrol/create_user.sql:
Title: Self cal stats
Body: This pull request makes changes to the file test/distributed/cases/zz_accesscontrol/create_user.sql. It adds a new SQL query and modifies an existing one. The changes include adding a comment with a reference to an issue and modifying the set global enable_privilege_cache statement.
Changes made in test/distributed/cases/zz_accesscontrol/create_user.sql:

Line 39: Added a new SQL query with a comment referencing issue #10473.
Line 45: Modified an existing SQL query by removing the comment referencing an issue.

Overall, the changes seem to be related to user creation and testing in the access control module.
Upon reviewing this pull request, I have identified a few issues that need addressing:

Inconsistent use of comments: The added SQL query on line 39 includes a comment referencing issue #10473, while the modified SQL query on line 45 removes a comment referencing an issue. It would be better to have consistent use of comments throughout the file.

Lack of explanation for changes: The pull request does not provide any explanation for why these changes are necessary. It would be helpful to include a brief explanation of the purpose or goal of these changes.

To optimize the changes made in this pull request, I would suggest the following:

Consistent use of comments: Ensure that all SQL queries have comments explaining their purpose or any relevant references to issues. This will make the code more understandable and maintainable.

Provide an explanation for changes: Add a brief explanation in the pull request body or comments within the code to clarify the purpose or goal of the changes. This will help reviewers and future maintainers understand the context and intent of the modifications.

By addressing these issues and optimizing the changes, the pull request will be more clear, understandable, and maintainable.


git merge style

i noticed when PR's get accepted, there are merge commits. as a suggestion to keep the git log a bit more tidy, possibly consider fast forward merges only (i.e. git merge --ff-only <BRANCH_TO_MERGE>)?

Thanks for the suggestion. FWIW the git logs you've observed are a direct result of the github tooling processing a merge-this-PR operation. We can certainly consider fast-forward only merges. Though the convenience of clicking "Merge PR" in github is often too tempting.

understood about the gh click to merge. while it takes a little more for ff merges, keeps things cleaner and easier to look through.
going to close the issue, just wanted to raise an observation about the repo logs

Alternatively, a git rebase workflow is very effective. Slightly more difficult to get people to do correctly but you get clean flat logs and github merges work fine when each committer rebases their work right before pushing and PRing.


Removed setup.py/.cfg from reusable apps docs.

Thanks Tim for your review!

@claudep Thanks :+1: In the next few days I'm going to release a new version of a 3rd party package (django-request), let me check this in practice.

Good idea!

@felixxm I guess you were short on time to work on django-request :sweat_smile:

@felixxm I guess you were short on time to work on django-request 😅

Yes, sorry for the delay. I wanted to add just one more feature before release :smiley: .

Trying out... :crossed_fingers:


Issue building the image with ADD run /

Hi thanks for sharing you job.
I having a issue building the image specifically at
Step 2 : ADD run /
lstat run: no such file or directory
do you know why?

NVM, Oh I see, RUN is a sh. sorry


[Bug] Do not expose functions in PromEntry

Codecov Report

Merging #49 (1a4c069) into master (b9ccf5a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #49   +/-   ##
=======================================
  Coverage   77.70%   77.70%
=======================================
  Files          35       35
  Lines        3938     3938
=======================================
  Hits         3060     3060
  Misses        706      706
  Partials      172      172

Impacted Files
Coverage Δ

entry/prom_entry.go
79.66% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9ccf5a...1a4c069. Read the comment docs.


Ollama integration

Extend UpTrain to integrate with Ollama and use locally running LLMs as evaluators

Done by #623


fix the ability to configure incompatible package settings on startup

Now users of our package can configure build settings as they like.
For example, a user can configure OnLoginExecuting before he configures UseDefaultLoginMiddleware.
He can also configure UseDefaultLoginMiddleware and UseCookieLoginMiddleware, although it looks very strange.
These cases show that the package build configuration can have incompatible settings, so it is necessary to correct this point.
To do this, you need:

Find cases in which the package build settings seem illogical/incompatible and describe them
According to the described cases, configure the package build settings

Note: result commit should be marked as BREAKING CHANGE

Below I described use cases of incompatible/illogical package build settings.
For some problems, I just left comments.
JwtAuthentication.Core.ApplicationBuilderExtensions
When you double-call UseCookieLoginMiddleware, the tests fall
app.UseCookieLoginMiddleware(new CookieAuthOptions
        {
            Key = "ExampleCookieName",
        }
    )
    .UseCookieLoginMiddleware(new CookieAuthOptions
    {
        Key = "ExampleCookieName",
    });

A logical problem when you call default login middleware after cookie login middleware. This configuration don't break tests and the launch of the application but it seems very strange because I don't know how exactly it should work
app.UseCookieLoginMiddleware(new CookieAuthOptions
        {
            Key = "ExampleCookieName",
        }
    );

app.UseDefaultLoginMiddleware();


Adding images

Adding images is difficult.  Could you make it easy to perform this task?

you can also use
example
btw I use brackets

i'm still using an 18 year old copy of Macromedia Homesite+ to edit HTML documents because for some reason nobody since then has been able to implement drag + drop image tags on a responsive text editor (none of the Electron-based text editors are responsive enough for me -- i never want to wait even a millisecond for a typed character to appear on a screen!)


Support TOTP as a second factor.

It would have to depend on the details of the implementation, but starting it in its own package can't hurt. we can always do optional dependencies and tell the user to add deps to avoid bloating our deps. So once it is ready we can decide :)


How to decode without verifying?

I could not find a way to decode a token without verifying. My use case is about a private signed token, for which I need to fetch a public key based on a client-id inside the payload. After this I want to verify with the retrieved public key.

I could not find a way to decode a token without verifying.

Jwt
      .decodeRawAll(
        header,
        JwtOptions(signature = false, expiration = false, notBefore = false)
      )

It will give you json strings of header and contents

I could not find a way to decode a token without verifying.

Jwt
      .decodeRawAll(
        header,
        JwtOptions(signature = false, expiration = false, notBefore = false)
      )

It will give you json strings of header and contents

My use case is about a private signed token, for which I need to fetch a public key based on a client-id inside the payload. After this I want to verify with the retrieved public key.

I am doing exactly that in this project
https://github.com/bilal-fazlani/akka-http-jwt-auth

My use case is about a private signed token, for which I need to fetch a public key based on a client-id inside the payload. After this I want to verify with the retrieved public key.

I am doing exactly that in this project
https://github.com/bilal-fazlani/akka-http-jwt-auth

package tech.bilal.akka.http.auth.adapter

import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper}
import com.fasterxml.jackson.module.scala.{DefaultScalaModule, ScalaObjectMapper}
import pdi.jwt.{Jwt, JwtOptions}
import tech.bilal.akka.http.auth.adapter.crypto.Algorithm
import tech.bilal.akka.http.oidc.client.{JsonUtil, OIDCClient, PublicKeyManager}
import tech.bilal.akka.http.oidc.client.models.JWTHeader

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scala.reflect.{ClassTag, classTag}
import scala.util.Try

class JwtVerifier(
    oidcClient: OIDCClient,
    publicKeyManager: PublicKeyManager,
    authConfig: AuthConfig
) {

  def verifyAndDecode[T:ClassTag](tokenString: String): Future[Option[T]] = {
    val jwtOptions =
      JwtOptions(signature = true, expiration = true, notBefore = true)
    for {
      (header, jsonPayloadContents) <-
        Future.fromTry(getKIDAndContents(tokenString))
      keyMayBe <- publicKeyManager.getKey(header.kid)
      key = keyMayBe.getOrElse(
        throw RuntimeException(
          s"could not find key with kid: ${header.kid}"
        )
      )
      issuer <- oidcClient.fetchOIDCConfig.map(_.issuer)
      _ = if (issuer != authConfig.issuer)
        throw RuntimeException("invalid token issuer")
      algo =
        authConfig.supportedAlgorithms
          .find(_.toLowerCase == header.alg.toLowerCase)
          .map(Algorithm.apply)
          .getOrElse(
            throw RuntimeException(s"unsupported algorithm - ${header.alg}")
          )
      publicKey <- Future.fromTry(algo.flatMap(_.publicKey(key, header)))
      _ = Jwt.validate(tokenString, publicKey, jwtOptions)
      token <-
        Future.fromTry(JsonUtil.fromJson[T](jsonPayloadContents))
    } yield Some(token)
  }

  private def getKIDAndContents(header: String): Try[(JWTHeader, String)] =
    Jwt
      .decodeRawAll(
        header,
        JwtOptions(signature = false, expiration = false, notBefore = false)
      )
      .flatMap {
        case (decodedHeader, contents, _) =>
          JsonUtil.fromJson[JWTHeader](decodedHeader)
          .map((_, contents))
      }
}

package tech.bilal.akka.http.auth.adapter

import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper}
import com.fasterxml.jackson.module.scala.{DefaultScalaModule, ScalaObjectMapper}
import pdi.jwt.{Jwt, JwtOptions}
import tech.bilal.akka.http.auth.adapter.crypto.Algorithm
import tech.bilal.akka.http.oidc.client.{JsonUtil, OIDCClient, PublicKeyManager}
import tech.bilal.akka.http.oidc.client.models.JWTHeader

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scala.reflect.{ClassTag, classTag}
import scala.util.Try

class JwtVerifier(
    oidcClient: OIDCClient,
    publicKeyManager: PublicKeyManager,
    authConfig: AuthConfig
) {

  def verifyAndDecode[T:ClassTag](tokenString: String): Future[Option[T]] = {
    val jwtOptions =
      JwtOptions(signature = true, expiration = true, notBefore = true)
    for {
      (header, jsonPayloadContents) <-
        Future.fromTry(getKIDAndContents(tokenString))
      keyMayBe <- publicKeyManager.getKey(header.kid)
      key = keyMayBe.getOrElse(
        throw RuntimeException(
          s"could not find key with kid: ${header.kid}"
        )
      )
      issuer <- oidcClient.fetchOIDCConfig.map(_.issuer)
      _ = if (issuer != authConfig.issuer)
        throw RuntimeException("invalid token issuer")
      algo =
        authConfig.supportedAlgorithms
          .find(_.toLowerCase == header.alg.toLowerCase)
          .map(Algorithm.apply)
          .getOrElse(
            throw RuntimeException(s"unsupported algorithm - ${header.alg}")
          )
      publicKey <- Future.fromTry(algo.flatMap(_.publicKey(key, header)))
      _ = Jwt.validate(tokenString, publicKey, jwtOptions)
      token <-
        Future.fromTry(JsonUtil.fromJson[T](jsonPayloadContents))
    } yield Some(token)
  }

  private def getKIDAndContents(header: String): Try[(JWTHeader, String)] =
    Jwt
      .decodeRawAll(
        header,
        JwtOptions(signature = false, expiration = false, notBefore = false)
      )
      .flatMap {
        case (decodedHeader, contents, _) =>
          JsonUtil.fromJson[JWTHeader](decodedHeader)
          .map((_, contents))
      }
}

Thanks a lot! Will close the issue :+1:

Thanks a lot! Will close the issue :+1:

I've been banging my head on this same issue.  I can call the decodeRawAll method:
  def decodeRawAll(jwt: String): Try[(String, String, String)] = {
    Jwt
    .decodeRawAll(
      jwt,
      JwtOptions(signature = false, expiration = false, notBefore = false)
    )
  }

What I get back however, I can't seem to figure out how to get the claim that is in the Try..    When I log the Try, I see the data, the three Strings are wrapped in a Success.  But for the life of me, I can't get it from there..   Any other example code of doing this?
Eric

@epugh Check the Scaladoc for Try: https://www.scala-lang.org/api/current/scala/util/Try.html
You can probably use a pattern match to extract the data you need
  def decodeRawAll(jwt: String): Try[(String, String, String)] = {
    val decoded = Jwt
      .decodeRawAll(
        jwt,
        JwtOptions(signature = false, expiration = false, notBefore = false)
      )

    decoded match {
      case Success([header, claim, signature]) => do your stuff
      case Failure(e) => failed to decode
    }
  }

I'm not 100% sure about the syntax, it's been years since I last wrote some Scala.


Implement dateTrunc operator logic

This is a rather minimalistic implementation of the $dateTrunc operator 😅
Can you merge the develop branch into your topic branch? This should remove all the unrelated test failures that were caused due to #903


Let DataPackage viewer handle GitHub branches and directories

I haven't thoroughly tested this, but it seems the DataPackage Viewer only handles GitHub repositories where a) the repository = the datapackage, b) the datapackage lives on the master branch.
It would be nice if subdirectories were also supported, e.g. https://github.com/enram/data-challenge/tree/master/data/bird-migration-altitude-profiles
And alternative branches, to validate a data package before merging to master or to show an alternative version, e.g. https://github.com/peterdesmet/traffic-signs-hansbeke/tree/simple-data-format

FIXED. Done in https://github.com/okfn/datapackage-identifier/pull/2

Great! Is this already live at http://data.okfn.org/tools/view? I can't get the two examples listed above to work (yet).


Docs storage

Student, group: Кривцкий В.В. 3530901/90202
URL: https://github.com/OGSegu/Docs-storage
Git approach: git-flow

`+


fails sends with project defined address

I have enable the custom field 'project-sender-email'and set one email address in one project, Then I tested like below:
adding a news in this project, but the mail is still sent by redmine global address.
I am not sure if my understanding is right.

Hi, I am having the same issue with 3.2.1 but I will look into the code and see what I can do

Hi,
This doesnt seem to work with Redmine 3.2.2. Am I missing something? Or perhaps it is conflicting with another plugin?

It seems this conflicts with Event Notifications plugin. When this was disabled then redmine crashes on saving the issue with "Undefined Method 'ascii_only?'"

I've updated the plugin fixing this issue, please try it again updating to the latest commit. I've tested it with redmine 3.3-stable


python3.4 test failed on debian build in chroot

Hello.
I use this code for build deb package testing.posgresql:
debain/control:
#!/usr/bin/make -f
export PYBUILD_NAME=testing.common.database
%:
	dh $@ --with python2,python3 --buildsystem=pybuild

this is debian jessie
install dh_python, python3-nose, etc
nose test on python 2.7 finish ok
but nose test on python 3.4 failed:
/usr/lib/python3.4/unittest/case.py:605: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmprcvrgg1s/Postgresql.log' mode='wt' encoding='ANSI_X3.4-1968'>
  outcome.errors.clear()

Traceback (most recent call last):
  File "/tmp/buildd/testing.postgresql-1.3.0/tests/test_postgresql.py", line 142, in test_stop_on_child_process
    pgsql = testing.postgresql.Postgresql()
  File "/usr/lib/python3/dist-packages/testing/common/database.py", line 89, in __init__
    self.start()
  File "/usr/lib/python3/dist-packages/testing/common/database.py", line 136, in start
    os.dup2(logger.fileno(), sys.__stdout__.fileno())
ValueError: underlying buffer has been detached

maybe rewrite this code to subproccess?

Thank you for reporting.
I already rewrote it to subprocess in HEAD.
Could you retry with it?
If it works fine, I will release it as new version.

Amazing!
All test complete successful!
We are waiting for release. Thank you very much, and hello from Russia Ideco PBX.

Thank you for confirmation!
I'll release it in nearly future. (Sorry, I'm busy now. so I can't work for release a while)


cellular response to amino acid starvation missing parent

Cellular response to amino acid starvation should have the parent response to amino acid starvation.

Looks like a bunch of these kinds of terms need to be fixed.


marathon rpm does not allow running as non-root

for centos6 the script at /usr/bin/marathon is packaged with 750 permissions not allowing non-root users to execute it and the init script needs some modifications too to make marathon run as non-root user

Thanks for reporting this.
We except contributions: https://mesosphere.github.io/marathon/docs/contributing.html

where can i find the packaging scripts?

Here: https://github.com/mesosphere/marathon-pkg/

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3254. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3254. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


Added jsdoc

To src/geom/triangle/Contains.js and  src/renderer/webgl/Utils.js

Thank you for spending time creating this PR, however, these docs have already been completed in the master branch.


Cleanup laser pointer when dropped

When an active laser pointer is dropped (via Player:DropWeapon) it triggers a lot of errors clientside until the weapon is picked up or removed.
[wire] addons/wire/lua/weapons/laserpointer/cl_init.lua:48: attempt to call method 'GetEyeTrace' (a nil value)
  1. unknown - addons/wire/lua/weapons/laserpointer/cl_init.lua:48 (x67)

This error happens because the clientside code expects SWEP:GetOwner() to exist so it can perform a trace while drawing the beam, but after it's dropped, it won't have an owner.
My patch just adds some cleanup functionality on SWEP:OnDrop serverside, and sets an IsHeld parameters in shared, which the client then looks at to determine if it should draw the beam. Just setting the IsHeld should fix this error, so if the cleanup functionality is unwanted (could technically break some things, I guess? maybe?) I could remove it.
This has been tested in sandbox; confirmed that it doesn't error anymore
Another option which might be fun:
On drop, disconnect the laser pointer from all receivers, but keep it active, then modify the clientside code to say "if it has an owner, draw the beam between the owner and the owner's aimpos, but if there is no owner (dropped), draw the beam in a forward-ly direction."
That way a dropped, active laserpointer could sit on the ground with the laser still on.
I dunno, sounds fun to me. If that's preferable to y'all, I can implement that instead, but I figured just solving the error was a good first step.

I think it'd be better if it actually did a trace with the gun and continued to work.

Cool, I'll start experimenting with that tonight. Converting this to a draft until then


Add hg19,hg38 as a reference genome choice

I want to edit the reference genomes in 567 samples from hg19 to hg19,hg38. The drop down menu does not have that as a choice. Can hg19,hg38 be added to the menu?

The short answer is "yes", see screenshot. The longer answer though points back to why this field is in PEGR, it is meant to provide the reference genome URL to be used for mapping in the pipeline. Are you asking if it would be possible to make the pipeline map to both of those reference genomes? Just trying to understand the intent here. And then what would be included as the URL to the reference file?

We did this for the Lis lab in run 557. They had human cells spiked into mouse samples. On the Queue sheet I listed it as mm10,hg38. It uploaded and ran against both genomes. Will wants to run Hainings samples I submitted this morning against both hg18 and hg38. When I went in to PEGR to edit the Queue, hg19,hg38 doesn't exist. I don't know what URL would be included to the reference file. Does that clarify what I am looking to do?

From: Gretta D Kellogg @.>
Sent: Thursday, April 14, 2022 3:10 PM
To: seqcode/pegr @.>
Cc: Anne Gardner @.>; Author @.>
Subject: Re: [seqcode/pegr] Add hg19,hg38 as a reference genome choice (Issue #255)
The short answer is "yes", see screenshot. The longer answer though points back to why this field is in PEGR, it is meant to provide the reference genome URL to be used for mapping in the pipeline. Are you asking if it would be possible to make the pipeline map to both of those reference genomes? Just trying to understand the intent here. And then what would be included as the URL to the reference file?
[image]https://user-images.githubusercontent.com/19175816/163459865-7fd1aa2f-ea61-4ec7-bd38-cb1197a5cb52.png
—
Reply to this email directly, view it on GitHubhttps://github.com/seqcode/pegr/issues/255#issuecomment-1099540756, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVUUSX2MMZOQSB7UPO4AZR3VFBUTZANCNFSM5TOP4OTQ.
You are receiving this because you authored the thread.Message ID: @.***>

Yes! That helps a lot. Thanks for clarifying. I need to convene with Will and Danying before recommending next steps. Short answer though is that we can certainly create those options for annotating the queue submissions.

In the queue form, it is ok to add the comma-delimited genome builds that need to be run per sample. So you can specify multiple genomes.


Audio play even when phone sound is set to MUTE

I am using the plugin inside my ionic app.
The sound plays correctly.
The problem is that when the sound setting in the phone is set to MUTE - It plays anyway.
The Vibration plugin for example don't start if the settings are on MUTE.
Do you think it can be fixed in the plugin?
Thanks!

MartinDomig had and solved this problem, too. There is a solution, detailed here.
In short: until the maintainers of this project fix this, try using MartinDomig forked version: https://github.com/MartinDomig/cordova-plugin-nativeaudio

This is for anyone else who has this issue.  Here is how I fixed it.
In NativeAudio.m line 10 replace #import <AVFoundation/AVAudioSession.h> with #import <AVFoundation/AVFoundation.h>
Line 48 replace [session setCategory:AVAudioSessionCategoryPlayback error:nil]; with [session setCategory:AVAudioSessionCategoryAmbient error:nil];
This worked perfect for me.  You can also remove line 32 to prevent a depreciation warning.

This is for anyone else who has this issue. Here is how I fixed it.
In NativeAudio.m line 10 replace #import <AVFoundation/AVAudioSession.h> with #import <AVFoundation/AVFoundation.h>
Line 48 replace [session setCategory:AVAudioSessionCategoryPlayback error:nil]; with [session setCategory:AVAudioSessionCategoryAmbient error:nil];
This worked perfect for me. You can also remove line 32 to prevent a depreciation warning.

Hello,
I have tried this, but still no sound when in silent mode :(

We are now maintaining a forked version (https://github.com/wizpanda/cordova-plugin-nativeaudio) of this plugin which fixes this issue.

@sagrawal31 does you fork also have a fix for Android?

Yes @soulfly our fork fixes this issue for Android

Yes @soulfly our fork fixes this issue for Android

@sagrawal31 tried your fork but I'm still getting sound when my device is in mute or vibrate (Android)


Tooltip Theme doesn's work

V3.0.2 stable
I create a Button and add a tooltip.
add a theme contains TooltipLabel to the button.
set a red color
but nothing happened!

Can't reproduce this in master.

This is fixed in 3.1.

Closing as fixed on 3.1 then.


[e2e]: go tag main config-ldflags-assets-tag slsa3

Repo: https://github.com/slsa-framework/example-package/tree/v20.0.75
Run: https://github.com/slsa-framework/example-package/actions/runs/2829727888
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-assets-tag.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-assets-tag.slsa3.yml
Trigger: push
Branch: v20.0.75
Date: Wed Aug 10 03:08:48 UTC 2022

Repo: https://github.com/slsa-framework/example-package/tree/v20.0.76
Run: https://github.com/slsa-framework/example-package/actions/runs/2837132028
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.tag.main.config-ldflags-assets-tag.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.tag.main.config-ldflags-assets-tag.slsa3.yml
Trigger: push
Branch: v20.0.76
Date: Thu Aug 11 03:23:27 UTC 2022
Tests are passing now. Closing this issue.


MD formatting appears to be messed up currently in master

Example: heading markdown is rendering as octothorp literals

I believe this has been fixed now so I'm closing it.


Method 1 & Step 3

seems like following configuration does not work.  "charts/webfrontend/values.dev.yaml"
I need to change values.yaml file contents instead of values.dev.yaml file.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 94dbff82-2dd2-7ded-6d0b-d685ea705cdf
Version Independent ID: 2ccb6f01-40c1-a096-1214-8ff938c0978a
Content: How to manage secrets when working with an Azure Dev Space - Azure Dev Spaces
Content Source: articles/dev-spaces/how-to/manage-secrets.md
Service: azure-dev-spaces
Product: unspecified
GitHub Login: @ghogen
Microsoft Alias: ghogen

@junpark12 Thank you for bringing this to our attention. Your feedback has been shared with the content author for further review.

@junpark12 Is this still an issue?  I think earlier versions of the tools didn't support values.dev.yaml but I think more current ones should.

#please-close
If still an issue with current tools and docs, please reopen.

@junpark12 We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


[Suggestion] Partial replacement of vanilla trees

Would it be possible to have a config that allows for a certain percentage or number per chunk replacement of vanilla trees rather than all? This would accomplish a couple of things:
First, a bit better performance on potato PCs but still a nice looking world gen.
Second, the ability to really see the forests spread as starting with a lesser number of dynamic trees would make it more obvious.
This mod is a breathtaking and beautiful addition to Minecraft. Amazing job!

Due to the difference of the generation that vanilla uses and that which this mod implements, I don't think that's possible. Hold for a reply from the author though.
What you can do on potato pc's is not enable replacing vanilla spawn trough config and then craft and plant the DynamicTree "saplings" manually.


Update Canal RBAC manifest: Removed calico-node ClusterRoleBinding

Fix type: Manifest fix for Canal install
Details: Removes unnecessary ClusterRoleBinding relating to calico-node ServiceAccount
Todos

[x] Tests (N/A)
[x] Documentation
[x] Release note

Release Note
None required

I don't believe there's any specific tests to update here or whether a release note is required, as it's just updating the RBAC manifest to drop a CRB that isn't actually applicable. Please correct me if I'm wrong though!

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Deploy preview for calico ready!
Built with commit 368551b2f3fc31bdc55e2e0e26b0e8a477233c34
https://deploy-preview-2569--calico.netlify.com

This is incorrect.  A canal system is flannel for networking + Calico for policy.  Calico still needs datastore access to implement policy.

Hey @fasaxc, the CRB that is dropped with this change references the calico-node ServiceAccount, however with the Canal deployment the SA used in the deployment is canal, and a relevant CRB is still added for this (unless I've misunderstood):
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: canal-calico
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: calico
subjects:
- kind: ServiceAccount
  name: canal
  namespace: kube-system

Hmm, maybe I'm wrong then!  Probably need a second opinion from someone like @caseydavenport

Yep, I think this looks good - that CRB doesn't appear to be in use.
@KashifSaadat sorry for the long delay on this.. would you mind also porting these changes to _includes/master? That way they will get picked up in future releases.

No worries :) Thanks @caseydavenport, I've just included the update as suggested and also rebased off master.


[Issue] Damage Log empty

Foundry v11.315
PF1e v10.4
Damage log remains empty when applying damage or healing.  Tested with just damage log & libwrapper activated.
Added/removed health through chatcards and manually, to no effect.
[Edit] no errors in console

Same issue after updating to PF1e v10.4 - running on The Forge

5053fad18a1ebb387590d8039537e9a9a148ccd5

5053fad18a1ebb387590d8039537e9a9a148ccd5

5053fad18a1ebb387590d8039537e9a9a148ccd5

Now fixed in v1.21.1


docs: Backport #122 to 8.1

Backports #122 to 8.1.

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-03-01T19:26:33.721+0000

Duration: 2 min 13 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
84

Skipped
0

Total
84

:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)


Legato error values in pressure converted to depth

The legato pressure sensor is used to calculate glider depth. When the sensor is switched off/malfunctioning, the seaexplorer reports a value of 9999 for pressure. This is converted to a depth of 9690 m by utils.get_glider_depth. While 9999 is a fairly obvious error value, 9690 is not.
This function has a filter step for "good" pressures which it later uses to interpolate over the bad depths. Should we exclude pressure ratings greater than the maximum pressure from the "good" subset of pressures used to calculate depth perhaps?

Proposal: We should catch 9999 data points and flag them. Perhaps convert them to nan? Otherwise they can be processed to other numbers that are not obvious error codes

Resolved in #143


BBQ - Meater+ wireless temperature probe

Meater+ wireless temperature probe
Found this project that has reverse-engineered the BLE communication for the Meater BBQ thermometer which I plan to contribute to or fork and add MQTT to integrate it with Home Assistant. My longer goal would be to create an integration for ESPHome to work with it.
Originally posted by @jcallaghan in https://github.com/jcallaghan/home-assistant-config/issues/21#issuecomment-625509945

Cloned this project down onto an old Raspberry Pi to give it a try.
git clone https://github.com/nathanfaber/meaterble.git
cd /meaterble/

When I run ./scan.sh it is able to see my device. It discovers the probe and the Meater+ block.
pi@raspberrypi:~/meaterble $ ./scan.sh
B8:1F:5E:00:A6:00 MEATER
B8:1F:5E:00:A6:00 MEATER
B8:1F:5E:06:6B:5B MEATER+
B8:1F:5E:06:6B:5B MEATER+

I'm getting an error when trying to run the ./readMeater.py B8:1F:5E:06:6B:5B script an error is throw. This might be down to it being an old image?!?

After removing the power to the Meater+ block I was able to connect using the following command.
sudo python ./readMeater.py B8:1F:5E:00:A6:00

I modified the readMeater python script to include the Paho MQTT client and am now able to broadcast the Meater tip and ambient temperature as well as the battery.

Shared on Twitter include a video of Home Assistant integration.

I publish the following MQTT topics when the Meater probe updates.
meater/0/ambient
meater/0/battery
meater/0/firmware
meater/0/last_updated
meater/0/mac
meater/0/probe_id
meater/0/tip

This is awesome @jcallaghan. really cool. I had to return my Meater as I wanted at a Meater+. So waiting for that. When it arrives I will definetly try out your code and let you know how it goes.

Inside the Meater+ block

Meater python connection script
I've created a script that will continuously load the readMeater.py script from the nathanfaber/meaterble project using this example. This allows me to continue to look for my Meater probe based on it's MAC address. If the Meater script errors or has exceptions like disconnects my script will just reload it. Additional to this I have also created a service so it loads on reboot etc.
I am going to read up on bluepy some more to see if I can build exception handling into the functions rather than rely on reopening the script but for now, this will do. My main goal is to refactor the python and get it working as a component on ESPHome.
Errors and exceptions I observed include:

IndexError: bytearray index out of range
bluepy.btle.BTLEDisconnectError: Device disconnected
bluepy.btle.BTLEDisconnectError: Failed to connect to peripheral B8:1F:5E:00:A6:00, addr type: public

Now I can run python meater_connect.py B8:1F:5E:00:A6:00 with my device address or use the service to run this script. The meaterble project includes a scan.sh which provides the address of Meater probes and blocks.
meater_connect.py
#!/usr/bin/python
from subprocess import Popen
import sys

device = filename = sys.argv[1]
while True:
    print("\nStarting Meater connection to " + device + ".....................")
    p = Popen("python ./readMeater.py " + device, shell=True)
    p.wait()

meater_mqtt.service
I created the following service.
[Unit]
Description=Meater Thermometer Monitor
After=network.target

[Service]
Type=simple
Restart=always
RestartSec=1
User=pi
WorkingDirectory=/home/pi/meaterble
ExecStart=/usr/bin/python3 meater_connect.py B8:1F:5E:00:A6:00

#StandardOutput=syslogConsola               # Output to syslog
#StandardError=syslogConsola                # Output to syslog

[Install]
WantedBy=multi-user.target

To install the service use the following commands to copy the service to the system folder, enable the service and start it.
sudo cp meater_mqtt.service /etc/systemd/system
sudo systemctl enable meater_mqtt.service
sudo systemctl start meater_mqtt.service

You can check on the status of the service with systemctl status meater_mqtt.service.
meater.py
I updated the following in the meater.py to broadcast the information over MQTT.
def __str__(self):
       import paho.mqtt.client as mqtt
	   client = mqtt.Client()
       client.connect("<IP_ADDRESS>", 1883)
	   client.loop_start()
	   mqtt_topic = "meater/"

	   client.publish(mqtt_topic + self.getID() + "/firmware", self.getFirmware())
       client.publish(mqtt_topic + self.getID() + "/last_updated", self._lastUpdate)
       client.publish(mqtt_topic + self.getID() + "/mac", self.getAddress())
       client.publish(mqtt_topic + self.getID() + "/probe_id", self.getID())
       client.publish(mqtt_topic + self.getID() + "/tip", round(self.getTipC(),2))
       return "%s %s probe: %s tip: %fF/%fC ambient: %fF/%fC battery: %d%% age: %ds" % (self.getAddress(), self.getFirmware(), self.getID(), self.getTipF(),self.getTipC(), self.getAmbientF(), self.getAmbientC(), self.getBattery(),time.time() - self._lastUpdate)

Update
Contributing to the original project. The updates I've made to nathanfaber/meaterble are working really well. Nathan has added me as a contributor to his project so I will work with him to include the changes I have made and share the MQTT work within his project.
Meater+ block power switch. Having taken the Meater+ block apart I now know where I can place a slide switch to turn the unit off without having to take the battery out each time I want my Home Assistant integration to work.
ESPHome integration. I have found some reference integrations I am hoping I will be able to use to get started with an integration for ESPHome.

Target temperature
I created a input_number sensor to maintain the target temperature for my Meater probe. While cooking dinner this evening I gave it a quick test. It is great to see this real-time in the history panel.

Update
Conversation with the Meater team.
Meater Block

Shared this in an existing Meater conversation in the Home Assistant community.

Meater MQTT
Seems it is possible to connect to the Meater MQTT service.
mosquitto_sub -h cloud.meater.com -p 8883 -u 'my-email-address-' -P 'my-meater-password' -t 'MEATERCloud/Devices/some-id-I-dont-know/MASTER' --verbose -d --capath /etc/ssl/certs/

Hi nice work indeed...would to try to integrate BLE meater probe in my BBQ charcoal blow fan temperature controller
where can I find the modified the readMeater.py python script with MQTT support
In the orginal meaterble.git seems not available
thanks
Davide

Any updates on this, now that Bluetooth is an integrated part of Home Assistant. Are there any plans to get this supported by the bluetooth proxy feature?

Hey, @jcallaghan any update on making this a esphome integration? I don't want to use a cloud version and only have a meater. Also don't want to use my phone as the middle man. Let me know if I can help!


Null check LocationUtil method parameters

Fixes issue #14

I don't really think this is needed - the Issue you referenced is caused when a Hologram is loaded but the world it is defined in does not exist


Implement $ function prefix to prevent autovectorization

Inspired by the @. macro, which includes a feature like this, I will add a $ prefix for functions that will prevent them from being auto-vectorized. For example, if you define a function called new_mean which calculates a mean, it would currently get auto-vectorized because it wouldn't get be picked by the current exclusion list.
After resolving this issue, referring to it as $new_mean in the code will prevent it from being vectorized.

Instead of using $, which invokes Julia interpolation, we will use a ~ prefix to denote functions that should not be vectorized.


Add possibility to generate server/peer/client cert based on condition

Example:
# defaults/main.yml
etcd_cert_generate_server: true

# tasks/main.yml
- name: Generate client-to-server (server) certificates
  ansible.builtin.include_tasks:
    file: generate-server.yml
  when: etcd_cert_generate_server | bool == true

Decided to use another scheme: pass input matrix with all required options to generate any certificate pair


Take author of the squashed changelog from the original change

Since the name is no longer taken from Git, we don't take the
date from git either. The mechanism to override the date (which
is there mainly for the tests) is changed to $CURRENT_DATE
(with a stricter format than what Git takes for AUTHOR_DATE).
Fixes: https://github.com/encukou/rpm-spec-merge-driver/issues/3

@hroncok, mind taking a look?


Unable to connect

Describe the bug
Upon upgrading to the latest version and after numerous restarts, I keep getting an error about unable to connect with a timeout error. Interestingly, the Gas stats seem to work. It's just for Electric.
Reproduction steps
Upgrade from an older Octopus version to the latest.
Expected behaviour
I expect it to work as before.
Tariff Code
Unknown
Integration Version
Latest
Home Assistant Version
Latest
Fresh Install?
After upgrading
Home Assistant Logs
2023-12-31 19:39:05.705 WARNING (MainThread) [custom_components.octopus_energy.api_client] Failed to connect. Timeout of ClientTimeout(total=15, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5) exceeded.
2023-12-31 19:39:05.706 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry A-0000C00E for octopus_energy
Traceback (most recent call last):
  File "/config/custom_components/octopus_energy/api_client/__init__.py", line 393, in async_refresh_token
    async with client.post(url, json=payload) as token_response:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 601, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 960, in start
    with self._timer:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/octopus_energy/__init__.py", line 83, in async_setup_entry
    await async_setup_dependencies(hass, config)
  File "/config/custom_components/octopus_energy/__init__.py", line 128, in async_setup_dependencies
    account_info = await client.async_get_account(config[CONFIG_MAIN_ACCOUNT_ID])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/octopus_energy/api_client/__init__.py", line 411, in async_get_account
    await self.async_refresh_token()
  File "/config/custom_components/octopus_energy/api_client/__init__.py", line 407, in async_refresh_token
    raise TimeoutException()
custom_components.octopus_energy.api_client.TimeoutException

Also I get the two lines:
2023-12-31 20:01:38.636 WARNING (MainThread) [custom_components.octopus_energy.coordinators.intelligent_dispatches] Failed to retrieve new dispatches. Next attempt at 2023-12-31 20:02:38.635948+00:00
2023-12-31 20:01:38.638 WARNING (MainThread) [custom_components.octopus_energy.coordinators.intelligent_settings] Failed to retrieve new intelligent settings. Next attempt at 2023-12-31 20:02:38.637731+00:00

Confirmation

[X] I confirm that I cannot find my solution within the documentation
[X] I confirm that I cannot find my solution within the FAQ

Im also getting no connection to octopus energy/home mini, it keeps saying failed to set-up.
Numerous restarts and reloads, but nothing. Octopus live app is working and receiving data from my home mini

Ive tried re-installing the intergration and nothing works. I have no access to my octopus data at all.

I've even tried rolling back HA Core as I updated to the latest version of that too. Still the same issue.

I'm seeing the same after updating yesterday.
`Logger: homeassistant.config_entries
Source: config_entries.py:402
First occurred: 15:01:33 (1 occurrences)
Last logged: 15:01:33
Error setting up entry A-C75F2105 for octopus_energy
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/init.py", line 83, in async_setup_entry
await async_setup_dependencies(hass, config)
File "/config/custom_components/octopus_energy/init.py", line 156, in async_setup_dependencies
await async_setup_electricity_rates_coordinator(hass, mpan, serial_number, is_smart_meter, is_export_meter)
File "/config/custom_components/octopus_energy/coordinators/electricity_rates.py", line 149, in async_setup_electricity_rates_coordinator
await hass.data[DOMAIN][coordinator_key].async_config_entry_first_refresh()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 265, in async_config_entry_first_refresh
await self._async_refresh(
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh
self.data = await self._async_update_data()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 256, in _async_update_data
return await self.update_method()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/coordinators/electricity_rates.py", line 122, in async_update_electricity_rates_data
hass.data[DOMAIN][key] = await async_refresh_electricity_rates_data(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/coordinators/electricity_rates.py", line 63, in async_refresh_electricity_rates_data
new_rates = await client.async_get_electricity_rates(tariff_code, is_smart_meter, period_from, period_to)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/api_client/init.py", line 708, in async_get_electricity_rates
return await self.async_get_electricity_standard_rates(product_code, tariff_code, period_from, period_to)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/api_client/init.py", line 642, in async_get_electricity_standard_rates
async with client.get(url, auth=auth) as response:
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 601, in _request
await resp.start(conn)
File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 965, in start
message, payload = await protocol.read()  # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 622, in read
await self._waiter
asyncio.exceptions.CancelledError
`

Still nothing for me……i sit in hope

I was seeing the same issue, but it seems to be working now? I wonder if a problem on the Octopus side?

@theneiljohnson It has to be, I've even did a downgrade to 9.0.2 and still no result.

 Even if i call it a couple times in quick succession, so I doubt it's a throttling issue.

Hmm, think i spoke to soon. I flicked debugging on to grab logs and then it started ok. I then switched debugging off and it started fine again. Ran for about 15 minutes and now its failed again.

Hmm, think i spoke to soon. I flicked debugging on to grab logs and then it started ok. I then switched debugging off and it started fine again. Ran for about 15 minutes and now its failed again.

@theneiljohnson does your logs contain a similar error message to mine?

Yes, pretty similar:
Traceback (most recent call last):
File "/config/custom_components/octopus_energy/api_client/init.py", line 393, in async_refresh_token
async with client.post(url, json=payload) as token_response:
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 601, in _request
await resp.start(conn)
File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 960, in start
with self._timer:
File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in exit
raise asyncio.TimeoutError from None
TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/init.py", line 83, in async_setup_entry
await async_setup_dependencies(hass, config)
File "/config/custom_components/octopus_energy/init.py", line 128, in async_setup_dependencies
account_info = await client.async_get_account(config[CONFIG_MAIN_ACCOUNT_ID])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/octopus_energy/api_client/init.py", line 411, in async_get_account
await self.async_refresh_token()
File "/config/custom_components/octopus_energy/api_client/init.py", line 407, in async_refresh_token
raise TimeoutException()
custom_components.octopus_energy.api_client.TimeoutException
Debug logs:
2024-01-01 01:31:07.443 WARNING (MainThread) [custom_components.octopus_energy.api_client] Failed to connect. Timeout of ClientTimeout(total=15, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5) exceeded.
2024-01-01 01:31:07.444 WARNING (MainThread) [custom_components.octopus_energy.coordinators.electricity_rates] Failed to retrieve new electricity rates for<PHONE_NUMBER>708/22J0045738 - using cached rates. Next attempt at 2024-01-01 01:30:51.913366+00:00
2024-01-01 02:00:59.444 WARNING (MainThread) [custom_components.octopus_energy.api_client] Failed to connect. Timeout of ClientTimeout(total=15, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5) exceeded.
2024-01-01 02:00:59.444 WARNING (MainThread) [custom_components.octopus_energy.coordinators.intelligent_settings] Failed to retrieve new intelligent settings - using cached settings. Next attempt at 2024-01-01 02:00:43.536510+00:00

@theneiljohnson check the logs with debugging enabled and reload the integration entity thats your account number.
I've noticed that my meters array is empty. After checking the Octopus app, I can't see any Electricity graphs which is extremely weird.
The log should look something like this:
[custom_components.octopus_energy.api_client] account:

I'm seeing similar in the app, I can see import usage for today but export and everything is is blank. My Home Mini data is coming through but inconsistently, seems to work fine for 5 minutes or so then reports that it's no longer being reported, then comes back again without me doing anything.
My gut feeling is that there is an issue on the Octopus side, I'd imagine some young project manager somewhere thought that making system upgrades on New Years Day was a super idea :-)

@theneiljohnson Yeah my home mini isn't coming through at all.
I've sent them an email - phone lines are unsurprisingly closed today.
I suspect something has broken, I noticed my stats hasn't been showing in the app for a few days. Extremely weird. I guess all I can do is wait to see what they say.

If it helps, i've just created this automation to try and restart the integration if it's unavailable. I initially did it based on the state change, but then it doesn't retry later if it fails on first restart.  This checks the integration state every 15 minutes and reloads if req'd.
description: Force restart of Octopus Integration if it becomes unavailable.
trigger:
  - alias: Check Octopus Integration State every 15 minutes
    platform: time_pattern
    minutes: /15
condition:
  - condition: state
    entity_id: binary_sensor.octopus_energy_a_cXXXXXX_intelligent_dispatching
    state: unavailable
    alias: Check to see if Intelligent Dispatching is Unavailable
action:
  - service: homeassistant.reload_config_entry
    target:
      entity_id: binary_sensor.octopus_energy_a_cXXXXXX_intelligent_dispatching
    data: {}
    alias: Reload Octopus Integration if Intelligent Dispatching is Unavailable
mode: restart```

If it helps, i've just created this automation to try and restart the integration if it's unavailable. I initially did it based on the state change, but then it doesn't retry later if it fails on first restart. This checks the integration state every 15 minutes and reloads if req'd.
description: Force restart of Octopus Integration if it becomes unavailable.
trigger:
  - alias: Check Octopus Integration State every 15 minutes
    platform: time_pattern
    minutes: /15
condition:
  - condition: state
    entity_id: binary_sensor.octopus_energy_a_cXXXXXX_intelligent_dispatching
    state: unavailable
    alias: Check to see if Intelligent Dispatching is Unavailable
action:
  - service: homeassistant.reload_config_entry
    target:
      entity_id: binary_sensor.octopus_energy_a_cXXXXXX_intelligent_dispatching
    data: {}
    alias: Reload Octopus Integration if Intelligent Dispatching is Unavailable
mode: restart

Awesome thank you for that!

Hello. Sorry for the late response. Timeout exceptions unfortunately are something that can't be fixed on my end other than increasing the timeout, but this doesn't guarantee success and results in HA reporting slow integration warnings which I then have issues raised against too (lose lose situation :))
The integration uses a variety of octopus APIs, both rest and graphql (the latter being more fully fledged) and I'm assuming they're using a microservice architecture. This means that one of the API commands can be functional while other parts aren't.
Based on reports in the OE forums it looks like other people had various API issues.
A future update will reduce the hit on the API during startup, which will hopefully help make it more stable.

My mini still wont connect to HA. All connected to the router and octo app no problems. But gets an unknown error when trying to connect to HA..
API is all working ok for other apps im using with octopus to.

Do you have any errors in your HA logs that might point at the issue (full details please)? If you're using the app and other apps and this integration to access your mini you might be hitting rate limits on the API and you'll need to adjust the frequency of the update in the integration.

I tried reducing the refresh rate, but it just wont even load into HA at all. It just says unknown error.
At the moment its not even intergrating into HA

Can you please submit your full HA logs or screenshots of the error

As i said its not logging any error…….this is the only error i see

Have you updated to the latest version of the integration? Even with this unknown error I would expect some errors or warnings in the HA logs.
Failing that, if you could set the logs of the integration to debug level (see below), try again and then look in your full logs for account:
# Example configuration.yaml entry
logger:
  default: warning
  logs:
    custom_components.octopus_energy: debug

Im new to all this dave and dont know were to paste that code…..i dont have file server intergrated yet.
Sorry

You would need to add it to your configuraion.yaml file. This can be done via standard HA setup if you install the File editor add-on. Once done, you need to restart your HA instance for the logging to take effect.
Alternatively, you can run the logging service in developer tools -> services and paste the following in yaml mode and execute the service which should set your log level. You shouldn't need to restart your HA instance this way.
service: logger.set_level
data:
  custom_components.octopus_energy: debug

Ok…i will get that done over the next few days and get back to you.
Thanks

After numerous HA updates and a few octopus updates, its now all working again.

Sorry for my late response - I contacted Octopus after I realised my Smart Meter magically disappeared from my account. After they re-added it, I was able to see my statistics again like normal. I can only imagine it was because the MPAN wasn't there, so the plugin had no way of requesting information for it.
I guess everyone here has it working again one way or another, so I'll close this issue now since it seems resolved. Thanks everyone for the help!


feat(admin_api): support map brackets syntax

Note for reviewers: use hide whitespace if reviewing from gh
Summary
This commit adds support for configuring maps using brackets syntax to
enclose the key:
http -f post :8001/plugins/                       \
name=opentelemetry                                \
config.endpoint=http://test.test/                 \
config.resource_attributes[service.name]=kong-dev

Checklist

[x] The Pull Request has tests
[x] A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
[x] (no) There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference
KAG-4827

@nowNick my initial objective with this PR was to keep the scope as small as possible, that is why you noticed similarities with existing code, but I have to say that I also agree all your suggestions are improvements. At the same time I don't know how we feel about adding a large refactor on top of this PR, so in the end I did pretty much everything you said (with some  additions as I'm about to answer on individual comments) but kept it as a separate commit so we can decide to split it out if needed.
I think this is good because the splitting of things resulted in a lot of new unit tests and some more order IMO.