Clarify Scorecard requirements in docs

We've recently been exploring the use of OpenSSF Scorecard Monitor for our various open source organizations, but hit a roadbump after finding that Scorecard Monitor was only discovering a handful of repositories in our orgs (regardless of which projects were listed in scope.json).
After some investigation, I realized that OpenSSF was automatically tracking a select group of our projects upstream, which is why some projects already scores and others didn't. I'd like to update the README docs to clarify these requirements, and provide some hints in case others experience similar issues.

I am super interested in getting feedback about the tool also in providing support to extend the adoption in open source organizations. Let me know if you are available for a quick chat 🙂

I am super interested in getting feedback about the tool also in providing support to extend the adoption in open source organizations. Let me know if you are available for a quick chat 🙂

@UlisesGascon That would be great! I'll try pinging you on OpenSSF Slack and we can go from there 🙌

Great! Thanks a lot :)


Incorrect LEFT JOIN evaluation result on OR conditions

Describe the bug
When the matching condition is on columns and combined with OR, the result retuned by LEFT JOIN is similar to INNER JOIN, which is incorrect.
LEFT JOIN should not reduce the number of rows returned, i.e. it should return all the records from the left table, and the matched records from the right table. When there is no match, the result is a NULL, i.e. display as empty.
INNER JOIN reduces the number of rows, i.e. it only returns the records that have matching values in both tables.
To Reproduce

Use DataFusion CLI:

# in dir datafusion/datafusion-cli
$ cargo build
$ ./target/debug/datafusion-cli

Write sample data

CREATE OR REPLACE TABLE employees(emp_id INT, name VARCHAR) AS VALUES (1, 'Alice'), (2, 'Bob'), (3, 'Carol');
CREATE OR REPLACE TABLE department(emp_id INT, department VARCHAR) AS VALUES (1, 'HR'), (3, 'Engineering'), (4, 'Sales');

This is how employees table and department table look like

> SELECT * FROM employees;
+--------+-------+
| emp_id | name  |
+--------+-------+
| 1      | Alice |
| 2      | Bob   |
| 3      | Carol |
+--------+-------+
3 row(s) fetched.
Elapsed 0.009 seconds.

> SELECT * FROM department;
+--------+-------------+
| emp_id | department  |
+--------+-------------+
| 1      | HR          |
| 3      | Engineering |
| 4      | Sales       |
+--------+-------------+
3 row(s) fetched.
Elapsed 0.006 seconds.

Query

> SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'Bob');
+--------+-------+-------------+
| emp_id | name  | department  |
+--------+-------+-------------+
| 1      | Alice | HR          |
| 1      | Alice | Engineering |
| 1      | Alice | Sales       |
| 2      | Bob   | HR          |
| 2      | Bob   | Engineering |
| 2      | Bob   | Sales       |
+--------+-------+-------------+ <-- should have one more row for Carol
6 row(s) fetched.
Elapsed 0.013 seconds.

> SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'NotExist1' OR e.name = 'NotExist2');
+--------+------+------------+
| emp_id | name | department |
+--------+------+------------+
+--------+------+------------+ <-- should have three rows, 1 row for Alice, 1 row for Bob, and 1 row for Carol
0 row(s) fetched.
Elapsed 0.014 seconds.

> SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'NotExist');
+--------+-------+-------------+
| emp_id | name  | department  |
+--------+-------+-------------+
| 1      | Alice | HR          |
| 1      | Alice | Engineering |
| 1      | Alice | Sales       |
+--------+-------+-------------+ <-- should have two more rows, 1 row for Bob and 1 row for Carol
3 row(s) fetched.
Elapsed 0.014 seconds.

Expected behavior
Postgres shows the expected results.

In psql cli, drop tables and create new tables with data

DROP TABLE IF EXISTS employees;
DROP TABLE IF EXISTS department;

# create tables
CREATE TABLE employees(emp_id INT, name VARCHAR);
CREATE TABLE department(emp_id INT, dept_name VARCHAR);

# write sample data
INSERT INTO employees (emp_id, name) VALUES (1, 'Alice'), (2, 'Bob'), (3, 'Carol');
INSERT INTO department (emp_id, dept_name) VALUES (1, 'HR'), (3, 'Engineering'), (4, 'Sales');

This is how two tables look like:

chunchun=# SELECT * FROM employees;
 emp_id | name
--------+-------
      1 | Alice
      2 | Bob
      3 | Carol
(3 rows)

chunchun=# SELECT * FROM department;
 emp_id |  dept_name
--------+-------------
      1 | HR
      3 | Engineering
      4 | Sales
(3 rows)

Query

SELECT e.emp_id, e.name, d.department
FROM employees AS e
LEFT JOIN department AS d
ON (e.name = 'Alice' OR e.name = 'Bob');
 emp_id | name  | department
--------+-------+-------------
      1 | Alice | HR
      1 | Alice | Engineering
      1 | Alice | Sales
      2 | Bob   | HR
      2 | Bob   | Engineering
      2 | Bob   | Sales
      3 | Carol |
(7 rows)

SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'NotExist1' OR e.name = 'NotExist2');
 emp_id | name  | department
--------+-------+------------
      1 | Alice |
      2 | Bob   |
      3 | Carol |
(3 rows)

SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'NotExist');
 emp_id | name  | department
--------+-------+-------------
      1 | Alice | HR
      1 | Alice | Engineering
      1 | Alice | Sales
      2 | Bob   |
      3 | Carol |
(5 rows)

Additional context
No response

SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'Bob');

Hmm, as the join filter is only on employees, I think it is pushdown through the Join. The results seem correct to me. I will verify it on Spark later.

Yea, just verified with Spark:
scala> df.show
+------+-----+-----------+
|emp_id| name|  dept_name|
+------+-----+-----------+
|     1|Alice|      Sales|
|     1|Alice|         HR|
|     1|Alice|Engineering|
|     2|  Bob|      Sales|
|     2|  Bob|         HR|
|     2|  Bob|Engineering|
|     3|Carol|       null|
+------+-----+-----------+

This will cause correctness issue on Comet too. I will take a look.

Could this be related to a bug in the optimizer (pushing down filter to the left where it shouldn't)? Maybe it has nothing to do with the join impl itself at all.

If we look at the following plan:

> EXPLAIN SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'Bob');
+---------------+----------------------------------------------------------------------------+
| plan_type     | plan                                                                       |
+---------------+----------------------------------------------------------------------------+
| logical_plan  | Left Join:  Filter: e.name = Utf8("Alice") OR e.name = Utf8("Bob")         |
|               |   SubqueryAlias: e                                                         |
|               |     Filter: employees.name = Utf8("Alice") OR employees.name = Utf8("Bob") |
|               |       TableScan: employees projection=[emp_id, name]                       |
|               |   SubqueryAlias: d                                                         |
|               |     TableScan: department projection=[department]                          |
| physical_plan | NestedLoopJoinExec: join_type=Left, filter=name@0 = Alice OR name@0 = Bob  |
|               |   CoalesceBatchesExec: target_batch_size=8192                              |
|               |     FilterExec: name@1 = Alice OR name@1 = Bob                             |
|               |       MemoryExec: partitions=1, partition_sizes=[1]                        |
|               |   MemoryExec: partitions=1, partition_sizes=[1]                            |
|               |                                                                            |
+---------------+----------------------------------------------------------------------------+

The filters in italics seem wrong. They indeed disappear in the example I gave above:

> EXPLAIN SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'Bob' OR d.department = 'FOOBAR');
+---------------+-------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+-------------------------------------------------------------------------------------------------------+
| logical_plan  | Left Join:  Filter: e.name = Utf8("Alice") OR e.name = Utf8("Bob") OR d.department = Utf8("FOOBAR")   |
|               |   SubqueryAlias: e                                                                                |
|               |     TableScan: employees projection=[emp_id, name]                                                    |
|               |   SubqueryAlias: d                                                                                |
|               |     TableScan: department projection=[department]                                                     |
| physical_plan | ProjectionExec: expr=[emp_id@1 as emp_id, name@2 as name, department@0 as department]                 |
|               |   NestedLoopJoinExec: join_type=Right, filter=name@0 = Alice OR name@0 = Bob OR department@1 = FOOBAR |
|               |     MemoryExec: partitions=1, partition_sizes=[1]                                                     |
|               |     MemoryExec: partitions=1, partition_sizes=[1]                                                     |
|               |                                                                                |
+---------------+-------------------------------------------------------------------------------------------------------+
2 row(s) fetched.
Elapsed 0.016 seconds.

These filters are also not present if have an AND clause instead:

> EXPLAIN SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' AND e.name = 'Bob');
+---------------+---------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+---------------------------------------------------------------------------------------+
| logical_plan  | Left Join:  Filter: e.name = Utf8("Alice") AND e.name = Utf8("Bob")                   |
|               |   SubqueryAlias: e                                                                    |
|               |     TableScan: employees projection=[emp_id, name]                                    |
|               |   SubqueryAlias: d                                                                    |
|               |     TableScan: department projection=[department]                                     |
| physical_plan | ProjectionExec: expr=[emp_id@1 as emp_id, name@2 as name, department@0 as department] |
|               |   NestedLoopJoinExec: join_type=Right, filter=name@0 = Alice AND name@0 = Bob         |
|               |     MemoryExec: partitions=1, partition_sizes=[1]                                     |
|               |     MemoryExec: partitions=1, partition_sizes=[1]                                     |
|               |                                                                                |
+---------------+---------------------------------------------------------------------------------------+
2 row(s) fetched.
Elapsed 0.013 seconds.

The doc of push_down_filter says:
/// For a given JOIN type, determine whether each side of the join is preserved.
///
/// We say a join side is preserved if the join returns all or a subset of the rows from
/// the relevant side, such that each row of the output table directly maps to a row of
/// the preserved input table. If a table is not preserved, it can provide extra null rows.
/// That is, there may be rows in the output table that don't directly map to a row in the
/// input table.
///
/// For example:
///   - In an inner join, both sides are preserved, because each row of the output
///     maps directly to a row from each side.
///   - In a left join, the left side is preserved and the right is not, because
///     there may be rows in the output that don't directly map to a row in the
///     right input (due to nulls filling where there is no match on the right).
///
/// This is important because we can always push down post-join filters to a preserved
/// side of the join, assuming the filter only references columns from that side. For the
/// non-preserved side it can be more tricky.

So this definition is debatable.
We could distinguish between all and subset situations.
For example, left join returns all of the rows from the left side, while returns subset of the rows from the right side.
And inner join returns subset rows of both sides.
When the join returns a subset of the relevant side, we can push the filter down to it.

I agree with @ozankabak in https://github.com/apache/datafusion/issues/10881#issuecomment-2169117921  that the problem is that the filter should not actually be pushed below the join
> EXPLAIN SELECT e.emp_id, e.name, d.department
FROM employees e
LEFT JOIN department d
ON (e.name = 'Alice' OR e.name = 'Bob');
+---------------+----------------------------------------------------------------------------+
| plan_type     | plan                                                                       |
+---------------+----------------------------------------------------------------------------+
| logical_plan  | Left Join:  Filter: e.name = Utf8("Alice") OR e.name = Utf8("Bob")         |
|               |   SubqueryAlias: e                                                         |
|               |     Filter: employees.name = Utf8("Alice") OR employees.name = Utf8("Bob") |
|               |       TableScan: employees projection=[emp_id, name]                       |
|               |   SubqueryAlias: d                                                         |
|               |     TableScan: department projection=[department]                          |
| physical_plan | NestedLoopJoinExec: join_type=Left, filter=name@0 = Alice OR name@0 = Bob  |
|               |   CoalesceBatchesExec: target_batch_size=8192                              |
|               |     FilterExec: name@1 = Alice OR name@1 = Bob                             | <----- it is incorrect to push this filter below the join
|               |       MemoryExec: partitions=1, partition_sizes=[1]                        |
|               |   MemoryExec: partitions=1, partition_sizes=[1]                            |
|               |                                                                            |
+---------------+----------------------------------------------------------------------------+

I also agree with @doki23  in https://github.com/apache/datafusion/issues/10881#issuecomment-2192078394 that the problem is related to the pushdown optimization
Different semantics of predicates in ON and WHERE
I think the core issue is that for OUTER joins, the semantics of predicates in the ON clause are different than predicates in the WHERE clause
Putting (e.name = 'Alice' OR e.name = 'Bob') in the ON clause means the predicate happens during the join, and thus for OUTER joins can't filter non-preserved rows, which is why the Carol row still appears:
postgres=# SELECT e.emp_id, e.name, d.dept_name
FROM employees AS e
LEFT JOIN department AS d ON (e.name = 'Alice' OR e.name = 'Bob');
 emp_id | name  |  dept_name
--------+-------+-------------
      1 | Alice | HR
      1 | Alice | Engineering
      1 | Alice | Sales
      2 | Bob   | HR
      2 | Bob   | Engineering
      2 | Bob   | Sales
      3 | Carol |          <-- this row doesn't match ON clause predicate
(7 rows)

However, when you put the same predicate in the WHERE clause it does filter the row, as expected
postgres=# SELECT e.emp_id, e.name, d.dept_name
FROM employees AS e
LEFT JOIN department AS d ON (true)
where (e.name = 'Alice' OR e.name = 'Bob');
 emp_id | name  |  dept_name
--------+-------+-------------
      1 | Alice | HR
      1 | Alice | Engineering
      1 | Alice | Sales
      2 | Bob   | HR
      2 | Bob   | Engineering
      2 | Bob   | Sales       <-- no "Carol" rows
(6 rows)

Proposed Fix
Thus I propose to fix this issue by changing predicate pushdown to NOT push predicates from the ON clause below OUTER joins. We can and should still push predicates for INNER joins or predicates from the WHERE clause (aka in Filters)

Agreed. This is Spark query plan:
scala> df.explain
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- BroadcastNestedLoopJoin BuildRight, LeftOuter, ((name#13 = Alice) OR (name#13 = Bob))
   :- FileScan parquet spark_catalog.default.employees[emp_id#12,name#13] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/spark-warehouse/employees], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<emp_id:int,name:string>
   +- BroadcastExchange IdentityBroadcastMode, [plan_id=42]
      +- FileScan parquet spark_catalog.default.department[dept_name#15] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/spark-warehouse/department], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<dept_name:string>

The filter predicate is not pushdown.


LOG in EXECUTION_LOGS is binary and How to convert it into readable format?

we are using azkaban with MySQL
found that in the EXECUTION_LOGS  table there is a column named LOG
which is binary blob stored.
Can some good guy tell me how to translate these binary files into human readable format?

It is a zip file...


Memory Leak in GwtMockitoTestRunner$GwtMockitoClassLoader

When running a large suite of JUnit test classes using the @GwtMockitoTestRunner, we consistently encounter a "java/lang/OutOfMemoryError".
We used the Eclipse Memory Analyzer to inspect the resulting heap dump file.
The analysis identified the problem suspect as:
273 instances of "com.google.gwtmockito.GwtMockitoTestRunner$GwtMockitoClassLoader", loaded by "com.ibm.oti.vm.BootstrapClassLoader @ 0x8004b120" occupy 1,519,903,320 (94.34%) bytes.
We are running the following versions of the related libraries:
JUnit - junit:junit:4.12
Mockito - org.mockito:mockito-all:1.10.19
GWTMockito - com.google.gwt.gwtmockito:gwtmockito:1.1.5
To run the tests, we are simply annotating our test classes with
@RunWith(GwtMockitoTestRunner.class)
Is there something we need to do in a tearDown() method to make sure the GwtMockitoTestRunner releases the gwtMockitoClassLoader it created?

Hi tcdavid,
Is there anything you or your team found during this time to fix this issue? Or +1 for an answer from the original developer of this library.
Thank you!

Hm this might be tricky to track down.  Does the profiler you're using provide information on which classes are referencing GwtMockitoClassLoader? I think there are three possibilities:

It's being referenced from GwtMockitoTestRunner, which is the class that should own it. This would imply that JUnit is keeping references to old runners around for some reason, which would be a bit surprising but probably not too hard to work around.
It's being referenced from Thread. We pass the classloader as the context class loader for the thread, which will maintain a reference to it. We're supposed to clean up and switch back to the original context class loader when the test finishes though. So if this is the problem it implies both that we're not cleaning up properly and whatever you're using to run the tests is keeping threads around that it doesn't need.
It's being referenced by some class loaded by the classloader. Every class maintains a reference to its classloader, so if you're somehow saving classes generated during each test (say, by putting them into a static list somewhere), that would cause the classloaders to be retained. It's also possible that GwtMockito is leaking something out to JUnit internally that would cause the reference to be held. This seems like the most likely culprit.

If you can tell which classes are holding on to the reference and whether they're in your code vs. GwtMockito/JUnit's that would go a long way towards tracking down the problem.

We have the same Problem in our project. Memory usage is 5GB with 3500 tests. Number of tests is going to grow, so we need to solve the problem somehow. I also see 266 instances of GwtMockitoTestRunner in the heap dump. This is exactly the number of tests I have with the @RunWith Annotation. Would it help if I provide you the dump @ekuefler ? Because I don't see how to get closer to the source of the problem.
Thank you!

If anyone has collected a heap dump that would definitely be useful for tracking down the problem.

Sweet, thanks! I downloaded the dumps and will be able to analyze them closer before too long.

Hi there, any news ? :)

Hi. Reproduced the problem on my side. @ekuefler it seems that we are in the first case you listed i.e.:

It's being referenced from GwtMockitoTestRunner, which is the class that should own it. This would imply that JUnit is keeping references to old runners around for some reason, which would be a bit surprising but probably not too hard to work around.

JUnit 4 default runner (org.junit.runners.BlockJUnit4ClassRunner) has the same problem (not garbage collected), but it does not reference an explicit Class Loader so it doesn't leads to memory leak.
Any idea about the "not too hard" workaround?

@ekuefler it seems it's a bit of both your first and third point:

It's being referenced from GwtMockitoTestRunner
It's being referenced by some class loaded by the classloader

It seems the first problem is really a JUnit4 one, because I can reproduce it with BlockJUnit4Runner (without memory leak because no additional reference to a classloader in the object). As you said, workaround is possible (gwtMockitoClassLoader = null in finally clause for example).
The big matter is the second one. Without saving any object, it occurs so it should definitely be a leak of some kind.

@ekuefler it seems it's a bit of both your first and third point:

It's being referenced from GwtMockitoTestRunner
It's being referenced by some class loaded by the classloader

It seems the first problem is really a JUnit4 one, because I can reproduce it with BlockJUnit4Runner (without memory leak because no additional reference to a classloader in the object). As you said, workaround is possible (gwtMockitoClassLoader = null in finally clause for example).
The big matter is the second one. There is a bunch of objects that references the class loader, without saving any object like you said, so it should definitely be a leak of some kind.

Hi,
The issue is linked to the use of "ThreadLocal" by Mockito (mentioned here and here).
When Mockito is used inside a unit test executed by the GwtMockitoTestRunner, instances of DefaultMockitoConfiguration and MockingProgressImpl objects are stored (as value) inside the ThreadLocalMap of the Thread executing the test.

The ThreadLocal instances used as keys in the ThreadLocalMap are referenced as static attributes of a class loaded by the GwtMockitoClassLoader, so even if WeakReference are used in the ThreadLocalMap, they won't be garbage collected too (before the GwtMockitoClassLoader is).
As a result, for each test the created GwtMockitoClassLoader stay in memory (which also prevent the garbage collection of the GwtMockitoTestRunner as the GwtMockitoClassLoader is an internal class of it).
I will try to propose a fix very soon, not sure it will be the proper way to do it but I have already done a POC that is working.

I've pushed a new snapshot version that contains this change: https://oss.sonatype.org/content/repositories/snapshots/com/google/gwt/gwtmockito/gwtmockito/1.1.9-SNAPSHOT/
Has anyone had a chance to try the new annotation yet and verify whether it's working as expected?


"Script timeout: exhausted allowed execution time" upon DataStoreReadData task execution.

Hello! I have made this report a few times in the server but decided to make a full report here as I feel I finally have enough information it it.
Over the past week, the admin will randomly break upon server start. This isn't 100% but it is a pretty high chance and causing lots of issues.
We found that it was a problem with the "DataStoreReadData" task, I'm assuming a simple loop error. Fxe believes its an issue with the "Queue" function.
More info:

(I am using a fork in this image just so I am able to track tasks, same error on original module as well)

https://github.com/Sceleratis/Adonis/issues/779


fix(dashboard): don't break storybook and don't show warnings during tests

This PR does three things:

Storybook was neglected for a while, and it had some errors - now it's fixed
Some tests had warnings not affecting the actual results but were annoying - now these are fixed
The README.md file of the dashboard didn't contain instructions on how to start it or how to run the tests - now it's fixed

Codecov Report

Merging #2045 (0861e41) into main (3dca085) will decrease coverage by 0.02%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main    #2045      +/-   ##
==========================================
- Coverage   87.12%   87.11%   -0.02%
==========================================
  Files          85       85
  Lines        9305     9312       +7
  Branches      491      492       +1
==========================================
+ Hits         8107     8112       +5
- Misses       1198     1200       +2

see 1 file with indirect coverage changes


feat: add global error handler

Which problem is this PR solving?

Adds global error handler #1415

Short description of the changes

This PR adds a global error handler and updates SDK components to use it. The default global error handler is configured with a ConsoleLogger at LogLevel.ERROR. This might be a questionable default and we can discuss alternatives.
CollectorExporterError was changed from an interface to a subclass. Other string based log messages were converted to error objects as well. This change means these errors will get a stack that is logged by the default handler.

I think the global error handler should be a part of api, otherwise we will always depend on sdk which I think is not desired. We should decouple sdk from api as much as possible.

I'm fine with wherever we decide to put this, but the global error handler is meant to log errors that come from the SDK. Quoting from #1415:

This error handler is for when the SDK itself errors, not when a monitored entity throws an error.

I think the global error handler should be a part of api, otherwise we will always depend on sdk which I think is not desired. We should decouple sdk from api as much as possible.

I'm fine with wherever we decide to put this, but the global error handler is meant to log errors that come from the SDK. Quoting from #1415:

This error handler is for when the SDK itself errors, not when a monitored entity throws an error.

I see this one both ways. Where have other SIGs implemented it?

Surveying Python and Go and it's a 50/50 split. Python has it has part of the SDK: https://github.com/open-telemetry/opentelemetry-python/pull/1080. Go has it as part of the API: https://github.com/open-telemetry/opentelemetry-go/blob/master/api/global/handler.go.

Surveying Python and Go and it's a 50/50 split. Python has it has part of the SDK: open-telemetry/opentelemetry-python#1080. Go has it as part of the API: https://github.com/open-telemetry/opentelemetry-go/blob/master/api/global/handler.go.

Asked in gitter https://gitter.im/open-telemetry/maintainers?at=5f68edb9ce5bbc7ffddc7b9c

As mentioned by @mwear in gitter, the discussion in https://github.com/open-telemetry/opentelemetry-specification/issues/696 seems to point to including this in the SDK.

I actually forgotten about those path (see #1561):
https://github.com/open-telemetry/opentelemetry-js/blob/d8b1be8cc9918c76b9063780cd40f857332ef5b0/packages/opentelemetry-metrics/src/export/Controller.ts#L48-L60
https://github.com/open-telemetry/opentelemetry-js/blob/master/packages/opentelemetry-tracing/src/MultiSpanProcessor.ts#L36

@vmarchaud I'm trying to get a little more context around what I should log from the PushController. While I could log an error at the location of the @todo log error, there's not much to log, all we have is a an ExportResult that is not ExportResult.SUCCESS. I'd be fine logging here, but the best we can probably do is log a static string as we don't have a handle on a error object. The collector exporter logs at a level above (before calling the resultCallback). I added a little more detail in this comment: https://github.com/open-telemetry/opentelemetry-js/issues/1561#issuecomment-703737042.

@mwear I've added a comment on the issue, i think this would need a clarification on the spec. I would like to log a static string (instead of reject) just to avoid the unhandledRejection (which will crash by default on node 15 if i'm not mistaken). And later on move the global error handler there (which would be simpler than putting it on each exporter anyway).
I think it make more sense to make a seperate PR for this

@vmarchaud Your comment on the issue makes sense and I think we need to have more discussion around it. I can replace the reject with a call to the global error handler on this PR, for now.

@vmarchaud I made the changes you requested. Let me know if that's what you had in mind. Codecov is coming in slim after the latest round of changes though.

Codecov is having a service outage but the error handler itself is tested so I think this is fine. If you fix the conflicts this can be merged.

FWIW, I increased the test coverage before rebasing.


Warema (WMS) Raffstores: only sensors work, not controls

I have 15 Warema raffstores (slats that can go up and down and have an adjustable slat angle of 0 - 80 degrees). They appear in HA, but only the sensors work (and they do correclty track movements that are currently triggered from Homee).
But neither the cover controls nor the slat rotation impulse control works. I am also not sure hoe the rotation impulse would work as it is an on/off switch. I think it should also be a separate cover entity that works in a percentage range.
Logger: homeassistant.components.cover
Source: helpers/entity_platform.py:368
Integration: Cover (documentation, issues)
First occurred: 10:07:49 (1 occurrences)
Last logged: 10:07:49

Error while setting up homee platform for cover
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 368, in _async_setup_platform
    await asyncio.gather(*pending)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1278, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 941, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1062, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 999, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 947, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/cover/__init__.py", line 318, in state
    if (closed := self.is_closed) is None:
                  ^^^^^^^^^^^^^^
  File "/config/custom_components/homee/cover.py", line 151, in is_closed
    if self.get_attribute(self._open_close_attribute).options.reverse_control_ui:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymee/model.py", line 159, in options
    return HomeeAttributeOptions(self._data["options"])
                                 ~~~~~~~~~~^^^^^^^^^^^
KeyError: 'options'

Logger: homeassistant.components.cover
Source: helpers/entity_platform.py:507
Integration: Cover (documentation, issues)
First occurred: 10:07:49 (1 occurrences)
Last logged: 10:07:49

Error adding entities for domain cover with platform homee
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1278, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 941, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1062, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 999, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 947, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/cover/__init__.py", line 318, in state
    if (closed := self.is_closed) is None:
                  ^^^^^^^^^^^^^^
  File "/config/custom_components/homee/cover.py", line 151, in is_closed
    if self.get_attribute(self._open_close_attribute).options.reverse_control_ui:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymee/model.py", line 159, in options
    return HomeeAttributeOptions(self._data["options"])
                                 ~~~~~~~~~~^^^^^^^^^^^
KeyError: 'options'

Hi,
did the controls work before?

This is my first installation of the integration. On 20 Jan 2024, at 00:29, Markus Adrario @.***> wrote:
Hi,
did the controls work before?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

OK, then I expect it is not related to the failures with the latest HA-release.
Can you provide me with a log of the devices as laid out here: https://github.com/Taraman17/hass-homee#homee-device-not-working-correctly

Or wait, looking at the logs, the current beta-release might fix your problem as well...

Indeed, the beta did make a big difference! I now have the two separate photosensor entities (but an error because the standard illuminance one changed form Klx to lx) and the raffstores connect and respond. The only issue left is that the cover entity shoes as unavailable and the “Down Position" control does not work. Identification and Slat rotation do respond.
Logger: homeassistant.components.cover
Source: helpers/entity_platform.py:507
Integration: Cover (documentation https://www.home-assistant.io/integrations/cover, issues https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A"integration%3A+cover")
First occurred: 16:43:32 (1 occurrences)
Last logged: 16:43:32
Error adding entities for domain cover with platform homee
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1282, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 945, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1066, in _async_write_ha_state
state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1003, in __async_calculate_state
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 951, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/cover/init.py", line 318, in state
if (closed := self.is_closed) is None:
^^^^^^^^^^^^^^
File "/config/custom_components/homee/cover.py", line 152, in is_closed
if self.get_attribute(self._open_close_attribute).options.reverse_control_ui:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pymee/model.py", line 159, in options
return HomeeAttributeOptions(self._data["options"])
~~~~~~~~~~^^^^^^^^^^^
KeyError: 'options'

On 20 Jan 2024, at 14:48, Markus Adrario @.***> wrote:
Or wait, looking at the logs, the current beta-release might fix your problem as well...
—
Reply to this email directly, view it on GitHub https://github.com/Taraman17/hass-homee/issues/46#issuecomment-1902098206, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH3AFDAJTZFI6KIYHSPAKETYPPDMJAVCNFSM6AAAAABBX4Q7ICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGA4TQMRQGY.
You are receiving this because you authored the thread.

Looking into this, I see, that I indeed need the logs as per https://github.com/Taraman17/hass-homee#homee-device-not-working-correctly

Closed with #55


Swift 2.2 Support

Any plan for Swift 2.2 support?

@sdtaheri Yes, I will. First of all, I have to install a new XCode.

@sdtaheri Now, it is compatible with Swift 2.2. It has some deprecated codes but it works.
I will fix those deprecated codes too when the new XCode becomes widely used.


I can't seem to solve both #322 and #323 at the same time

I encountered a problem trying to solve issue #322, if i install Tensorpack the problem (deepdoctecion.extern having no attribute named
TPFrcnnDetector) seems to go away, but at the same time the analyzer stops working (see issue #323).
The only way to make the analyzer work again is uninstalling Tensorpack, but that causes the first problem (issue #322)

I am not sure what you mean with “to make the analyzer work again” ?
#322 is required because the TF models are wrapped with Offline-predictors based on Tensorpack.
Solving #323 requires to have a GPU available.
Both issues are not exclusive.


DDP Tool failing to get ddp profile for devices

What happened?
Deploy DP from main with configMap specifying DDP Profile
Device did not get picked up
In the code, I can see we run it as cmd := ddpExecCommand("ddptool", "-l", "-a", "-j", "-s", dev)
I ran ddptool inside of DP container with this flag and you can see it returned error 5, which is what showed in DP logs.
I ran against PCI Address instead and got the DDP profile.
/ # ddptool
Intel(R) Dynamic Device Personalization Tool
DDPTool version <IP_ADDRESS>
Copyright (C) 2019 - 2021 Intel Corporation.

NIC  DevId D:B:S.F      DevName         TrackId  Version      Name
==== ===== ============ =============== ======== ============ ==============================
001) 158B  0000:18:00.1 enp24s0f1       -        -            No profile loaded
002) 1592  0000:AF:00.0 enp175s0np0     0        <IP_ADDRESS>     ICE Wireless Edge Package
003) 1592  0000:B2:00.0 enp178s0np0     0        <IP_ADDRESS>     ICE OS Default Package
/ # ddptool -l -a -j -s enp175s0np0
{"DDPInventory":{
                "error": "5"
                "message": "No driver available"
        }
}
/ # ddptool -l -a -j -s 0000:AF:00.0
{
        "DDPInventory": {
                "device": "1592",
                "address": "0000:AF:00.0",
                "name": "enp175s0np0",
                "display": "Intel(R) Ethernet Network Adapter E810-2C-Q2",
                "DDPpackage": {
                        "track_id": "0",
                        "version": "<IP_ADDRESS>",
                        "name": "ICE Wireless Edge Package"
                }
        }
}

What are the minimal steps needed to reproduce the bug?
Just deploy from main
Seems to be a bug with DDP Tool? we pass the pci address to the PCI.

I can see we do pass the pci address, so not sure why it's not working. it should work.

/ # /usr/bin/ddptool -l -a -j -s 0000:AF:00.0
{
        "DDPInventory": {
                "device": "1592",
                "address": "0000:AF:00.0",
                "name": "enp175s0np0",
                "display": "Intel(R) Ethernet Network Adapter E810-2C-Q2",
                "DDPpackage": {
                        "track_id": "0",
                        "version": "<IP_ADDRESS>",
                        "name": "ICE Wireless Edge Package"
                }
        }
}
/ #

I checked the full cmd, and it failed but looked valid.
I ran the exact cmd inside of pod, and this worked.

I was never in favor of using that ddp tool :(
if you can resume the work that was done here https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin/pull/387 it will be great to have a single valid interface to use ddp without the need for external programs

I was never in favor of using that ddp tool :( if you can resume the work that was done here #387 it will be great to have a single valid interface to use ddp without the need for external programs

Yes - although I don't think the netlink pkg will work with FVL cards - so it needs to be tested. will set something up internally for this validation.

oh you maybe right :(
so maybe in https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin/pull/387 we can have a fallback to this file and lets have a fix for the running of the ddp tool

I've updated #387
I also checked ddp-tool, and tried the latest version - it does not work against VF's
I've created a bug for that team to fix the issue, I will create separate PR to update ddp-tool version once it's ready and validated.

Hi @Eoghan1232 any update on the DDP tool?

Hey @SchSeba so they only recently got back to me, and seems the latest version of DDP works, I need to retest this and update the version we use in the repo
Possibly they need to release the version they tested with as it doesn't seem to be public.
Also the version of ice driver may matter, so I will see if there is docs on the min supported version, etc and update our readMe

I am closing this one we already have support for devlink API instead of this DDP tool


DMP-4523 Update the associated audio files for marking for deletion

Jira link
DMP-4253
Change description

Merged both audio file and associated audio tables into one table in the mark for deletion journey
FIx govuk heading component styling
Fix redundant delete column on file deletion confirmation screen

Spotted an issue when testing with audio id 101153 on staging, it's showing the audio twice for some reason


Cpu and Cuda device mismatch error

So in the line 422 in dbw.py
I notice that you define a val_blocks which is on cpu and the self.get_opacities() is on the gpu
so there will be a error raised
My personal solution is to revise the line 422 from val_blocks = torch.linspace(0, 1, self.n_blocks + 1)[1:]
to
val_blocks = torch.linspace(0, 1, self.n_blocks + 1)[1:].to(self.bkg.device)
and line 429 from values = torch.cat([torch.zeros(NFE) , val_blocks.repeat_interleave(self.BNF)])
to
values = torch.cat([torch.zeros(NFE).to(self.bkg.device) , val_blocks.repeat_interleave(self.BNF)])
and the problem is solved
i am not sure whether this is a common issue or just happened on my device
so just remark it here.

Hi louhz, what is your pytorch version? I have never encountered the issue so it may be linked with a new pytorch behavior related to indexing
Anyway I will fix it tomorrow, thanks for the feedback and solution!

Hi louhz, what is your pytorch version? I have never encountered the issue so it may be linked with a new pytorch behavior related to indexing
Anyway I will fix it tomorrow, thanks for the feedback and solution!

I am using pytorch1.13+cuda 11.7,which is the environment i used for  nerfstudio 0.3.1

Got it thanks, it is indeed a new feature implemented in pytorch 1.13 as explained below (screenshot from this release note)


Class Designer has become 'optional' component with VS 2017

As of VS 2017, the class designer appears to have become an optional component which must be installed separately. This page and others should mention this. (But this was the first page I was brought to by Google!)
Source of information: https://stackoverflow.com/q/42233831/1768779

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9c8eb126-b9c4-b81f-ee87-97ba405c0c25
Version Independent ID: efdcd83c-0280-dbeb-97de-b4da4895c5bc
Content: How to: Add Class Diagrams to Projects (Class Designer) - Visual Studio
Content Source: docs/ide/class-designer/how-to-add-class-diagrams-to-projects.md
Service: unspecified
GitHub Login: @gewarren
Microsoft Alias: gewarren

@ChrisMaddock thanks so much, i'll update this topic.

This is fixed now. Thanks again!

Thanks!


Tracking issue: Provide unit tests for sha512Validator

Origial issue: https://github.com/RedHatInsights/insights-data-schemas/issues/44

Implemented in https://github.com/RedHatInsights/insights-data-schemas/pull/52

Implemented in https://github.com/RedHatInsights/insights-data-schemas/pull/52


Report error on keyword used as a identifier/type name

class int {}; // error
enum return {} // error
// etc...

Maybe it'd be better to just warn about this?


Services: building v3.2.2: abs added

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_4ctbq_gz8b/clone/recipe/meta.yaml:24: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_4ctbq_gz8b/clone/recipe/meta.yaml:36: missing_pip_check: For pypi packages, pip check should be present in the test commands
ERROR: /tmp/abs_4ctbq_gz8b/clone/recipe/meta.yaml:44: http_url: http://pytest-qt.readthedocs.io/en/latest/ is not https
ERROR: /tmp/abs_4ctbq_gz8b/clone/recipe/meta.yaml:44: missing_license_family: The recipe is missing the about/license_family key.
WARNING: /tmp/abs_4ctbq_gz8b/clone/recipe/meta.yaml:44: missing_description: The recipe is missing a description
Errors were found

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_7bc1pihns8/clone/recipe/meta.yaml:24: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_7bc1pihns8/clone/recipe/meta.yaml:36: missing_pip_check: For pypi packages, pip check should be present in the test commands
ERROR: /tmp/abs_7bc1pihns8/clone/recipe/meta.yaml:44: http_url: http://pytest-qt.readthedocs.io/en/latest/ is not https
WARNING: /tmp/abs_7bc1pihns8/clone/recipe/meta.yaml:44: missing_description: The recipe is missing a description
ERROR: /tmp/abs_7bc1pihns8/clone/recipe/meta.yaml:44: missing_license_family: The recipe is missing the about/license_family key.
Errors were found

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_17s61rltqy/clone/recipe/meta.yaml:33: missing_test_requirement_pip: pip is required in the test requirements.
ERROR: /tmp/abs_17s61rltqy/clone/recipe/meta.yaml:45: missing_license_family: The recipe is missing the about/license_family key.
WARNING: /tmp/abs_17s61rltqy/clone/recipe/meta.yaml:45: missing_description: The recipe is missing a description
Errors were found

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_bcago7gvdo/clone/recipe/meta.yaml:46: missing_license_family: The recipe is missing the about/license_family key.
WARNING: /tmp/abs_bcago7gvdo/clone/recipe/meta.yaml:46: missing_description: The recipe is missing a description
Errors were found


Enabled pointer cursor to the Start Learning btn

Fixes Issue
Fixes #210
This button is present on the home page. I just enabled the cursor to be pointing at it when the user hovers on this btn
Changes proposed
To add cursor : pointer to the hp_cont_1_btn class.

/lgtm
Sorry for the delay in responding


User: Ensure user data are separated corretly

Looks much better now. Another round of check has to be run, but for now it's ok.


[posix] RCP(USB CDC ACM) hard reset support

This commit handles the situation when RCP(USB CDC) performs hard reset. This commit brings an alternative to pseudo reset by handling the connection with RCP(USB CDC ACM) by reset the socket connection after the device is reset.
When RCP is connected by USB CDC and we reset it, the RCP device is numerated in host operating system.
So far for RCP(USB CDC) pseudo reset has been performing meaning only specific parts of firmware were reset thanks to  results the connection hasn't been broken. For hard resetting RCP(USB CDC) device the host needs to handle the connection properly.
It uses udev library to check if device enumeration has ended if libudev-dev package is present in the host system otherwise it waits few seconds.
It works based on symlink to the RCP device created by udev in /dev/serial/by-id/ directory.

As a network protocol, Thread should just reset software states, which doesn't necessarily need to reset the chip. Resetting the chip also makes it not friendly when BLE coexists on the the chip. Should we change the implementation of reset to just re-initialize Thread instance? @jwhui @abtink @simonlingoogle

As a network protocol, Thread should just reset software states, which doesn't necessarily need to reset the chip.

Thread is a network protocol, but OpenThread is not merely Thread.
OpenThread is a Thread solution that runs on bare metal / RTOS / POSIX, etc. When running on bare metal, OpenThread provides interfaces (e.g. CLI) to manage the chip because there is no other thing running on the chip.

Resetting the chip also makes it not friendly when BLE coexists on the the chip.

If OpenThread is running as a RTOS Task, I would agree that OpenThread should not reset the whole system. RTOS OpenThread solution can freely remove reset command.

Should we change the implementation of reset to just re-initialize Thread instance?

I think we shouldn't.
I would strongly prefer reset to reset as much state as it can:

On bare metal CHIP, reset should better use system-specific software reset provided by the chip.
On POSIX, reset should restart the whole process (e.g. by exec).

The potential issues of just reinitializing the Thread instance:

reinitializing does not give us a clean start that we can trust. It's very difficult to make everything right after a reinitializing. We can never say that "Now, OpenThread is now free of reinitializing bugs."
It's very easy to introduce new reinitializing bugs. There are many such bug fix OpenThread PRs. Debugging these bugs has taken us significant time and effort.

That being said, I think the current reset command is suitable for bare metal and POSIX, but we should also make reinitializing Thread instance work for RTOS (which I think is difficult).
Thoughts?

@lmaciejonczyk Thanks for this enhancement. I have some questions regarding this PR:

Can we always reset the socket connection no matter how RCP is reset? This can save us the RESET_SOCKET configuration.
Is it possible to wait for the RCP device reset without using libudev , for example by trying to interact with RCP device using via spinel protocol until it's working?
Could hard reset cause the device filename to change: e.g. from /dev/ttyUSB0 to /dev/ttyUSB1?

* Can we always reset the socket connection no matter how RCP is reset? This can save us the `RESET_SOCKET` configuration.

We can but it makes sense only for devices connected by USB CDC ACM. For hardware UART the connection is not broken after hard reset.

* Is it possible to wait for the RCP device reset without using `libudev` , for example by trying to interact with RCP device via spinel protocol until it's working?

It's not possible. The socket is invalid because the device has changed filename. The connection is broken until we reset socket.

* Could hard reset cause the device filename to change: e.g. from `/dev/ttyUSB0` to `/dev/ttyUSB1`?

Yes this is what actually happens when device is connected by USB CDC ACM. Because of that we need to use symlink and wait until device is numerated after the device is reset.

As a network protocol, Thread should just reset software states, which doesn't necessarily need to reset the chip.

Thread is a network protocol, but OpenThread is not merely Thread.

OpenThread should focus on Thread protocol. OpenThread is not a platform.

OpenThread is a Thread solution that runs on bare metal / RTOS / POSIX, etc.
When running on bare metal, OpenThread provides interfaces (e.g. CLI) to manage the chip because there is no other thing running on the chip.

If a platform needs a reset command, they can be provided by platform specific commands, like diag reset. I don't think it reasonable for a network protocol to request resetting the whole chip.

When running on POSIX, OpenThread has the legal right to control the whole process.

Resetting the chip also makes it not friendly when BLE coexists on the the chip.

If OpenThread is running as a RTOS Task, I would agree that OpenThread should not reset the whole system. RTOS OpenThread solution can freely remove reset command.

Should we change the implementation of reset to just re-initialize Thread instance?

I think we shouldn't.
Note that reset is supposed to do (chip) software reset:

This can be removed.

/**
 * This function performs a software reset on the platform, if supported.
 *
 * @param[in] aInstance  The OpenThread instance structure.
 *
 */
void otPlatReset(otInstance *aInstance);

On bare metal CHIP, software reset should better use system-specific software reset provided by the chip.
On POSIX, software reset should restart the whole process (e.g. by exec).

The potential issues of just reinitializing the Thread instance:

reinitializing does not give us a clean start that we can trust. It's very difficult to make everything right after a reinitializing. We can never say that "Now, OpenThread is now free of reinitializing bugs."
It's very easy to introduce new reinitializing bugs. There are many such bug fix OpenThread PRs. Debugging these bugs has taken us significant time and effort.

I don't think a software's quality should be guaranteed by restart, just like we don't like the operating system don't fixes its bugs, but simply suggests users reboot the computer.

The current reset command is software reset. We can certainly have another thread reset command for reinitializing, but software reset is exactly what we want for tests.
That being said, I think the current reset command is suitable for bare metal and POSIX, but we should also make reinitializing Thread instance work for RTOS (which I think is difficult).
Thoughts?

The reset command has been causing issues for a long time. That's why pseudo-reset was introduced. We should improve the quality by fixing bugs instead of ignoring and adding workarounds. Thoughts?

As a network protocol, Thread should just reset software states, which doesn't necessarily need to reset the chip.

Thread is a network protocol, but OpenThread is not merely Thread.

OpenThread should focus on Thread protocol. OpenThread is not a platform.

Changes from this PR have been done in POSIX platform code of OpenThread repository. Therefore I would say it's platform specific not protocol specific feature. If POSIX platfrom had separated repository I would commit there and here we could focus only on Thread protocol.

OpenThread is a Thread solution that runs on bare metal / RTOS / POSIX, etc.
When running on bare metal, OpenThread provides interfaces (e.g. CLI) to manage the chip because there is no other thing running on the chip.

If a platform needs a reset command, they can be provided by platform specific commands, like diag reset. I don't think it reasonable for a network protocol to request resetting the whole chip.

OpenThread requests reset platform. How this is performed it's platform specific and we shouldn't discuss here if it's reasonable or not to reset the whole chip. It's the matter of platform coders. What this PR is about is to provide option to make it work if someone chooses to reset the whole chip.

/**
 * This function performs a software reset on the platform, if supported.
 *
 * @param[in] aInstance  The OpenThread instance structure.
 *
 */
void otPlatReset(otInstance *aInstance);

On bare metal CHIP, software reset should better use system-specific software reset provided by the chip.
On POSIX, software reset should restart the whole process (e.g. by exec).

The potential issues of just reinitializing the Thread instance:

reinitializing does not give us a clean start that we can trust. It's very difficult to make everything right after a reinitializing. We can never say that "Now, OpenThread is now free of reinitializing bugs."
It's very easy to introduce new reinitializing bugs. There are many such bug fix OpenThread PRs. Debugging these bugs has taken us significant time and effort.

I don't think a software's quality should be guaranteed by restart, just like we don't like the operating system don't fixes its bugs, but simply suggests users reboot the computer.

The main reason which stands behind void otPlatReset(otInstance *aInstance); is to provide clean start of the platform including CHIP.  It's far easier here (in OpenThread POSIX platform) to make sure it's working with resetting CHIP than trying provide CHIP reinitializing state espacially if it's working under RTOS.

The current reset command is software reset. We can certainly have another thread reset command for reinitializing, but software reset is exactly what we want for tests.
That being said, I think the current reset command is suitable for bare metal and POSIX, but we should also make reinitializing Thread instance work for RTOS (which I think is difficult).
Thoughts?

The reset command has been causing issues for a long time. That's why pseudo-reset was introduced. We should improve the quality by fixing bugs instead of ignoring and adding workarounds. Thoughts?

Using void otPlatReset(otInstance *aInstance); as way of "fixing bugs" almost never works out. It simply introduces unstable solution. To provide production quality bugs need to be fixed anyway. But void otPlatReset(otInstance *aInstance); is used also for providing clean start state which is the main concern here for CHIPS working under RTOS and communicate by USB CDC ACM. This is not about adding workarounds this is about choosing better solution.

@jwhui @abtink Any thoughts here?

Sorry for lat reply. Somehow I missed this PR fully.
While I agree with some of the points from @bukepo  on that we need to be very selective on new APIs in OT (specially if they are OS related functionality and are not related/used by OT core), on the reset API I think the model we currently have makes sense and agree  @lmaciejonczyk and @simonlingoogle  on this.

OpenThread requests reset platform. How this is performed it's platform specific and we shouldn't discuss here if it's reasonable or not to reset the whole chip. It's the matter of platform coders. What this PR is about is to provide option to make it work if someone chooses to reset the whole chip.

I agree with the above statement. I think it is fine and valid for OT core to request a reset of platform layer. How it is done and implemented is up to platform layer implementation. For example, on NCP the platform code may opt to trigger a full chip reset, but on a posix platform it can behave differently (e.g. Finalize() the otInstance and then re-initialize it, or choose to restart the process, etc).

@jwhui The PR is approved and CI is green, can it be merged now?


Incorrect agent result

fragFINN.de HTTP-Collector

Would you like to send a PR to add this?

I'll give it a try…

Great, thanks!


Bugfix: Exact username match on installation

The install script was only checking if it found anything containing the word kurento, instead of an exact match.
I had a user already added for my middleware service (which contained the word kurento), so the install script failed to create the kurento user because it thought it already existed.
Fix is to just use an exact match with grep.

Hi there. Thanks for your PR.
I'm waiting for a Kurento member to verify that this patch is reasonable to test. If it is, they should reply with check out please on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

There were errors, for info, please see...


live server proxy config

 I want to configure multiple paths like nginx, how to configure the live server.Thanks.

Multiple proxies doesn't support httpProxyServer


Casper Node 1.4.4 not available in official Debian Repository

The last version available in https://repo.casperlabs.io/releases is 1.4.3-0.
Can I rely on this repository to have always the latest version, or should I not rely on it and always build the binary by myself?

Casper-node is never distributed from a debian repo.  casper-client and casper-node-launcher are distributed there.
casper-node-launcher is currently at 1.0.0.
casper-client used to be part of the casper-node repo, but was broken out into its own repo: https://github.com/casper-ecosystem/casper-client-rs
casper-client is backwards compatible, and forwards compatible to the casper-node software (however, might need to be updated if new APIs are introduced in RPC.   You should not expect casper-client to be published with every casper-node binary release.


Port copying error

Parser threw the following error when copying this port (autogenerated from HDL Designer's State Machine tool from the legacy design)
ENTITY MRB_trigger IS
   PORT(
      CE            : IN     std_logic;
      MRB           : IN     std_logic;
      clk_80        : IN     std_logic;
      tacan_mode    : IN     std_logic_vector (6 DOWNTO 0);
      MRB_P_trigger : OUT    std_logic;
      MRB_S_trigger : OUT    std_logic;
      MRB_window    : OUT    std_logic;
      NRT_trigger   : OUT    std_logic
   );

Error created:
vhdl-mode: Interface beginning found.
vhdl-mode: Interface end found.
Traceback (most recent call last):
  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 818, in run_
    return self.run(edit)
  File "C:\Users\mnorton\AppData\Roaming\Sublime Text 3\Packages\vhdl_mode\vhdl_mode.py", line 150, in run
    _interface.parse_block()
  File "C:\Users\mnorton\AppData\Roaming\Sublime Text 3\Packages\vhdl_mode\vhdl_lang.py", line 997, in parse_block
    self.strip_head_tail()
  File "C:\Users\mnorton\AppData\Roaming\Sublime Text 3\Packages\vhdl_mode\vhdl_lang.py", line 935, in strip_head_tail
    self.if_string = self.if_string[hs.end():ts.start()]
AttributeError: 'NoneType' object has no attribute 'start'

Found solution, may be a few days to get it implemented though.  In class Interface, method strip_head_tail, change the tail_pattern to:
        tail_pattern = 'end\s?(?:{})?\s?(?:{})?\s?;'.format(self.type, self.name)

Seems to have been an error with counting spaces, and accounting for a space between the final word and the semicolon (which is a strange style, but since this was machine generated, it's not completely out of the realm).

Fixing this in 1.1.2 today.


Can we integrate other payment gateways

I would like to know if it is possible to integrate other local payment gateways than stripe ?

There were requests on Discord for Paypal and Paddle.me
Thread: https://ptb.discord.com/channels/686873244791210014/1232775770263392368/1233880861200355359

So, I was looking from Indian markets prespective to payment gateways like Razorpay, Instamojo, Cashfree etc.. I would love to take this if we have a documentation on how to do this

@infomiho I'm sure you mean Paddle.com. After researching this topic, it seems like Paddle.com is one of the established solutions. I have read many links, especially I like this discussion https://www.reddit.com/r/SaaS/comments/13noyxv/paddle_vs_lemon_squeezy_experiences


gunicorn configuration needs to be updated to support https streaming

@sblack-usu @martinseul @dtarb I tried to implement https streaming of SOLR results in django to avoid unneeded buffering in local memory or files, and discovered that our gunicorn configuration doesn't support that. Further, we are incurring an unnecessary store-and-forward delay on every gunicorn transaction due to NGINX proxy buffering, so that django streaming doesn't give any performance improvement due to the way gunicorn is set up.
The purpose of proxy buffering is to keep django from getting held up doing tcp to slow clients and/or over slow internet connections. Instead, NGINX manages those TCP transactions. So we can't just turn this off, we need to replace it with something.
Solution is to turn proxy_buffering off in NGINX, and at the same time switch gunicorn to running in multi-threaded mode to do the same thing as proxy buffering. The gunicorn "threads" interact with slow clients. This will support classical https streaming from Django itself. Right now, we only have streaming from NGINX via reverse proxy.
Further, some of the current gunicorn configuration -- including a read timeout of 5 minutes -- is ridiculous. When a request stays open in django and wedges, gunicorn takes an incredibly long time to recover.

There are whole books on configuring this.
See how does django streaming work? for an initial introduction to what streaming does.
See  gunicorn concurrency for the role of threads and pseudo-threads in gunicorn execution.  See particularly the entry on gevent pseudo-threads which solves our hammering problems.

The magic configuration change seems to be:
gunicorn --worker-class=gevent --worker-connections=1000
which moves caching into gevent pseudo-threads at 1000 threads per worker, doing pretty much the same thing as NGINX proxy_buffering on for our configuration.
What this means is that a gevent worker will work on up to 1000 slow client messages at the same time, timesharing between them, to in essence "buffer" the TCP streams.

Note also that this change works in both request and response directions and enables streaming requests direct to Django as well.  I doubt I can make large-file-download faster via this mechanism, but it's worth considering for large-file upload!

Can we test this out on beta and maybe have @ndebuhr test it out with selenium?

@sblack-usu I will write up a separate PR today that makes the changes I recommend and we can mess with it on beta.

This has been addressed.

This has been addressed.


older version in docker

I downloaded docker image and an "update bug" appears.
The problem is, that apparently, this is an old version with known bug, solved by typing:
sudo chown -R coder:users /usr/local/lib
I'm not sure if I've done something wrong, or there is an old version of files in docker.

I updated the latest tag to 3.1.1, thanks!


Possibility to implement an inspector

I'm seeking a way to implement an editor of arbitrary structs using immediate mode GUI. Imagine code like below:
void Inspect<T>(T t) {
  for (const auto& f : rfl::fields<T>()) {
    if (f.is_type<int32_t>()) {
      int32_t value = f.get_as<int32_t>(t);
      int32_t new_value = DrawInt32Editor(value);
      if (new_value != value) {
        f.set_value(t, new_value);
      }
    } else if (f.is_type<std::string>()) {
      ...
    }
  }
}

Currently the get<name|id> or replace<name|id> must be static, which makes the above way to iterate over fields difficult. Is something like this possible to implement in or on top of reflect-cpp?

Was able to get sth to work with
view = rfl::to_view(foo);
view.apply([](const auto& f) {
  if (typeid(*f.value()) == typeid(uint32_t)) {
    ...
  } else if (typeid(*f.value()) == typeid(float)) {
    ...
  }
});

I have a feeling this might be the best way already?

Hi @stakira ,
yes, .apply(...) is meant for exactly these kind of uses cases. The problem with runtime iteration is that you don't have any kind of type safety.
Here is how I would rewrite your example:
#include <type_traits>

const auto view = rfl::to_view(foo);

view.apply([](const auto& f) {
  using Type = typename rfl::remove_cvref_t<decltype(f)>::Type;
  if constexpr (std::is_same<Type, uint32_t*>()) {
    ...
  } else if constexpr (std::is_same<Type, float*>()) {
    ...
  }
});

The beauty about this solution is that you now have compile-time type safety. Once you are inside the branches of the constexpr if, the compiler will know that f.value() is in fact an uint32_t* or float*.
By the way, you can also differentiate by name:

const auto view = rfl::to_view(foo);

view.apply([](const auto& f) {
  if constexpr (f.name() == "f1") {
    ...
  } else if constexpr (f.name() == "f2") {
    ...
  }
});

Question: How would you like to see the documentation improved to make this a bit more clearer?

@stakira, does this resolve the issue? Can we close it?

Thanks!
I realized the apply() is unrolling the tuple using recursive templates, therefore using a regular for loop is not possible.
And the "right way" to deal with this situation seems to be defining a template and its specializations, e.g.:
template <typename T>
void Print(const T& t) {}
void Print(const uint32_t& t) {}
void Print(const float& t) {}

view.apply([&](const auto& f) {
  Print(*f.value());
});

My question is answered. Feel free to close.
Regarding documentation, I think what I'm trying to do is a very common use case. It would be really useful to mention examples we have above in the "Reflective programming" section. To be honest, when you have to put a name or index in the template, like view.get<1> or rfl::get<0>, reflection is not all that useful. If you cannot supply names or indexes as variables, you will have to know the struct type beforehand, then it's not that different from just accessing fields regularly.

@stakira , all right, thank you for your feedback. I will improve the documentation based on your suggestions, because I think that you are right. This is a common use case.
I think the real problem is that this is documented in rfl::NamedTuple:
https://github.com/getml/reflect-cpp/blob/main/docs/named_tuple.md
But I don't think anyone facing this problem would check the documentation for rfl::NamedTuple and then also understand that rfl::to_view(...) creates the named tuple you need to iterate over your struct.


Bug fix: Only return blocks once indexed

Background
There are two block-related endpoints:
let router = Router::new()
   .route("/", get(Self::home))
   .route("/block-count", get(Self::block_count))
   .route("/block/:query", get(Self::block))
   .route("/bounties", get(Self::bounties))

/block-count returns how many blocks have been indexed by ord
/block returns blocks the node knows about regardless of whether ord indexed them
This difference around indexing could possibly lead to users accidentally sending away their ordinals if their wallet uses these APIs to query user artifacts and the wallet believes a block was indexed when it really wasn't.
I discovered this issue after a reorg which stops ord indexing.
Change
This PR queries the database of indexed blocks to determine if a block is truly indexed before returning from /block. Blocks not indexed by ord are effectively hidden.
Potential issues
When /block is called with a block hash, the database currently doesn't have an efficient index to check whether that hash is indexed. The code walks the block hashes known from newest to oldest to check. We could add an index but I wonder whether that is worth doing because at most it will read a few 10s of MBs and only for early blocks.
Testing
Manual

Queried /block with both height and hash during a reorg when node and ord are out of sync. Checked that blocks were filtered.
Queried again when not in a reorg and blocks appear normally.

Raising priority on this. Users might lose their artifacts. Since BTC is experiencing a reorg and ordinals.com is reindexing, but if any wallet is using /block to determine if ord is indexed, they may think a UTXO doesn't have artifacts when it does.

On second look, I believe this affects /tx endpoint as well. This could be bad.

If I use ord in the "normal" way described in the handbook (i.e. no sparrow wallet or similar), is it possible that I lose artifacts or does this problem only apply to other situations?
What can I do to ensure that I do not lose artifacts until the problem is solved by a new version of ord?

The /block endpoint only return a block and no inscriptions so I don't see how this affects acccidentally sending those.
Closing this since it is kind of old. Please reopen if still relevant.


[bug] Authentication issues between the proxy and the hub when restarting the hub

When the hub is shutdown from the Admin tab sometimes it doesn't manage to come restart because authentication issues between the hub and the proxy:
[I 2024-07-22 11:25:50.641 JupyterHub app:2885] Running JupyterHub version 4.1.5
[I 2024-07-22 11:25:50.641 JupyterHub app:2915] Using Authenticator: builtins.GenericOAuthenticatorCSCS
[I 2024-07-22 11:25:50.641 JupyterHub app:2915] Using Spawner: firecrestspawner.spawner.SlurmSpawner
[I 2024-07-22 11:25:50.641 JupyterHub app:2915] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-4.1.5
[I 2024-07-22 11:25:50.646 JupyterHub app:1683] Loading cookie_secret from /home/juhu/jupyterhub_cookie_secret
[W 2024-07-22 11:25:50.704 JupyterHub oauth2:1055] GenericOAuthenticatorCSCS.username_key is deprecated in GenericOAuthenticatorCSCS 16.0.0, use GenericOAuthenticatorCSCS.username_claim instead
[I 2024-07-22 11:25:50.707 JupyterHub app:2005] Not using allowed_users. Any authenticated user will be allowed.
[I 2024-07-22 11:25:50.729 JupyterHub app:2954] Initialized 0 spawners in 0.004 seconds
[I 2024-07-22 11:25:50.734 JupyterHub metrics:279] Found 1 active users in the last ActiveUserPeriods.twenty_four_hours
[I 2024-07-22 11:25:50.734 JupyterHub metrics:279] Found 1 active users in the last ActiveUserPeriods.seven_days
[I 2024-07-22 11:25:50.735 JupyterHub metrics:279] Found 1 active users in the last ActiveUserPeriods.thirty_days
[I 2024-07-22 11:25:50.735 JupyterHub app:3168] Not starting proxy
[E 2024-07-22 11:25:50.740 JupyterHub proxy:950] api_request to proxy failed: HTTP 403: Forbidden
[E 2024-07-22 11:25:50.740 JupyterHub app:3408]
    Traceback (most recent call last):
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/app.py", line 3406, in launch_instance_async
        await self.start()
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/app.py", line 3172, in start
        await self.proxy.get_all_routes()
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/proxy.py", line 990, in get_all_routes
        resp = await self.api_request('', client=client)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/proxy.py", line 954, in api_request
        result = await exponential_backoff(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/utils.py", line 241, in exponential_backoff
        ret = await maybe_future(pass_func(*args, **kwargs))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py311/lib/python3.11/site-packages/jupyterhub/proxy.py", line 937, in _wait_for_api_request
        return await client.fetch(req)
               ^^^^^^^^^^^^^^^^^^^^^^^
    tornado.httpclient.HTTPClientError: HTTP 403: Forbidden

The log of the proxy has many of these
11:25:04.654 [ConfigProxy] warn: 403 GET /api/routes
11:25:20.659 [ConfigProxy] warn: 403 GET /api/routes
11:25:32.131 [ConfigProxy] error: 503 GET /admin connect ECONNREFUSED <IP_ADDRESS>:8081
11:25:32.168 [ConfigProxy] error: 503 GET /favicon.ico connect ECONNREFUSED <IP_ADDRESS>:8081

The issue happened once while staring for the first time after deploying.

I think the problem was that there was a typo in the part of secret.yaml that checks if the configProxyAuthToken is defined:
{{- if $existingSecret }}
  {{- $token = index $existingSecret.data "configProxyAuthToken"}}
{{- end -}}

Fixing the typo should solve the problem.
That's fixed in chart-v0.6.2. Now checking if the issue persists.

Fixing the typo above seems to have solve the issue.

The main problem was not the thing above, but that the value of configProxyAuthToken defined in the secret changes with argocd deployments. Apparently, argocd may re-render helm charts during sync operations, even if one hasn't made any changes in the repo. It doesn't happen with helm deployments.


[FLINK-17204][connectors/rabbitmq] Make RMQ queue declaration consistent between source and sink

What is the purpose of the change
This pull request makes the queue declaration in the RabbitMQ Source and Sink consistent. Previously, the Source asserted a durable queue while the Sink asserted a transient queue. This made using the same queue for input and output impossible (as described in the ticket FLINK-17204). The rationale for setting queues to durable by default is taken from the RabbitMQ UI defaults, shown here:

The classes can still be overridden if end users want different default semantics.
Brief change log

The RMQSink asserts durable queues

Verifying this change
This change added tests and can be verified as follows:

Added a unit test RMQSourceTest.testOpenCallDeclaresQueueInStandardMode(), similar to the existing tests in RMQSinkTest, which confirm that the queue is asserted with consistent parameters
Updated the existing unit tests in RMQSinkTest to assert that durable queues are created

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

I couldn't find any mention of the current configuration in the JavaDocs [RMQSink] [RMQSource], though I'm happy to add docs around this if that would be helpful.

Rebased from master and addressed comments.

Ran all tests locally and passed, unsure why CI is failing but it seems unrelated.


Need special Proxy wrapper for Foo.prototype, Reflect.getPrototypeOf(f)

During Object.freeze() testing, I would often see one of these return an unwrapped object while the other returned a proxy -- and both were supposed to be pointing to the same value.  That's unacceptable.

The need for a special proxy has gone away.  We just needed to consistently set the prototype object on the appropriate shadow targets.


Website is not working

The website http://treeify.chikei.io/ is not working

This site is not affiliated with this project.


PPO-Recurrent Bug: Policy Arguments 'shared_lstm' and 'enable_critic_lstm' passed wrongly if using recurrent Cnn/MultiInput policies

Describe the bug
The policy classes RecurrentActorCriticCnnPolicy and RecurrentMultiInputActorCriticPolicy wrongly initialize its parent class 'RecurrentActorCriticPolicy' with regard to the arguments 'shared_lstm' and 'enable_critic_lstm'. The argument 'shared_lstm' is missing in the keyword arguments of the constructor method of both the CNN and Multi-Input policies and is also not passed to the parent class constructor using positional arguments. Thus the argument 'enable_critic_lstm' (as passed by policy_kwargs from the model class) is passed on the position of the argument 'shared_lstm' to the constructor method of 'RecurrentActorCriticPolicy'. Thus, when using one of the two subclass policies, the 'enable_critic_lstm' argument actually controls the 'shared_lstm' setting in the policy initialization, while 'shared_lstm' is either None or when passed, an error occurs.
Code example
from sb3_contrib import RecurrentPPO

model = RecurrentPPO(
    "CnnLstmPolicy",
    "CarRacing-v0",
    verbose=1,
    policy_kwargs=dict(shared_lstm=True, enable_critic_lstm=False),
)
model.learn(5000)

This code gives the error:
File ".../stable-baselines3-contrib/sb3_contrib/ppo_recurrent/ppo_recurrent.py", line 147, in _setup_model
    self.policy = self.policy_class(
TypeError: __init__() got an unexpected keyword argument 'shared_lstm'

This bug can be fixed by adding the 'shared_lstm' parameter in four positions, two for each of the subclass policies.

thanks for reporting the bug, will be happy to receive a PR that fixes this issue =)

PS: If you don't plan to do a PR, please tell us so I will see when I can do it ;)

Hi @araffin, thanks for the reply. I will submit a PR by today ;).

I have a question before submitting the PR regarding tests (which I don't have much experience with):
I see that there is a test that is meant to detect a bug like this one, however it only does that for the MlpPolicy. Adding the two other policies to that specific test (as a test parameter) would probably make the test_cnn and test_dict_obs redundant. Would that be a problem? Thanks for the help!

It might be simpler to add cases to the others, but it's up to you, you can also add tests to that one https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/blob/master/tests/test_lstm.py#L89


[stable/jenkins] chart YAML parse error

Describe the bug
A clear and concise description of what the bug is.
Version of Helm and Kubernetes:
helm v3.2.1 k8s: 1.16
Which chart:
chart version 2.1.1 is ok
chart version 2.2.0 is broken
What happened:
Error: YAML parse error on jenkins/templates/config.yaml: error converting YAML to JSON: yaml: line 189: could not find expected ':'
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:

@zsjohny Can you provide a values.yaml file which can be used to reproduce this error?

📢 stable/jenkins has moved to https://github.com/jenkinsci/helm-charts. If still applicable, please reopen the issue there. Thanks!
/close


API key docs: add clientSecret to the last step

Bug: #59

Thank you ☺

You're welcome, but I should actually be thanking you for developing this! I'm using it with 2 extensions to upload nightly builds to the CWS and it works perfectly :)


Which Face-Parsing code was used to produce parsing directory in Obama example?

Hello,
Could you point out to which repository or library did you use for face parsing (mouth and neckhead)?
The paper mentions the use of BiSeNet V2 framework, but I am not able to figure out which is the exact code which was used to produce the outputs.
I tried using dg-face-parsing (https://pypi.org/project/dg-face-parsing/) but it would require processing to exactly get the mouth and neckhead images as for Obama example.
Thank You

You can try https://github.com/zllrunning/face-parsing.PyTorch.


👷(tests) fix tests for mysql

Purpose
When Django runs tests, it creates a temporary database with the name "test_{database_name}". The problem is that our fun user does not have the right to create a new database.
Proposal
We can safely run our tests with the mysql root user.
Fixes https://github.com/openfun/richie/issues/1594

@sampaccoud you forgot to add a line on the changelog.

I don't think we need an entry in the changelo as this is only scaffholding... not part of the richie product that we ship...


Feature: circle ci

Add Circle-CI

Fixed with help from this article
https://echobind.com/blog/2017-09-25-setting-up-ember-with-circle-ci-2.0/
Basically, updated the docker image with the -browser flag and removed the CHROME_BIN env.
Still very basic but it works for now.


Move under the /cucumber organization

This repo is not (yet ?) under the Cucumber organization but might be.
For the moment I'm just trying/playing with meta (https://github.com/mateodelnorte/meta) and we'll see how it works when adding various branches for gherkin-8 release.

Done :)


Reduce number of lines required to import all the photos from the images folder.

Title

Whole folder imports


skip the cases when MCO was customized installation

Signed-off-by: hchenxa<EMAIL_ADDRESS>
if the MCO was created and already exists, no need to check the default value.
if the MCO CR did not have retentionConfig, update MCO will throws null point exception, so in this case, only update the storage size and test this part.

/cherry-pick release-2.3


Missing noun

Hi - this sentence is missing a noun between "only" and "for":
"The only for Power BI Premium is a 24-hour refresh per dataflow."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b3ec2a31-ae63-38f7-00e8-234bcedd571d
Version Independent ID: ff4d4e25-80f5-cac3-49c8-a85b7cf1c476
Content: Dataflows Limitations, restrictions and supported connectors and features - Power BI
Content Source: powerbi-docs/transform-model/dataflows/dataflows-features-limitations.md
Service: powerbi
Sub-service: pbi-dataflows
GitHub Login: @davidiseminger
Microsoft Alias: davidi

Hi, @kasavin -- good catch! Assigning to the author to fill in the missing word. I suspect it's "limit" -- "The only limit for Power BI Premium is a 24-hour refresh per dataflow." but I'm checking with the author to be sure.

Thanks for the close read. I've added that missing word, and the change should be live later today. Thanks!


Add $perSecond macro for computing rate for counter-type metrics

First of all, thanks for the great grafana plugin.
What we are missing in our environment is a $perSecond macro that would be very similar to existing $rate macro, but would compute runningDifference of value as well
$rate is now defined as
SELECT t, v/runningDifference(t/1000) vRate FROM
$perSecond should be defined as:
SELECT t, runningDifference(v)/runningDifference(t/1000) vRate FROM
One of the usecase is to see per second network traffic throughput in bytes/s taken from ifconfig (actually collected as a counter type metric by collectd).
Basically, $rate is now usefull for Gauge type metrics, whereas $perSecond would be useful for Counter metrics.

Hi @JiriHorky
I'm glad plugin helps you in your work!
I didn't understand perSecond macros. The rate shows values-per-second. Let's check the following example.
I have some test data in table events. The table resolution is 1 minute:
─name──────┬─type─────────────────────────────────┬─default_type─┬─default_expression─────┐
│ EventDate │ Date                                 │ DEFAULT      │ toDate(EventTime)      │
│ EventTime │ DateTime(\'UTC\')                    │ DEFAULT      │ toStartOfMinute(now())

The $timeSeries query:
SELECT
    (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,
    count() AS c
FROM events
GROUP BY t
ORDER BY t ASC

┌─────────────t─┬─────c─┐
│<PHONE_NUMBER>000 │ 34785 │
│<PHONE_NUMBER>000 │ 35009 │
│<PHONE_NUMBER>000 │ 35136 │
│<PHONE_NUMBER>000 │ 34723 │
│<PHONE_NUMBER>000 │ 35458 │

The result shows how many events were collected per minute. To calculate per-second we need to divide result as following:

34785 / 60 = 579.75
35009 / 60 = 583.48
35136 / 60 = 585.6
...

Let's apply $rate:
SELECT
    t,
    c / runningDifference(t / 1000) AS cRate
FROM
(
    SELECT
        (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,
        count() AS c
    FROM events
    GROUP BY t
    ORDER BY t ASC
)

┌─────────────t─┬─────────────cRate─┐
│<PHONE_NUMBER>000 │               inf │
│<PHONE_NUMBER>000 │<PHONE_NUMBER>333333 │
│<PHONE_NUMBER>000 │             585.6 │

We see the same result (except of first point because because of runningDifference compare first point with 0).
The expression SELECT t, runningDifference(v)/runningDifference(t/1000) vRate FROM, as I understand, will show acceleration. I mean, how fast throughput is growing per second, but not the actual throughput.

Hi @hagen1778 ,
thanks for reaching back. For the input that you described, the $rate macro works well. But for "counter" metrics, i.e. metrics that only grow, your input would look like this:
┌─────────────t─┬─────c─┐
│<PHONE_NUMBER>000 │ 34785 │
│<PHONE_NUMBER>000 │ 69794 │
│<PHONE_NUMBER>000 │ 104930 │
│<PHONE_NUMBER>000 │ 139653 │
│<PHONE_NUMBER>000 │ 175111 │

And for this input, you need to "derive" even the value, not just the time. Please note that this is very standard way of collecting metrics - it is called Counter in collectd, and your "bytes transferred" from "ifconfig" output are collected like that.
In standard graphite, function "perSecond()" computes the rate I am looking for, therefore my suggestion for the template name.

Hi @JiriHorky ,
Sorry for delay! Now I see the issue. The macros should get the max value from every period, substract the value from previous period and divide by the number of seconds in the period.
Let's introduce new data:
SELECT
    Time,
    Packets
FROM issue_78

┌────────────────Time─┬─Packets─┐
│ 2018-08-31 10:51:00 │    1000 │
│ 2018-08-31 10:52:00 │    2000 │
│ 2018-08-31 10:53:00 │    3000 │
│ 2018-08-31 10:54:00 │    4000 │
│ 2018-08-31 10:55:00 │    5000 │
│ 2018-08-31 10:56:00 │    6000 │
│ 2018-08-31 10:57:00 │    7000 │
│ 2018-08-31 10:58:00 │    8000 │
│ 2018-08-31 10:59:00 │    9000 │
└─────────────────────┴─────────┘

Then $perSecond should be following:
SELECT
    t,
    runningDifference(c) / runningDifference(t) AS cRate
FROM
(
    SELECT
        intDiv(toUInt32(Time), 120) * 120 AS t,
        max(Packets) AS c
    FROM issue_78
    GROUP BY t
    ORDER BY t ASC
)

┌──────────t─┬──────────────cRate─┐
│<PHONE_NUMBER> │                nan │
│<PHONE_NUMBER> │ 16.666666666666668 │
│<PHONE_NUMBER> │ 16.666666666666668 │
│<PHONE_NUMBER> │ 16.666666666666668 │
│<PHONE_NUMBER> │ 16.666666666666668 │
└────────────┴────────────────────┘

Is the output correct?

Hi, yes, that's exactly what I was looking for. :+1:
Now, the interesting incarnation of this enhancement is #80 - I would like to see what's the solution for that :)

Please, see https://github.com/Vertamedia/clickhouse-grafana/releases/tag/1.8.0


Remove virtual disks from maas disk utilisation check

With the recent implementation to iterate over all SCSI devices to install maas utilisation alarms, we introduced a new error. This maas  disk utilisation does now include cinder attached volumes if you run those playbooks after a system has been handed over. We basically need to amend this logic to exclude all disks with the model "VIRTUAL-DISK"

The playbook maas_local.yml is affected from this issue

backport is here https://review.openstack.org/#/c/255833/

Can we retarget the backport for the kilo branch?

@d34dh0r53 The backport was already done via https://github.com/rcbops/rpc-openstack/pull/718

ok, closing


create_tagging_chain returning an array string not respecting enum values

System Info
Good day, I'm attempting to use create_tagging_chain_pydantic to tag certain markets wherein a product is used. The product can be used across multiple markets. Unfortunately the chain doesn't appear to respect the enum values passed to it for a return type of an array of string (it works correctly for a [single] string output).
class Tags(BaseModel):
    market_verticals: List[str] = Field(...,
                                        enum=["pharmaceutical", "biotech", "food and beverage", "cosmetic",
                                              "industrial", "nutraceuticals", "agriculture", "other"],
                                        description="""describes which of the market verticals of the various uses
                                          of the product
                                        """)

def main():
    llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")

    chain = create_tagging_chain_pydantic(Tags, llm)

    market_verticals = chain.run(
        "The product is used in the production of food additives, pharmaceuticals, and cosmetics")

Output:
market_verticals=['food additives', 'pharmaceuticals', 'cosmetics']

Amending the prompt, produces the expected output:
class Tags(BaseModel):
    market_verticals: List[str] = Field(...,
                                        enum=["pharmaceutical", "biotech", "food and beverage", "cosmetic",
                                              "industrial", "nutraceuticals", "agriculture", "other"],
                                        description="""describes which of the market verticals of the
                                            various uses of the product. Market verticals are:
                                            pharmaceutical, biotech, food and beverage, cosmetic, industrial,
                                            nutraceuticals, agriculture and other.
                                        """)

Output:
market_verticals=['food and beverage', 'pharmaceutical', 'cosmetic']

$ python3 -V
Python 3.10.10

$ pip show langchain
Name: langchain
Version: 0.0.240
Summary: Building applications with LLMs through composability
Home-page: https://www.github.com/hwchase17/langchain
Author:
Author-email:
License: MIT
Location: /home/tim/Documents/git/ai-magellan/.venv/lib/python3.10/site-packages
Requires: aiohttp, async-timeout, dataclasses-json, langsmith, numexpr, numpy, openapi-schema-pydantic, pydantic, PyYAML, requests, SQLAlchemy, tenacity
Required-by: kor

Who can help?
No response
Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[X] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
Write and run the code below:
from typing import List
from dotenv import load_dotenv
from langchain.chat_models import ChatOpenAI
from langchain.chains import create_tagging_chain, create_tagging_chain_pydantic
from langchain.prompts import ChatPromptTemplate
from enum import Enum
from pydantic import BaseModel, Field

class Tags(BaseModel):
    # enum doesn't work with array
    market_verticals: List[str] = Field(...,
                                  items={"type": "string"}, enum=["pharmaceutical", "biotech", "food and beverage", "cosmetic",
                                                                  "industrial", "nutraceuticals", "agriculture", "other"],
                                  description="""describes which of the market verticals of the
                                            various uses of the product
                                        """)

def main():
    llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613", verbose=True)

    chain = create_tagging_chain_pydantic(Tags, llm, verbose=True)

    market_verticals = chain.run(
        "Vitamin B is is used in the production of food additives, pharmaceuticals, and cosmetics")

    print(market_verticals)

if __name__ == "__main__":
    load_dotenv()
    main()

Expected behavior
Expected output:
market_verticals=['food and beverage', 'pharmaceuticals', 'cosmetics']

Same here. Even when i add in the description "Possible values are between 0 and 1, inclusive", it mostly returns 7.

i have a very similar issue with sentiment analysis.


Dead link

https://docs.microsoft.com/en-us/azure/devops/server/install/install-proxy-setup-remote?view=azure-devops#configure-azure-devops-proxy-server-1
The "blog post" link is a dead link

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8f1b119c-e0a8-5143-bece-d2d0b8556bf6
Version Independent ID: d819bd4f-5272-d350-348e-dd00cc9417c3
Content: Install Proxy Server - Azure DevOps
Content Source: docs/install/install-proxy-setup-remote.md
Product: devops-server
Technology: tfs-admin
GitHub Login: @aaronhallberg
Microsoft Alias: aaronha

@MarRich1 - Thank you for the feedback. I'm updating the article to point to the correct post, now archived, Proxy Upgrades: How to Verify Skipped Proxy Endpoints.


fix build fails with upstream master branch of mruby(latest)

I fixed some fails and do make test, and it has no error.
To enable mrbtest, I fix build_config.rb's host build configuration temporary while I do make test

Travis CI would fail because upstream of mruby's configuration of building mrbtest is disabled.

It fails on travis ci for NoMethodError but not in my local. are there any differences on travis ci?


More steps toward GPU compatibility

Fix type instability in precipitation in test
Remove one abstract type from one function
Start extracting the floating point type from the state
Ensure that time is always Float64 even in the savet array

Looks good! Feel free to merge (ideally after squashing the commits down to 1)

I'd like to keep the commits split because they are logically distinct and each one of them implements a different (and working) change


fix(ui): svg props

:tada: This PR is included in version 3.0.0-beta.15 :tada:
The release is available on:

GitHub release
v3.0.0-beta.15

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 3.0.0 :tada:
The release is available on:

GitHub release
v3.0.0

Your semantic-release bot :package::rocket: