[SPARK-22036][SQL] Decimal multiplication with high precision/scale often returns NULL

What changes were proposed in this pull request?
When there is an operation between Decimals and the result is a number which is not representable exactly with the result's precision and scale, Spark is returning NULL. This was done to reflect Hive's behavior, but it is against SQL ANSI 2011, which states that "If the result cannot be represented exactly in the result type, then whether it is rounded or truncated is implementation-defined". Moreover, Hive now changed its behavior in order to respect the standard, thanks to HIVE-15331.
Therefore, the PR propose to:

update the rules to determine the result precision and scale according to the new Hive's ones introduces in HIVE-15331, which reflect SQLServer behavior;
round the result of the operations, when it is not representable exactly with the result's precision and scale, instead of returning NULL

A more detailed explanation is available here: https://mail-archives.apache.org/mod_mbox/spark-dev/201712.mbox/<CAEorWNAJ4TxJR9NBcgSFMD_VxTg8qVxusjP%2BAJP-x%2BJV9zH-yA%40mail.gmail.com>.
How was this patch tested?
modified and added UTs. Comparisons with results of Hive and SQLServer.

Test build #85116 has started for PR 20023 at commit 3037d4a.

Test build #85116 has finished for PR 20023 at commit 3037d4a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85116/
Test PASSed.

@cloud-fan @dongjoon-hyun @gatorsmile @rxin @viirya I saw you worked on this files. Maybe you can help reviewing the PR. For further details about the reasons of this PR, please refer to the e-mail I sent on the dev mail list. Thank you.

Ideally we should not change behaviors as possible as we can, but since this behavior is from Hive and Hive also changed it, might be OK to follow Hive and also change it? cc @hvanhovell  too

@cloud-fan yes, Hive changed and most important at the moment we are not compliant with SQL standard. So currently Spark is returning results which are different from Hive and not compliant with SQL standard. This is why I proposed this change.

In am generally in favor of following the SQL standard. How about we do this. Let's make the standard behavior the default, and add a flag to revert to the old behavior. This allows us to ease users into the new behavior, and for us it can provide some data points on when we can remove the old behavior. I hope we can remove this for Spark 2.4 or later.
At the end of the day it will be a bit more work, as I'd definitely would make an effort to isolate the the two behaviors as much as possible.

thanks for looking at this @hvanhovell. The reasons why I didn't introduce a configuration variable for this behavior are:

As far as I know, currently there is no way to read reliably a configuration in catalyst;
Also in Hive, the behavior was changed without introducing any configuration to switch back to the previous behavior;
Many people are complaining about the current Spark behavior in the JIRA and therefore it seems that the previous behavior is neither desired nor useful to users.

Let me know if you don't agree with these arguments. Thanks.

I don't fully agree :)...

You can use SQLConf.get for this. Or you can wire up the rules using the SessionStateBuilder.
I am reluctant to change this for a minor version. I don't think the Hive approach is very good for UX.
People are also going to complain if do change it.

Thank you for pining me, @mgaido91 . The approach of PR looks good to me.
BTW, do we need to borrow more Hive 2.2 test cases?

@hvanhovell, as far as 1 is regarded, I was referring to this comment and this PR where it is explicitly stated that using SQLConf here is not safe and it shouldn't be done. Let me know if I am missing something.
I am sorry, but I think I haven't fully understood what you meant by

Or you can wire up the rules using the SessionStateBuilder

may I kindly ask you if you could elaborate this sentence a bit more? Thank you very much.

Thank you for your review @dongjoon-hyun. I think what we can do is add more test to the whitelist in HiveCompatibilitySuite, updating them according to HIVE-15331. Were you thinking to this or something different? Thanks.

I thought adding more cases into decimals.sql. :) But, for now, never mind about that~

I think that we should be careful on suddenly changing behavior.

Thanks for your efforts! This change needs more careful review and investigation.
Could you post the outputs of Oracle, DB2, SQL server and Hive? Are their results consistent?

@gatorsmile, please refer to the e-mail to the dev mail list for further details. I run the script I added to the tests in this PR, the results are:

Hive behaves exactly as Spark after this PR;
SQLServer the same, even though on additions and subtractions it seems to maintain one more precision digit in some cases (I am running SQLServer 2017, since Hive implementation, and therefore this too, are inspired to SQLServer2005, there might have been a small behavior change in this case). Anyway, differently from Hive and Spark it throws an exception in case 3 described in the email (it is compliant to SQL standard, point 3 of the email is out of scope of this PR, I will create another PR for it once we agree on how to handle that case);
Oracle and Postgres have nearly infinite precision. Thus it is nearly impossible to provoke a rounding on them. If we force a precision loss on them (point 3 of the email, out of scope of this PR) they throw an exception (compliant to SQL standard and SQLServer);

Here you are the outputs of the queries.
Hive 2.3.0 (same as Spark after PR)
0: jdbc:hive2://localhost:10000> create table decimals_test(id int, a decimal(38,18), b decimal(38,18));
No rows affected (2.085 seconds)
0: jdbc:hive2://localhost:10000> insert into decimals_test values (1, 100.0, 999.0), (2, 12345.123, 12345.123), (3, 0.1234567891011, 1234.1), (4,<PHONE_NUMBER>23456789.0, 1.123456789123456789);
No rows affected (14.054 seconds)
0: jdbc:hive2://localhost:10000> select id, a+b, a-b, a*b, a/b from decimals_test order by id;
+-----+---------------------------------------+---------------------------------------+----------------------------+----------------------------+
| id  |                  _c1                  |                  _c2                  |            _c3             |            _c4             |
+-----+---------------------------------------+---------------------------------------+----------------------------+----------------------------+
| 1   | 1099.00000000000000000                | -899.00000000000000000                | 99900.000000               | 0.100100                   |
| 2   | 24690.24600000000000000               | 0E-17                                 | 152402061.885129           | 1.000000                   |
| 3   | 1234.22345678910110000                | -1233.97654321089890000               | 152.358023                 | 0.000100                   |
| 4   |<PHONE_NUMBER>23456790.12345678912345679  |<PHONE_NUMBER>23456787.87654321087654321  |<PHONE_NUMBER>04130467.515623  |<PHONE_NUMBER>97814272.043109  |
+-----+---------------------------------------+---------------------------------------+----------------------------+----------------------------+

SQLServer 2017
1> create table decimals_test(id int, a decimal(38,18), b decimal(38,18));
2> insert into decimals_test values (1, 100.0, 999.0), (2, 12345.123, 12345.123), (3, 0.1234567891011, 1234.1), (4,<PHONE_NUMBER>23456789.0, 1.123456789123456789);
3> select id, a+b, a-b, a*b, a/b from decimals_test order by id;
4> GO

(4 rows affected)
id
----------- ---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------------------------------
          1                  1099.000000000000000000                  -899.000000000000000000                             99900.000000                                  .100100
          2                 24690.246000000000000000                      .000000000000000000                         152402061.885129                                 1.000000
          3                  1234.223456789101100000                 -1233.976543210898900000                               152.358023                                  .000100
          4<PHONE_NUMBER>23456790.123456789123456789<PHONE_NUMBER>23456787.876543210876543211<PHONE_NUMBER>04130467.515623<PHONE_NUMBER>97814272.043109

Postgres and Oracle
postgres=# create table decimals_test(id int, a decimal(38,18), b decimal(38,18));
CREATE TABLE
postgres=# insert into decimals_test values (1, 100.0, 999.0), (2, 12345.123, 12345.123), (3, 0.1234567891011, 1234.1), (4,<PHONE_NUMBER>23456789.0, 1.123456789123456789);
INSERT 0 4
postgres=# select id, a+b, a-b, a*b, a/b from decimals_test order by id;
 id |               ?column?                |               ?column?                |                        ?column?                         |               ?column?
----+---------------------------------------+---------------------------------------+---------------------------------------------------------+---------------------------------------
  1 |               1099.000000000000000000 |               -899.000000000000000000 |              99900.000000000000000000000000000000000000 |                0.10010010010010010010
  2 |              24690.246000000000000000 |                  0.000000000000000000 |          152402061.885129000000000000000000000000000000 |                1.00000000000000000000
  3 |               1234.223456789101100000 |              -1233.976543210898900000 |<PHONE_NUMBER>29667510000000000000000000000 |            0.000100037913541123085649
  4 |<PHONE_NUMBER>23456790.123456789123456789 |<PHONE_NUMBER>23456787.876543210876543211 |<PHONE_NUMBER>04130467.515622620750190521000000000000000000 |<PHONE_NUMBER>97814272.043109406191131436
(4 rows)

Spark before the PR
scala> sql("create table decimals_test(id int, a decimal(38,18), b decimal(38,18)) using parquet")
res0: org.apache.spark.sql.DataFrame = []

scala> sql("insert into decimals_test values (1, 100.0, 999.0), (2, 12345.123, 12345.123), (3, 0.1234567891011, 1234.1), (4,<PHONE_NUMBER>23456789.0, 1.123456789123456789)")
res1: org.apache.spark.sql.DataFrame = []

scala> sql("select id, a+b, a-b, a*b, a/b from decimals_test order by id").show(truncate = false)
+---+-------------------------------------+-------------------------------------+-------+-------------------------------------+
|id |(a + b)                              |(a - b)                              |(a * b)|(a / b)                              |
+---+-------------------------------------+-------------------------------------+-------+-------------------------------------+
|1  |1099.000000000000000000              |-899.000000000000000000              |null   |0.100100100100100100                 |
|2  |24690.246000000000000000             |0E-18                                |null   |1.000000000000000000                 |
|3  |1234.223456789101100000              |-1233.976543210898900000             |null   |0.000100037913541123                 |
|4  |123456789123456790.123456789123456789|123456789123456787.876543210876543211|null   |109890109097814272.043109406191131436|
+---+-------------------------------------+-------------------------------------+-------+-------------------------------------+

In DB2, a * b will just stop with an error message: SQL0802N Arithmetic overflow or other arithmetic exception occurred.
Thus, it might be more straightforward for us to follow what DB2 does.

Regarding the rules for deciding precision and scales, DB2 z/OS also has its own rules: https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/sqlref/src/tpc/db2z_witharithmeticoperators.html
Could you compare it with MS SQL Server?

it looks like other databases are very careful about precision lose. However, following DB2 and throwing exception is pretty bad for big data applications, but returning null is also bad as it violates SQL standard.
A new proposal: can we increase the max decimal precision to 76 and keep max scale as 38? Then we can avoid precision lose IIUC.

@gatorsmile I answered to your comments about DB2 in the e-mail.
@cloud-fan that would help, but not solve the problem. It would just make the problem being generated by bigger numbers.
As you can see from the e-mail, DB2 behavior is actually in accordance to SQL standards and the other DBs, it just have a smaller maximum precision. And the case of throwing an exception is point 3 of my e-mails and it is out of scope of this PR, because I think we best discuss before which is the right approach in that case and then I can eventually create a PR.
Therefore, also DB2 behavior is aligned to the other SQL engines, the SQL standard and Spark is the only one which is currently behaving differently.

db2 => create table decimals_test(id int, a decimal(31,18), b decimal(31,18))
DB20000I  The SQL command completed successfully.
db2 => insert into decimals_test values (1, 2.33, 1.123456789123456789)
DB20000I  The SQL command completed successfully.
db2 => select a * b from decimals_test

1
---------------------------------
SQL0802N  Arithmetic overflow or other arithmetic exception occurred.
SQLSTATE=22003

I might not get your point. Above is the result I got. This is your scenario 3 or 2?

@gatorsmile that is scenario 3. I will explain you why and after I will do and errata corrige of the summary I did in my last e-mail, because I made a mistake about how DB2 computes the result precision and scale, sorry for that.
Anyway, what you showed is an example of point 3 because DB2 computes the result type as DECIMAL( MIN(31, p1 + p2), MIN(31, s1 + s2) ). Therefore, in your case the result type was DECIMAL(31, 31). Since your result had more than 0 significant digits, it was out out of the range of the representable values and an overflow exception was thrown.
You can reproduce case 2 as follows:
db2 => create table decimals_test (id int, a decimal(31,31), b decimal(31,31))
DB20000I  The SQL command completed successfully.
db2 => insert into decimals_test values(1, 0.12345678912345678912345689, 0.12345678912345678912345)
DB20000I  The SQL command completed successfully.
db2 => select a*b from dd

1
---------------------------------
 .0152415787806736785461049526020

As you can see a truncation occurred.
Now, let me amend my table to summarize the behavior of the many DBs:

Rules to determine precision and scale

Hive, SQLServer (and Spark after the PR): I won't include the
exact formulas, anyway the relevant part is that in case of precision
higher that the maximum value, we use the maximum available value (38) as
precision and the maximum between the needed scale (computing according the
relevant formula) and a minimum value guaranteed for the scale which is 6.
DB2: computes the result type as DECIMAL( MIN(31, p1 + p2), MIN(31, s1 + s2) ).
Postgres and Oracle: NA
SQL ANSI 2011: no indication
Spark now: if the precision needed is more than 38, use 38 as
precision; use the needed scale without any adjustment.

Behavior in case of precision loss but result in the range of the
representable values

Oracle, Hive, SQLServer (and Spark after the PR): round the result.
DB2: truncates the result (and sets a warning flag).
Postgres: NA, it has infinite precision...
SQL ANSI 2011: either truncate or round the value.
Spark now: returns NULL.

Behavior in case of result out of the range of the representable
values (i.e overflow)

Oracle, DB2, SQLServer: throw an exception.
Postgres: NA, they have nearly infinite precision...
SQL ANSI 2011: an exception should be raised
Spark now, Hive: return NULL (for Hive, there is a open ticket to
make it compliant to the SQL standard).

Thanks for your careful summary!
We do not have a SQLCA. Thus, it is hard for us to send a warning message back like what DB2 does.. Silently losing the precision looks scary to me. Oracle sounds like following the rule, If a value exceeds the precision, then Oracle returns an error. If a value exceeds the scale, then Oracle rounds it.
SQL ANSI 2011 does not document many details. For example, the result type of DB2's division is different from either our existing rule or the rule you changed. The rule you mentioned above about DB2 is just for multiplification.
I am not sure whether we can finalize our default type cocersion rule DecimalPrecision now. However, for Hive compliance, we can add a new rule after we introduce the new conf spark.sql.typeCoercion.mode. See the PR https://github.com/apache/spark/pull/18853 for details. The new behavior will be corrected if and only if spark.sql.typeCoercion.mode is set to hive.
Could you first improve the test cases added in https://github.com/apache/spark/pull/20008 ?

Thanks for your analysis @gatorsmile. Actually the rule you specified for Oracle is what it uses when casting, rather then when doing arithmetic operations. Yes DB2 has rather different rules to define the output type of operations. Anyway, we can have a behavior practically identical to DB2 by changing the value of MINIMUM_ADJUSTED_SCALE to 31. Therefore, I'd propose, instead of using the configuration you pointed out, to use a configuration for the MINIMUM_ADJUSTED_SCALE, changing which we can have both the behavior of Hive and SQLServer and the one of DB2. What do you think?
The reason why I am suggesting this is that my first concern is not Hive compliance, but SQL standard compliance. Indeed, as you con see from the summary, on point 1 there is not a uniform behavior (but this is OK to SQL standard since it gives freedom). But on point 2 we are the only ones who are not compliant to SQL standard. And having this behavior by default doesn't look the right thing to do IMHO. On point 3, only we and Hive are not compliant. Thus I think also that should be changed. But in that case, we can't use the same flag, because it would be inconsistent. What do you think?
I can understand and agree that loosing precision looks scary. But to me returning NULL is even more scary if possible: indeed, NULL is what should be returned if either if the two operands are NULL. Thus queries running on other DBs which relies on this might return very bad result. For instance, let's think to a report where we join a prices table and a sold_product table per country. In this use case, we can assume that if the result is NULL, it means that there was no sold product in that country and then coalesce the output of the multiplication to 0. This would work well on any DB but Spark. With my proposal of tuning the MINIMUM_ADJUSTED_SCALE, each customer can decide (query by query) how much precision loss they can tolerate. And if we agree to change point 3 behavior to the SQL standard, in case of it is not possible to meet their desires we throw an exception, giving them the choice about what to do: allow more precision loss, change their input data type, etc. etc. This is the safer way IMHO.
I would ne happy to help improving test cases. May I just kindly ask you how you meant to do that? What would you like to be tested more? Would you like me to add more test cases in scope of this PR or to open a new one for that?
Thank you for your time reading my long messages. I just want to take the best choice and give you all the elements I have to decide for the best all together.
Thank you.

Following ANSI SQL compliance sounds good to me. However, many details are vendor-specific. That means, the query results still varies even if we can be 100% ANSI SQL compliant.
To avoid frequently introducing behavior breaking changes, we can also introduce a new mode strict for spark.sql.typeCoercion.mode. (Hive is also not 100% ANSI SQL compliant) Instead of inventing a completely new one, we can try to follow one of the mainstream open-source databases. For example, Postgres.
Before introducing the new mode, we first need to understand the difference between Spark SQL and the other. That is the reason why we need to write the test cases first. Then, we can run them against different systems. This PR clearly shows the current test cases do not cover the scenarios of 2 and 3.

Thanks @gatorsmile. Then should I create a follow up PR for #20008 in order to cover the cases 2 and 3 before going on with this PR or can we go on with this PR and the test cases added in this PR?

The problem found by this PR is just one of the issues that returns NULL when SPARK SQL is unable to process it. Below is another example. I believe we can find more.
SELECT CAST('a' AS TIMESTAMP)

Before we deciding how to fix these issues (one by one or as a whole), we need to do more investigation and identify all of them. We also need to clearly document our current behaviors and then our users can know what is the result they can expect.
Yeah! Please go ahead to create a new PR for adding more tests. Thanks!

@gatorsmile created #20084 for adding tests. Thanks.

I think for big data, throwing exception during runtime is pretty bad. I think it's ok to be incompatible with SQL standard for some cases, and return null.
Personally I feel it's ok to follow SQL standard and truncate if precision lose, but we should not follow SQL standard to throw exception when overflow, at least this should not be the default behavior.

@cloud-fan thanks for your feedback. Honestly I don't have a string opinion on how we should behave in that case. That's why I wanted some feedbacks from the community on that point, while this PR covers only the other case, ie. rounding in case of precision loss in accordance to Hive and SQLServer's behavior.
Anyway, IIUC now this PR is blocked by #18853 to introduce spark.sql.typeCoercion.mode and use that property to decide whether to keep the previous behavior or the one proposed here.

hmmm, even we can add many modes like hive mode, postgres mode, etc. we should still make the default/native mode reasonable. It seems more reasonable to truncate the values instead of returning null, in the case of precision lose. @gatorsmile WDYT? Overflow is another story and we should still return null, but introduce a strict mode to throw exception for this case.

Sounds OK to me. Add a conf for this behavior change and also document the behavior change in migration section?

thanks @gatorsmile, do you have any suggestion for the conf name?


bug: multicall does not return proper type inference

Is there an existing issue for this?

[X] I have searched the existing issues

Package Version
1.3
Current Behavior
When using multicall, the documentation indicates the return type should be inferred from the ABI/method-name used as input.
But in practice it's being set to unknown[] even when allowFailure is set to false
Expected Behavior
The array of objects returned should be typed as an object with specific keys/values, and not unknown[]
Steps To Reproduce
import { multicall } from 'wagmi/actions'

const ACCESSORIES = {
  address: '0x8d33303023723dE93b213da4EB53bE890e747C63',
  abi: parseAbi([
    'function totalAccessories() external view returns (uint256)',
    'function balanceOf(uint256 rescueOrder) external view returns (uint256)',
    'function ownedAccessoryByIndex(uint256 rescueOrder, uint256 ownedAccessoryIndex) external view returns ((uint232 accessoryId, uint8 paletteIndex, uint16 zIndex))',
    'function accessoryInfo(uint256 accessoryId) external view returns ((uint16 totalSupply, uint16 availableSupply, bytes28 name, address manager, uint8 metabyte, uint8 availablePalettes, bytes2[4] positions, bool availableForPurchase, uint256 price))',
  ])

let rs = await multicall({
  contracts: [
    {
      ...ACCESSORIES,
      functionName: 'ownedAccessoryByIndex',
      args: [1289n, 0n]
    }
  ],
  allowFailure: false
})

The type of the return variable rs should be {accessoryId: bigint, paletteIndex: number, zIndex number}[], but it is being set as unknown[]
Within the same code context, using readContract with the same inputs works as expected:
const testRs = await readContract({
  ...ACCESSORIES,
  functionName: 'ownedAccessoryByIndex',
  args: [1289n, 0n]
})

Here testRs has an inferred type of {accessoryId: bigint, paletteIndex: number, zIndex number}, which shows the ABI parameter is being set up properly and readContract is able to use it, but multicall isn't.
Link to Minimal Reproducible Example (StackBlitz, CodeSandbox, GitHub repo etc.)
No response
Anything else?
Using Typescript 5.1 and inspecting types inside Visual Studio Code IDE

Can you put together a complete minimal reproducible example? Works fine in this TypeScript Playground. You should add a const assertion to ACCESSORIES (see below).
import { multicall } from '@wagmi/core'
import { parseAbi } from 'viem'

const ACCESSORIES = {
  address: '0x8d33303023723dE93b213da4EB53bE890e747C63',
  abi: parseAbi([
    'function totalAccessories() external view returns (uint256)',
    'function balanceOf(uint256 rescueOrder) external view returns (uint256)',
    'function ownedAccessoryByIndex(uint256 rescueOrder, uint256 ownedAccessoryIndex) external view returns ((uint232 accessoryId, uint8 paletteIndex, uint16 zIndex))',
    'function accessoryInfo(uint256 accessoryId) external view returns ((uint16 totalSupply, uint16 availableSupply, bytes28 name, address manager, uint8 metabyte, uint8 availablePalettes, bytes2[4] positions, bool availableForPurchase, uint256 price))',
  ]),
} as const // <----- need this

const res = await multicall({
  allowFailure: false,
  contracts: [
    {
      ...ACCESSORIES,
      functionName: 'ownedAccessoryByIndex',
      args: [1289n, 0n],
    },
  ],
})
res
// ^? const res: [{ accessoryId: bigint; paletteIndex: number; zIndex: number; }]

I fiddled with this some more and found for my use-case there were actually two factors at play:

Object literals not being declared "as const"
Variables being passed as arguments to the function that were number types instead of bigint

Here's an example of my use-case with the types broken: Typescript playground
import { parseAbi } from 'viem'
import { multicall } from '@wagmi/core'

const ACCESSORIES = {
  address: '0x8d33303023723dE93b213da4EB53bE890e747C63',
  abi: parseAbi([
    'function totalAccessories() external view returns (uint256)',
    'function balanceOf(uint256 rescueOrder) external view returns (uint256)',
    'function ownedAccessoryByIndex(uint256 rescueOrder, uint256 ownedAccessoryIndex) external view returns ((uint232 accessoryId, uint8 paletteIndex, uint16 zIndex))',
    'function accessoryInfo(uint256 accessoryId) external view returns ((uint16 totalSupply, uint16 availableSupply, bytes28 name, address manager, uint8 metabyte, uint8 availablePalettes, bytes2[4] positions, bool availableForPurchase, uint256 price))',
  ])
} as const // <-- This is needed

async function fetchAccessories(rescueOrder: number) {
  const multicalls = [
    {
      ...ACCESSORIES,
      functionName: 'ownedAccessoryByIndex',
      args: [rescueOrder, 1n], // <-- If either of these are passed as numbers and not bigints, rs becomes unknown
    } as const // <-- This is needed
  ]
  let rs = await multicall({ contracts: multicalls, allowFailure: false })
  // ^?
}

The issue is that rescueOrder is coming into the function as a number and is passed as an argument that way, when it's supposed to be a bigint. When used in this way, there doesn't seem to be any alerting that the wrong type of args are being passed in (you just have to infer that from the inferred type being unknown?)?
Is there a better way to structure this such that the mismatch in types becomes more evident?

Add a const assertion to multicalls and you should be good.
import { parseAbi } from 'viem'
import { multicall } from '@wagmi/core'

const ACCESSORIES = {
  address: '0x8d33303023723dE93b213da4EB53bE890e747C63',
  abi: parseAbi([
    'function totalAccessories() external view returns (uint256)',
    'function balanceOf(uint256 rescueOrder) external view returns (uint256)',
    'function ownedAccessoryByIndex(uint256 rescueOrder, uint256 ownedAccessoryIndex) external view returns ((uint232 accessoryId, uint8 paletteIndex, uint16 zIndex))',
    'function accessoryInfo(uint256 accessoryId) external view returns ((uint16 totalSupply, uint16 availableSupply, bytes28 name, address manager, uint8 metabyte, uint8 availablePalettes, bytes2[4] positions, bool availableForPurchase, uint256 price))',
  ])
} as const // <-- This is needed

async function fetchAccessories(rescueOrder: number) {
  const multicalls = [
    {
      ...ACCESSORIES,
      functionName: 'ownedAccessoryByIndex',
      args: [rescueOrder, 1n], // <-- If either of these are passed as numbers and not bigints, rs becomes unknown
    } as const // <-- This is needed
- ]
+ ] as const
  let rs = await multicall({ contracts: multicalls, allowFailure: false })
  // ^?
}

Oh, of course; for that simplified example that makes sense, as the array is being declared as a literal. Thanks! Making it one step more complicated (would this be best as a "discussion" item or a guide page in the docs to cover these sort of stumbling blocks?): if the array is assembled piecemeal (either from a loop or as the result of different function calls), then the array cannot be declared "as const" after its assembled: Playground example
import { parseAbi } from 'viem'
import { multicall } from '@wagmi/core'

const ACCESSORIES = {
  address: '0x8d33303023723dE93b213da4EB53bE890e747C63',
  abi: parseAbi([
    'function totalAccessories() external view returns (uint256)',
    'function balanceOf(uint256 rescueOrder) external view returns (uint256)',
    'function ownedAccessoryByIndex(uint256 rescueOrder, uint256 ownedAccessoryIndex) external view returns ((uint232 accessoryId, uint8 paletteIndex, uint16 zIndex))',
    'function accessoryInfo(uint256 accessoryId) external view returns ((uint16 totalSupply, uint16 availableSupply, bytes28 name, address manager, uint8 metabyte, uint8 availablePalettes, bytes2[4] positions, bool availableForPurchase, uint256 price))',
  ])
} as const // <-- This is needed

async function fetchAccessories(rescueOrder: number, accessoryCount: number) {
  const multicalls = []
  for (let i = 0n; i < accessoryCount; i++) {
    multicalls.push({
      ...ACCESSORIES,
      functionName: 'ownedAccessoryByIndex',
      args: [rescueOrder, 1n], // <-- If either of these are passed as numbers and not bigints, rs becomes unknown
    } as const) // <-- This is needed
  }
  let rs = await multicall({ contracts: multicalls, allowFailure: false })
  // ^?
}

The "bug" here is the value passed into the args isn't exactly right (passing in a number when it should be a bigint). In this situation is there a better way to make it obvious there's a bigint/number mismatch, or convert and ignore the discrepancy?


[Vue 3] Clarify Docs about Vue 3 compatibility

Is your feature request related to a problem? Please describe.
Our team wants to upgrade to Vue3. I can't find anything in the official docs or on github about breaking changes or migration details.
Describe the solution you'd like

Answer the main question whether / how jss is compatible with Vue 3.
Add details about migration if necessary

Describe alternatives you've considered

No details in official docs
No previous questions in github issues
No search results found

We do not provide upgrade steps. Therefore, please refer to the official guide herehttps://v3-migration.vuejs.org/ and for JSS related changes please refer this PR


Make version number easily copyable

Maybe with a button like the repo clone button on GitHub.com?
Or at least make the label selectable.

Maybe we can simply copy the version number by tapping on it. I see no harm in copying the version number accidentally. :)

I see no harm in copying the version number accidentally.

There is. You replace the contents of the clipboard of the user. I never like something that unintentionally replaces the contents of my clipboard as I sometimes have data which I would like to use later or which is not yet stored somewhere.

You have a valid concern @sivaraam! Maybe

we can have a copy icon beside the version number.
we can copy on multiple taps instead of one
we can make it selectable as @nicolas-raoul suggested

The first and last are nice but I'm not sure if the second one is necessary. It just makes the unintended surprise less likely. IMO, we should avoid surprises completely. We could do that by just adding a copy button or making it selectable. Even just one of them would satisfy the need.

I was simply suggesting multiple ways to do it. Obviously, we won't be adding all the ways to copy a version number! Once there is a consensus about how to go about it, a beginner can pick this up. :)

Seems I misunderstand part of it, sorry. Anyways my point is don't introduce unpleasent surprises. So, your option 1 and 3 would be fine :)

I'm good with either making it selectable or having a copy button. :)

Is this for the version number on the About screen? I would like to work on this! :)

@sup95 Yes exactly!
The consensus seems to be that solutions 1 or 3 of https://github.com/commons-app/apps-android-commons/issues/2250#issuecomment-450958591 are good, so feel free to implement the one you prefer, the issue is yours :-)

Reopened, see #2290

Fixed again in #2347


Update blog URL

Removes blog subdomain and adds blog subdirectory. A recent change to the Fennel URL structure was causing redirect issues.

Cheers!


Switch to C++/WinRT -optimized

C++/WinRT has a mode for generating the component projection called "Optimized". The big advantage here is that constructing objects and calling static methods of things that are in the same component can go to the implementation directly and avoid going through GetActivationFactory.
To make this work, the C++/WinRT generated code now needs to know how to find our implementation. The cppwinrt tool generates module.g.cpp (which replaces our existing DllMain implementation that used WRL for factory registration) and a bunch of YourType.g.cpp files that are scaffolding we need to plug into. The generated code assumes that the implementation type is called winrt::Microsoft::UI::Xaml::(etc)::implementation::YourType and the factory type is called winrt::Microsoft::UI::Xaml::(etc)::factory_implementation::YourType. Since ours are not, we need to do something to glue this together.
The strategy I'm using is that each type now needs glue that looks like this:
namespace winrt::*namespace*
{
    namespace implementation { using YourType = ::YourType; }
    namespace factory_implementation { using YourType = ::YourTypeFactory; }
}

And then you can include the new cppwinrt "YourType.g.cpp" file which connects the dots for the Optimized implementation.
I've put as much of this as I can into the code generator, and as you can see from all the new files that got added to the dev\Generated directory that codegen now generates a YourType.properties.cpp file even if the type doesn't have properties or events and only has an activation factory. This way I can codegen the aliasing and the #include of the .g.cpp file.
Another advantage is that because module.g.cpp is generated from everything that needs a factory based on the IDL information, we won't ever make the mistake again of "forgetting" this glue code (the CppWinRTActivationFactory stuff before) because it will be a compile error.
This also allows us to finally remove WRL #includes from the project, which I've also done here.

Nice! Don't forget to define WINRT_LEAN_AND_MEAN if possible.

Nice! Don't forget to define WINRT_LEAN_AND_MEAN if possible.

Thanks for the reminder! I forgot about that one. Everything seems to still compile -- would it be obvious if we had a dependency on something that it excluded?

Would we expect this change to be causing run time crashes? the test failures look like crashes https://helix.dot.net/api/2019-06-17/jobs/96dea496-8f79-4fa9-8534-41358b07383a/workitems/Windows.UI.Xaml.Tests.MUXControls.ApiTests.RepeaterTests.LayoutTests/files/000007_Screenshot.jpg

would it be obvious if we had a dependency on something that it excluded?

The affect of this option is described here: https://github.com/microsoft/xlang/pull/613
In particular, if you rely on (2) and (3) then you will get a compiler error as the specializations are not found by the compiler. If however you rely on (1) then you will not get a compiler error but instead get a "{guid}" formatted string rather than the traditional "type.name" string. This may be an issue if you rely on the latter for some reason, but C++/WinRT itself does not depend on this and strips it out as it adds a ton of additional specializations to a PCH that generally go unused.

WINRT_LEAN_AND_MEAN does appear to be causing at least one issue -- I filed a bug on cppwinrt for it: https://github.com/microsoft/cppwinrt/issues/446
And the other test issues were some over-zealous cleanup in this change. I'm hopeful the next test run is green.
...and of course as I write this comment @kennykerr has already fixed the cppwinrt bug. 😝

For now in this PR I'll leave out WINRT_LEAN_AND_MEAN and we can do it as part of another change.


Add interviews questions of Top IT Companies

Lacs of students attend interviews for top IT Companies like TCS, Infosys, TechMahindra, Wipro, CapGemini, Accenture, Cognizant.
Kindly share your latest interview questions for the benefit of community!

I'm working on this issue

I will do Complete research and create good Quality Questions. Please Assign it to me.


Add storage of time and coordinates to actions

We are currently not getting timestamps or coordinates for actions when doing queries. This was an extremely important tool in older versions and should really have been implemented from the start in this version.

Hover over the result message in your chat for addition information.
Make sure you have added the -extended flag to your lookup command.

Thanks for that. Is it possible to get a way for that to be enabled by default?


In some cases, there may be an 0a byte before the 0x3c

In some cases, checking the first character fails. I'm not sure if there are other edge cases, but here's a stopgap fix.
It would probably be better to look for the byte signature rather than just at the first or 2nd byte.
The code throws if bplistParser or plist.parse fail anyway...
Tested working on my side and the tests pass.

@hexploitable - As you mention, I'm not sure this is the best solution to the problem since two 0x0a characters would slip by as would 0x20. I think you're right that it probably would be better to scan for the signature, but we'd have to think about the best way to do that without scanning all the way through the entire file in a worst case scenario.


Memo Issues with certain Samsung phone

log.txt
I am using the almemo component for android as the Delphi supplied one has problem on samsung phones. The AlMomo works fine but the app dies if i try to double click or long press. Is it something that you have a look. I have attached the log cat file.
Thanks

Hmm, i test on many samsumg phone, what version of the android api is on the phone ?
the error: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/alcinoe/R$dimen;
mean you maybe forget to include in the deployment of your project all the files located in C:\Dev\Alcinoe\lib\jar\com.alcinoe\res
these file are mostly used in Android api <= lollipop (because on  api <= lollipop the copy/past is not a popup but an ugly top bar on the app that we can't easily emulate on delphi app)
i know it's painfull but it's how android work :(
see the little explanation i made under Alcinoe\demos\ALFmxControls_source\android\AddRJavaToClassesDex.bat and use DeployMan.bat to include automatiquelly all the file

Ok, this is the problem ! you must also deploy all the resource files ... see the alfmxControls demo for example. You can use DeployMan.bat to not have to add manually each file one by by in the deployment.

Thanks it did work with the changes that you suggested.


[Vulkan] Improved error message for extern calls passed to SPIR-V codegen

Previously, the codegen indicated that there was an extern call.  Now, also indicate what that extern call is, to aid in debugging.

Potential reviewer: @masahi


docs: Fix broken intra-doc-link.

Note: I merged this with the "fix" tag rather than "docs", to make release-please cut a new release for this change.

Okay!


Module not found?

Not sure if I am doing something wrong but I keep seeing this?
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/fpods/Desktop/dev/side-projects/wa-bot/node_modules/threads-api/build/threads-api' imported from /Users/fpods/Desktop/dev/side-projects/wa-bot/node_modules/threads-api/build/index.js
    at new NodeError (node:internal/errors:399:5)
    at finalizeResolution (node:internal/modules/esm/resolve:326:11)
    at moduleResolve (node:internal/modules/esm/resolve:945:10)
    at defaultResolve (node:internal/modules/esm/resolve:1153:11)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:838:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:77:40)
    at link (node:internal/modules/esm/module_job:76:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Hi, @FarazPatankar! I think @Mineru98 here fixed this bug on #8 (Thanks!).
I'll release and new version of this package after it gets merged, and we'll test it again!

Works. Thanks!


Can't delete a property

If you try to delete the padding property using the code editor here on the "Hello word!!" and then try to apply your changes it won't work. But I see that is working properly when I just change the property.

Thanks, I'll take a look at it.

Great!!! thx!


Fix for evolve_for_xp False + others

Fix for evolve_for_xp False that was not taken into account (Same intend as #5706)
Do not display any message when zero candy earn from buddy (#5702)
A bit stronger pokemon name fixing
Some refactoring
Update one config example
Add more things to doc

👍  This is pretty nice.

Can't review now. Someone please do it and merge, if agreed.


add in a function to the note command

i would like it if the note command could have an add argument to add in a cuboid location something like:
- note add cu@location|location as:name
also a remove with a cuboid list would be cool but the add portion is mainly what i am interested in right now if it is possible :D

Why can't you just
- note <CUBOID.full>|location|location as:CUBOID
?

wow i didnt even see you respond to this lol.... only thing is for that method is how would i remove the new cuboid? or even the old one? is it stored as a list of cuboids like in a flag cuz if so then i could use cuboid.get[2] or whatever


Allow querying on field that do not exist on all models

Currently, if you query on a field that do not exist on all models, lifter will raise an error. Making this behaviour optional, would be an improvement. Example:
manager.filter(User.optional_field.exists(), User.optional_field == 'value')

This was first raised on reddit

+1
It would be good to have the opposite filter also, ie: User.optional_field.does_no_exist().
I wonder if optional_field is not None (since people are used to dict.get returning None if the lookup fails) or optional_field is Exists would make sense.

I don't like the idea of created two methods, instead, you can use the ~ operator, that inverts a query (this work with any query ;).
Filter objects with the field:
manager.filter(Model.optional_field.exists())

Filter objects without the field:
manager.filter(~Model.optional_field.exists())


Bug with GTNH fork of NEI

Cant interact with inventory after looking up a recipe, issue gets solved after removing backhand.
https://github.com/GTNewHorizons/NotEnoughItems/issues/332
https://paste.ee/p/zpSnk

This is still a problem. Removing the backhand fixes the issue.
For us, it's more than just the inventory. Anything that opens any type of GUI doesn't work including the pause menu and chat. Dying results in a glitched screen as well.

Same here. It breaks every GUI once whatever condition is reached.
Nothing can be opened. Even alt-tabbing the game doesn't open the pause menu.
The only way out is to completely close the game.

Should be fixed, it was a severe coding issue on my end that kept GUIs from being opened to prevent Backhand's container mismatching with other containers. Really sorry about this, it's a huge bug that's been around for a long while.
If the issue persists after the next update, shoot me a message and I'll reopen this and get back to it ASAP.
Thank you all for your patience and the report!

It's a shame you couldn't keep the offhand slot.
Even if it was some weird special thing that wasn't technically a slot, but didn't necessitate you replacing the player inventory container.
But, I'll take it in exchange for being able to play the modded game with Backhand again.


[SPARK-37960][SQL] A new framework to represent catalyst expressions in DS v2 APIs

What changes were proposed in this pull request?
This PR provides a new framework to represent catalyst expressions in DS v2 APIs.
GeneralSQLExpression is a general SQL expression to represent catalyst expression in DS v2 API.
ExpressionSQLBuilder is a builder to generate GeneralSQLExpression from catalyst expressions.
CASE ... WHEN ... ELSE ... END is just the first use case.
This PR also supports aggregate push down with CASE ... WHEN ... ELSE ... END.
Why are the changes needed?
Support aggregate push down with CASE ... WHEN ... ELSE ... END.
Does this PR introduce any user-facing change?
Yes. Users could use CASE ... WHEN ... ELSE ... END with aggregate push down.
How was this patch tested?
New tests.

ping @huaxingao cc @cloud-fan

thanks, merging to master!

@cloud-fan Thank you for your review. @HyukjinKwon Thank you too.


Custom frontend queries

What queries do we want to be custom-made for the front end? Is there any specific one we require for the management and display of calls and incidents?

There is an image in #6 for reference.

Adding map and search endpoints for easy data display.


No window open after startup

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: Windows
scrcpy version: 1.24
installation method: Windows release
device model: Xiaomi M2004J7AC
Android version: 11

Describe the bug
It seems that there is something wrong with demuxer thread.
PS D:\Dev\scrcpy> ./scrcpy -V verbose
scrcpy 1.24 <https://github.com/Genymobile/scrcpy>
DEBUG: ADB device found:
DEBUG:     -->   (usb)      ozuww8cykf45hio7            device  M2004J7AC
DEBUG: Device serial: ozuww8cykf45hio7
DEBUG: Using server (portable): D:\Dev\scrcpy\scrcpy-server
D:\Dev\scrcpy\scrcpy-server: 1 file pushed, 0 skipped. 86.2 MB/s (41159 bytes in 0.000s)
[server] INFO: Device: Xiaomi M2004J7AC (Android 11)
DEBUG: Server connected
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
INFO: Renderer: direct3d
DEBUG: Trilinear filtering disabled (not an OpenGL renderer)
DEBUG: Using icon (portable): D:\Dev\scrcpy\icon.png
INFO: Initial texture: 1080x2400
DEBUG: Starting demuxer thread
(End of log)

PS D:\Dev\scrcpy> ./scrcpy -V verbose -r ./1.mkv
scrcpy 1.24 <https://github.com/Genymobile/scrcpy>
DEBUG: ADB device found:
DEBUG:     -->   (usb)      ozuww8cykf45hio7            device  M2004J7AC
DEBUG: Device serial: ozuww8cykf45hio7
DEBUG: Using server (portable): D:\Dev\scrcpy\scrcpy-server
D:\Dev\scrcpy\scrcpy-server: 1 file pushed, 0 skipped. 58.2 MB/s (41159 bytes in 0.001s)
[server] INFO: Device: Xiaomi M2004J7AC (Android 11)
DEBUG: Server connected
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
INFO: Renderer: direct3d
DEBUG: Trilinear filtering disabled (not an OpenGL renderer)
DEBUG: Using icon (portable): D:\Dev\scrcpy\icon.png
INFO: Initial texture: 1080x2400
DEBUG: Starting demuxer thread
DEBUG: Starting recorder thread
INFO: Recording started to matroska file: ./1.mkv
(After a while I press Ctrl+C)
DEBUG: User requested to quit
DEBUG: quit...
DEBUG: Server disconnected
DEBUG: End of frames
DEBUG: Receiver stopped
DEBUG: Server terminated
ERROR: Recording failed to ./1.mkv
DEBUG: Recorder thread ended

logcat.txt

The device encoder does not produce any data.
Try with another encoder: https://github.com/Genymobile/scrcpy#encoder

The device encoder does not produce any data.
Try with another encoder: https://github.com/Genymobile/scrcpy#encoder

Thank you very much, it now works properly.
The encoder scrcpy used before is OMX.MTK.VIDEO.ENCODER.AVC
After I change the encoder to c2.android.avc.encoder or OMX.google.h264.encoder, it works!


Add option to allow specifying if an assert can be nullable

Having now the need to have something like NullOrUuid, this would be great 👍. It would also make it possible to deprecate NullOrDate and NullOrString in favour of a wider-reaching approach.


Add Neon EVM Devnet Support

Description
Added neonDevnet.ts in references/packages/chains/src
Added import for neonDevnet.ts in references/packages/chains/src/index.ts
Additional Information

[x] I read the contributing docs (if this is your first contribution)

Your ENS/address: 0xDf2AC80661DAB9010f9B9cc88e43f3d2E1035fF4

Hey @tmm , thank you for your suggestions. Please merge the changes:)


Allow overriding ServiceFabricDeploy publish profile settings

The service endpoint was updated and will now override:
a. Server cert thumbprint
b. Cluster endpoint

This was top priority because it was confusing that the authentication was used from the service endpoint, but not the endpoint itself.

VSTS Task input overrides:
a. App parameters file
b. Upgrade settings

This is designed to mirror our publish scenario in VS. Since the user can specify these in the task, the publish profile is now optional.

Looks good to me. Approving.

Approved by M1 (devinb). @ericsciple this is ready to merge


There is error with my rule not running even producing error with start nginx

Describe the bug
myenv

openresty <IP_ADDRESS>
Modsecruity-nginx 1.0.0

Logs and dumps
[root@localhost web]# nginx
nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /var/local/share/patch_rules/dynamic.conf. Line: 2. Column: 150. Expecting an action, got:  "noauditlog,\ in /etc/nginx/vhost/80.conf:3

/var/local/share/patch_rules/dynamic.conf.
SecRule TX:REAL_IP "!@pmf /var/local/share/patch_rules/patch_rule_datas/WhiteIpUrlList/url12.data"
     "noauditlog,\
      redirect:/403_forbidden.html,\
      id:282560035,\
      phase:request,\
      chain"
          SecRule REQUEST_FILENAME "@contains /winter"

In other machine , I run it well , but now it not run; what reason is it .

I have tried it many times and it can be loaded if the string without '', but the crs-rules can be run if ruletxt having ''.

Hi @ibeata
have you tried to avoid newline before the action? something like:
SecRule TX:REAL_IP "!@pmf /var/local/share/patch_rules/patch_rule_datas/WhiteIpUrlList/url12.data" "noauditlog,\
      redirect:/403_forbidden.html,\
      id:282560035,\
      phase:request,\
      chain"
          SecRule REQUEST_FILENAME "@contains /winter"

?

Hi @ibeata
have you tried to avoid newline before the action? something like:
SecRule TX:REAL_IP "!@pmf /var/local/share/patch_rules/patch_rule_datas/WhiteIpUrlList/url12.data" "noauditlog,\
      redirect:/403_forbidden.html,\
      id:282560035,\
      phase:request,\
      chain"
          SecRule REQUEST_FILENAME "@contains /winter"

?

Not useful too, it can be effect unless seting all line in one line.

@theMiddleBlue Not useful too, it can be effect unless seting all line in one line.  this error pruduced when i   remove iptables in cenots7. and then i can't run many packages.

Hi @ibeata,
Can you share with us what is your ModSecurity version? I want to check the version of your libModsecurity.

i have redeploy it and run ok. i will open it until it happens. so i would close it at now.


Base Sorcerer archetype can't take Second Bloodline mythic ability

Describe the bug
Base Sorcerer archetype can't take Second Bloodline mythic ability. Crossblooded Sorcerer can take Second Bloodline, but the only option is secondary bloodline he chose when taking the first level of Crossblooded Sorcerer. I tried to reproduce it in unmodded game, but everything was fine. Perhaps bloodline fix from recent patch caused the issue?
To Reproduce
In game respec or DLC1 character creation.
Screenshots
Base Sorcerer archetype

Crossblooded Sorcerer
In this case, primary bloodline is gold dragon, and secondary bloodline is brass .

Additional context
Game version: 2.0.6l
Mods using:

TabletobTweaks-Core
TabletobTweaks-Base
TabletobTweaks-Rework
Bubble Buffs
Toybox(a bag of Tricks)

Enabled TTT-Base options referring to Bloodline:

Draconic Bloodline Descriptions
Patch Elemental Bloodline Descriptions
Bloodline Restrictions
Bloodline Ascendance
Second Bloodline
Second Bloodrager Bloodline

Can confirm same problem.
Can also confirm that nine-tailed heir cannot take Second Bloodline, and that Sage can only take Arcane.

Can confirm same problem.
Can also confirm that nine-tailed heir cannot take Second Bloodline, and that Sage can only take Arcane.

Guessing they changed the selection to no longer ignore prerequisites which
is breaking this
On Sat, Dec 31, 2022, 7:16 PM pheonix99 @.***> wrote:

Can confirm same problem.
Can also confirm that nine-tailed heir cannot take Second Bloodline, and
that Sage can only take Arcane.
—
Reply to this email directly, view it on GitHub
https://github.com/Vek17/TabletopTweaks-Base/issues/568#issuecomment-1368314672,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWSQPEK4FR5QSSR7PT7CHDWQDLGTANCNFSM6AAAAAATNYPIPU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Guessing they changed the selection to no longer ignore prerequisites which
is breaking this
On Sat, Dec 31, 2022, 7:16 PM pheonix99 @.***> wrote:

Can confirm same problem.
Can also confirm that nine-tailed heir cannot take Second Bloodline, and
that Sage can only take Arcane.
—
Reply to this email directly, view it on GitHub
https://github.com/Vek17/TabletopTweaks-Base/issues/568#issuecomment-1368314672,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWSQPEK4FR5QSSR7PT7CHDWQDLGTANCNFSM6AAAAAATNYPIPU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Karma plugin: allow multiple users to gain karma in a single post

Is your feature request related to a problem? Do you think it's something that needs to be implemented in slackscot (versus your integrating bot)? Please describe
The karma plugin only works for one user per message. Example: if a user posts the following in a Slack channel/thread, only user1 will gain karma:
@user1++ @user2++ @user3++

We've often has to individually give karma to multiple users, which is a bit of a pain.
Describe the solution you'd like
The aforementioned example message should give karma to multiple users all at once. Multiple responses by the Slack bot for each user gaining karma is acceptable.
Describe alternatives you've considered
N/A
Additional context
Seems like it would be as simple as iterating over the regexp matches found here

@saidmasoud : Oh yeah, that makes sense. And yes, it should be easy to generalize and support multiple karma recordings in the same message.
Do you want to give that a shot? Otherwise, I can probably do it when I find a few minutes. I believe there are already decent tests to validate the plugin, too.
Thanks!

Sure, I'll take a look when I get a chance!

Awesome. Thanks! Let me know if you run into any issues.

Fixed by PR https://github.com/alexandre-normand/slackscot/pull/162. Thanks, @saidmasoud !


add descriptive errors for var(..)

adds context to error generated by  dotenv::var("MISSING_VAR")
before
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: EnvVar(NotPresent)'

to
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: EnvVar(NotPresent): MISSING_VAR'

Closing because I don't see this as the final format of the error API.
Feel free to contribute in the discussions.


feat(fx-2614): support multiple colors in filters

Closes: FX-2614
Let's take a moment to appreciate that this error-prone commit — https://github.com/artsy/force/commit/e7ba68752b3a1dfba5609eff7aefb149bfc860e8 — shouldn't even exist. Let's convert the input to an input object and never do this again 😅

Draft because it's Friday at 5:45. Will open on Monday 🦢

Should be good to go @mzikherman


I'm not authenticated

I have used the GetRoles api and staticwebapp.config.json for my static web app. I get authorized and get in, but recieve a 403 error due to authentication. when I check /.auth/me I only get below:
{
"clientPrincipal": {
"identityProvider": "aad",
"userId": "<User_Id>",
"userDetails": "<My_Email",
"userRoles": [
"anonymous",
"authenticated"
]
}
those two roles are the default roles and the one I have configured with AAD groups under roleGroupMappings won't show up.
anyone else is having this issue?
P.S. I have used the config file and api files as it's shown in this repo in my Falkon project.

Hi,
We have experienced an issues where some people are not granted the roles that they should receive. Currently it seems like this issue presents itself for people with æ, ø, or å in their names. Could you confirm if this is the case for you as well?
We have explored some options for fixing this, and we are working on implementing them for this template repo as well. I will keep you posted

Ok, so here is the fix, which is based on this repo. See the branch related to this issue for the changes. The branch will not be merged until Microsoft fixes a bug for which this branch has a some-what hacky workaround for. Finally, this solution also includes logging using Application Insights, which is optional)
1. This branch contains two updated files
- /api/package.json - (Only necessary if you want Application Insights
- /api/GetRoles/index.js (Remove the stuff related to Application Insights if you don't want this)
1b. (Optional) Connect Insight Analytics:

In Azure Portal, go to Static Web App > YOUR_STATIC_WEB_APP > Application Insights > Enable Application Insights. Create new if you don't have one.
Verify that the value static web app > Configuration > APPINSIGHTS_INSTRUMENTATIONKEY matches the newly created application insight "Instrumentation Key"

2. Create custom roles
The new solution no longer requires you to have the link between roles and AAD groups hardcoded in the JS-code.

In Azure Portal, go to AAD > App Registration > YOUR_APP_REG > App roles > Create app role.
Display name: "reader"
 Allowed member types: User/Groups
 Value: "reader"
 Description: "reader"

Enterprise Applications > YOUR_APP_REG > Users and groups > Add user/group (might need to wait some minutes before the role created in previous step is visible here)
User and groups: Select person or AAD Group
Select a role: Role that you created in previous step


Quick fix for deploy hangs when deployment includes evicted pods

cc @Shopify/cloudplatform
This PR
This is a quick fix for the deploy problems we've been seeing when there are evicted pods managed by a deployment. The current behaviour is that the deploy hangs until timeout because the number of pods is greater than the number of replicas available, whereas the script requires it to be exactly equal. IMO it would be better to keep this assertion but make it against the new replicaset's pods exclusively. However, this is affecting enough people to justify this patch in the meantime... it's so easy that I really should have proposed it a while ago, though by doing it now I was able to reproduce something similar in a test (see below).
This change also fixes the fact that the set looked at by @rollout_data.values.uniq.length == 1 can include unavailableReplicas.
Testing
I made this change on top of the branch with integration test framework, and it passes all those tests. I also wrote the following regression test, which hangs forever without the patch and passes with it:
  def test_dead_pods_in_old_replicaset_are_ignored
    fixture_set = load_fixture_data("basic", ["configmap-data", "web"])
    deployment = fixture_set["web"]["Deployment"].first
    container = deployment["spec"]["template"]["spec"]["activeDeadlineSeconds"] = 1
    deploy_loaded_fixture_set(fixture_set, wait: false)

    sleep 1 # make sure to hit DeadlineExceeded on at least one pod

    deploy_fixture_set("basic", ["web", "configmap-data"])
    pods = kubeclient.get_pods(namespace: @namespace, label_selector: "name=web,app=basic")
    running_pods, not_running_pods = pods.partition { |pod| pod.status.phase == "Running" }
    assert_equal 1, running_pods.size
    assert not_running_pods.size >= 1
  end

I'm using the unrealistic scenario of a deployment with a pod specifying activeDeadlineSeconds because I wasn't able to figure out how to manually evict a pod in a reasonable timeframe. This scenario also produces a dead pod that hangs around on redeploy, as you can see from the assertions.
Alternatives

Leave as is and wait for this to be fixed via replicaset awareness
Assert that the number of pods must be >= number desired (not sure the assertion adds value, as it could theoretically pass midway through a rollout that allows surge)
Teach deploys to manually delete evicted pods before they start. IMO this should be the job of something like a Pod GC controller, not deploys.

omg could this be the reason of a problem described in https://github.com/Shopify/shopify/pull/104878?


Java documentation: Referenced callback() method does not exist on ForwardChainExpectation

Most (all?) documentation examples for response callbacks use the ForwardChainExpectation#callback method which does not exist. From what I can see, this method has been replaced by a respond() overload starting in fb7f1796122a235d349bdd9caa83ae11e696f4cf.
See for instance:
new MockServerClient("localhost", 1080)
	.when(
		request()
			.withPath("/some/path")
	)
	.callback(
		request -> {
			if (request.getMethod().getValue().equals("POST")) {
				return response()
					.withStatusCode(ACCEPTED_202.code())
					.withHeaders(
						header("x-object-callback", "test_object_callback_header")
					)
					.withBody("an_object_callback_response");
			} else {
				return notFoundResponse();
			}
		}
	);

Which version of the document are you looking at.  Can you please send me a link to the page because this has already been updated.

I copy-pasted that from a page on mock-server.com. This is very strange, but I can't find that example anymore! I looked at my whole browsing history and did not find the version using calback(). I remember it came from your site and the example had both java 7 and 8 versions (using lamba).
The closest I could find is http://www.mock-server.com/mock_server/creating_expectations.html#button_response_method_or_closure_callback
I guess this can be closed then.

Maybe I updated the documentation since you copied the code.  I certainly fixed some out of date documentation recently.


Article for Shell sort

📚 Documentation
Add an article for shell sort.
Have you read the Contributing Guidelines on Pull Requests?
Yes

@HarshCasper please assign this to me


Spent process 2024

this branch introduced the confirmation status enum and includes a number of wallet data structure refactors.
this branch has new tests

Phewf...   big PR!   I am looking at it.

i dreamnt up a way to improve the wallet load tests

fixes #663


⚠ Bofrost.at has degraded performance

In 5011a4f, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1287 ms

Resolved: Bofrost.at performance has improved in ed4820f after 7 minutes.


Split bandit_feedback to train and test when is_timeseries_split=True

Overview
This PR modifies the output of the obtain_batch_bandit_feedback function in obp/dataset/real.py.
In the current implementation, even if is_timeseries_split=True, one dict containing the key such as ..._test was returned.
In this PR, if is_timeseries_split=True, split to create two dict and return them to users.
Note
The modifications due to the changes in this PR  are limited to the scope of test module.
Therefore, for example, for the following file, it is expected that execution will result in an error due to this PR.
https://github.com/st-tech/zr-obp/blob/c870a20c3bb6f3a7d1b42bf1ebed01a5f2239aa6/benchmark/ope/train_regression_model.py#L48

@nmasahiro Thanks! Can you address the following two points?

I think n_rounds key is necessary for bandit_feedback_test as it might be helpful in some cases. It should be int(self.n_rounds - n_rounds_train)

sample_bootstrap_bandit_feedback( should be modified to adjust to the change
https://github.com/nmasahiro/zr-obp/blob/b0700f0857dddfb6b81be1edb5adfa2db58b7bd9/obp/dataset/real.py#L263-L307

I will modify the benchmark scripts later by myself.

Thank you for the review.

I fixed the above points.
In addition to this, I remove reward_test from the files.

Could you check it, please?

@nmasahiro Thanks! LGTM!


Add options to remove download progress bar from output

HI we use goenv manage the GO versions available to our CI/CD environment.   We automate the installation of GO versions  from a job. The log gets very cluttered with the download progress bar.
It would be nice if there was an option that would disable the display of the progress bar.
Something like
--noprogess

Or maybe
--quiet

Looing to see how easy it would be I did find code in build-go to disable progress:
  if [[ $DISABLE_PROGRESS_BAR == "true" ]]; then
    options="-s"
  else
    options="--progress-bar"
  fi

SO setting
~$ DISABLE_PROGRESS_BAR=true goenv install 1.19.5 -s
Downloading go1.19.5.linux-amd64.tar.gz...
-> go1.19.5.linux-amd64.tar.gz
Installing Go Linux 64bit 1.19.5...
Installed Go Linux 64bit 1.19.5 to /home/sarnott/.goenv/versions/1.19.5

@ChronosMasterOfAllTime  - Thanks Muchly

@ChronosMasterOfAllTime  - Thanks Muchly

You're very welcome!


Create a branch patch/content/head and add head content template

Should be created under feature/body.

The branch has been created, and the head content has been added as well.


[Question] I did not get the desired result using SNMP.

Question
When I use a custom SNMP template to retrieve the power status and fan status, the value is not 'OK', but the number 4 and 3.

Why is my field type set to 1, but instead of getting a string, it retrieves a number?

Since the code returns a number instead of the string you were expecting, could you help us fix it?

or (VariableBinding binding : vbs) {
if (binding == null) {
continue;
}
Variable variable = binding.getVariable();
String value;

if (variable instanceof TimeTicks timeTicks) {
    String timeValue = timeTicks.toString(FORMAT_PATTERN);
    oidsValueMap.put(binding.getOid().toDottedString(), timeValue);
} else {
    String valueString = binding.toValueString();
    if (valueString.matches("^[a-zA-Z]+\\(\\d+\\)$")) {
        value = valueString.substring(0, valueString.indexOf("(")).trim();
    } else {
        value = valueString;
    }
    oidsValueMap.put(binding.getOid().toDottedString(), value);
}

}
I am an operations engineer and I don't understand Java code. However, I sent the SnmpCollectImpl.java file to ChatGPT, and it helped me write a chunk of code that I replaced in the data collection part of SnmpCollectImpl.java. But the compilation keeps failing, and I can't resolve it. What should I do?

Hello, in our project, we utilize snmp4j to implement the SNMP protocol. When performing GET operations to query the power status or fan status, snmp4j only provides a numerical representation of the query results. As shown in your command line screenshot, a value of 4 for power status indicates that the power is on, while a value of 3 for fan status indicates that the fan is operating normally. This is reflected in the code snippet oidsValueMap.put(binding.getOid().toDottedString(), binding.toValueString()); within snmpCollectImpl.java.
Although this approach is not as intuitive as using "on" or "ok", it does function properly, and we can still set up alerts based on these values. Currently, I haven't come up with an optimal solution to make the outputs more descriptive.

ok,thanks


Low performance with Qemu's pi2 and pi3 machines

Compared with versatilepb + dhruvvyas90/qemu-rpi-kernel.
The total values are not important in this case, as these are for comparison against each other.
All done on the same machine with the same internet connection, docker on x86 mac (so the container itself is running inside a linux vm):

RAM
speedtest download
speedtest upload
cpu test single thread
cpu test 4 threads
disk write

bare docker container
8 GB
189.43 Mbit/s
77.49 Mbit/s
10 s
10 s
0.5 s

qemu versatilepb
256 MB
3.33 Mbit/s
6.84 Mbit/s
13 s
12 s
18 s

qemu pi2
1 GB
0.67 Mbit/s
0.62 Mbit/s
15 s
6 s
539 s

qemu pi3
1 GB
0.55 Mbit/s
0.72 Mbit/s
26 s
10 s
520 s

Speedtest: speedtest-cli --secure
cpu single thread: sysbench --num-threads=1 --validate=on --test=cpu --cpu-max-prime=1000 run
cpu 4 threads: sysbench --num-threads=4 --validate=on --test=cpu --cpu-max-prime=1000 run
disk write: dd if=/dev/zero of=~/test.tmp bs=1M count=256 conv=fsync
https://github.com/carlosperate/docker-qemu-rpi-os/blob/37ac36b820cf98f1da1acf61d1ffc27fcf1de978/benchmark-vm.py#L96-L118

Been looking into this and there are a few things to take in consideration.

The original dockerpi project did not accurately emulate a Raspberry Pi 1, it had the commonly used versatilepb qemu machine configuration

https://www.qemu.org/docs/master/system/arm/versatile.html
The kernel and device tree used from dhruvvyas90/qemu-rpi-kernel has been built for this machine type
Probably okay for most people, but it's more like the Pi OS userspace running on top of a custom kernel and virtualised board
The 256 MB of RAM are a limitation that cannot be increased

The Raspberry Pi 2 and 3 modes do use the qemu raspi machine types, and the kernel and device tree that come with the Pi OS images

https://www.qemu.org/docs/master/system/arm/raspi.html
However as shown above, the performance is quite low

Similarly to the Raspberry Pi 1 versatilepb machine, the virt machine type can be used with a custom kernel, but more RAM

https://www.qemu.org/docs/master/system/arm/virt.html
Has a built the kernel in the Docker image as well

Added in commit: 49c922b8b9e30984767105cd8b9be960732ae095

RAM
speedtest download
speedtest upload
cpu test single thread
cpu test 4 threads
disk write

qemu versatilepb
256 MB
3.33 Mbit/s
6.84 Mbit/s
13 s
12 s
18 s

qemu virt
1 GB
10.31 Mbit/s
10.75 Mbit/s
10 s
4 s
14 s

Been looking into this and there are a few things to take in consideration.

The original dockerpi project did not accurately emulate a Raspberry Pi 1, it had the commonly used versatilepb qemu machine configuration

https://www.qemu.org/docs/master/system/arm/versatile.html
The kernel and device tree used from dhruvvyas90/qemu-rpi-kernel has been built for this machine type
Probably okay for most people, but it's more like the Pi OS userspace running on top of a custom kernel and virtualised board
The 256 MB of RAM are a limitation that cannot be increased

The Raspberry Pi 2 and 3 modes do use the qemu raspi machine types, and the kernel and device tree that come with the Pi OS images

https://www.qemu.org/docs/master/system/arm/raspi.html
However as shown above, the performance is quite low

Similarly to the Raspberry Pi 1 versatilepb machine, the virt machine type can be used with a custom kernel, but more RAM

https://www.qemu.org/docs/master/system/arm/virt.html
Has a built the kernel in the Docker image as well

Added in commit: 49c922b8b9e30984767105cd8b9be960732ae095

RAM
speedtest download
speedtest upload
cpu test single thread
cpu test 4 threads
disk write

qemu versatilepb
256 MB
3.33 Mbit/s
6.84 Mbit/s
13 s
12 s
18 s

qemu virt
1 GB
10.31 Mbit/s
10.75 Mbit/s
10 s
4 s
14 s


implement custom signer (priv_key param)

idea: pass a private key and sign the tx requests created by Generator.
Was in Spammer -- I removed it but it might still be the right place for it. The generator creates tx requests, so there needs to be an intermediate step to sign all the requests.
The generator could give them the right nonces, gas prices, etc, but because the system generates all tx requests up-front, this job becomes considerably tougher. To correctly calculate the gas prices needed for all transactions in advance, we have to consider how block times may affect the outcome, how much gas our transactions will use, what the target gas limit of the block is; something something EIP1559. If your node (e.g. anvil) processes each tx request into a block, the base fee could look a lot different than if you process a block every 12 seconds. This makes signing all tx requests in advance quite challenging.
A simple but slow and bad alternative is to sign requests on the fly, checking the gas price and revising the request before signing & sending. This would trash the performance. Generating our payloads in advance should yield the best performance.
For now, we're sending unsigned requests with the assumption that the account is unlocked on the target node, and the Generator includes the from field in the tx requests it generates.

see #14


feat:allow lending with LM V2 during migration period

@py-ro is there any brownie script to change the liquidity mining impl in the LoanTokenLogic ?
function setLiquidityMiningAddress(address LMAddress) external onlyOwner;

BTW I cannot find the case were you test that the onTokensDeposited is invoked after the migration process has started (but not finished). My questions are:

Will that be done by re-targeting using the setLiquidityMiningAddress ?
Are you testing the following case:
a. User deposits on v1
b. Migration starts but user funds were not moved yet
c. User mints and tokens are deposited into v2
d. User funds are migrated from v1


Deletes are not deleting all needed state and are taking a long time

Currently a delete of an experiment can take a long times (10s of seconds). It doesn't recognized merged experiments so deletes too much stuff while at the same time leaving around items that should have been deleted.

Documented how delete should work for the different items.  This will require "garbage collection" to properly delete the rest of the items in the database.
Updated documentation and code to work with merged experiments.


Changing text to remove the trailing # in C#

marked ignores the trailing #. I tried escaping with \ but that didn't work.
A trailing   is an option , where we get to keep the text as is, but I don't know how it'll look like in parsers.

Hi @fashaikh, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

What library can't handle trailing #'s? This seems like a fragile place to be - we're going to keep accidentally adding trailing #'s. What breaks when that happens?

This library seems to be parsing the markdown into html.
For trailing # where the markdown begins with a # it seems to ignore the trailing # and displays C# as C


FIX confusing UX of CMS permissions: Pages/Files

Description
Currently Pages and Files permissions have a really confusing UX as:

Content permissions given to the page/file in Security Groups is not visible.
Content permissions given to the page/file in SiteConfig is not visible. e.g. this group can edit pages.

Potential AC
As a CMS user I want to be able to have a clear understanding of all the user permissions affecting a specific page, so that I can be informed when editing page permissions.
Improve visibility of permissions in Pages/Files
Surface SiteConfig permissions in Page/File permission settings
Permissions given to Security Groups set within the SiteConfig, is not visible within Page settings, causing confusion on what is already set.

[ ] Surface permissions of Setting/Access (SiteConfig) in Page/File permissions UI as read-only tag(s)
[ ] Create tooltip for tag: This group already has been given access via "Settings"

Surface Security/Group permissions in Page/File permission settings
When Content permissions are assigned to Security Groups, their permissions are not visible in Page/File permission settings. This cause confusion to what is set.

[ ] Surface Content permissions of Groups within Page/File permissions UI as read-only tag(s).
[ ] Create tooltip for tag: This group already has been given access via "Security/Groups"

Clarify what Edit means
Currently "Who can edit content" permissions include publishing-related permissions (this includes publishing, unpublishing, archiving etc). This should be separate controls, one with only edit-related and only publication-related permissions.

[ ] Remove publishing-related permissions from "Who can edit content" controls
[ ] Create a publishing-related permissions option called "Who can publish content?"

Wireframe flows
Select: Page settings
https://projects.invisionapp.com/dsm/silverstripe/silverstripe/section/components/5bf228be8d9a690013f1ae39

 (5) Review of permissions in CMS, with recomendations in GitHub


Firefox for Android 79+ supports inputmode.

This is shipped by Firefox for Android 79.
See https://bugzilla.mozilla.org/show_bug.cgi?id=1631681

Confirmed, thanks!


Coroutine Matchers

How would this work?

How would this work?

https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-test

The 4.0 coroutines support does everything we need from this so closing.


Installation failure

While following the install instructions from the README.md I received the following error while running mix deps.get:

Getting surface (https://github.com/msaraiva/surface.git - v0.1.0-rc.1)
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 3101 (delta 3), reused 7 (delta 3), pack-reused 3084
Receiving objects: 100% (3101/3101), 1.06 MiB | 5.71 MiB/s, done.
Resolving deltas: 100% (2201/2201), done.
Dependencies have diverged:

phoenix_live_view (Hex package)
the dependency phoenix_live_view in mix.exs is overriding a child dependency:

In mix.exs:
{:phoenix_live_view, "~> 0.14.6", [env: :prod, repo: "hexpm", hex: "phoenix_live_view"]}

In deps/surface/mix.exs:
{:phoenix_live_view, [env: :prod, git: "https://github.com/phoenixframework/phoenix_live_view.git", tag: "597c5ddf8af2ca39216a3fe5a44c066774de3abd"]}

Ensure they match or specify one of the above in your deps and set "override: true"
** (Mix) Can't continue due to errors on dependencies

Hi @jburky!
Do you have both, :phoenix_live_view and :surface as dependencies in your mix.exs? If so, can you try to remove :phoenix_live_view and let just :surface there? Then run mix deps.get and compile again.
Please let me know it works. If it does, I'll update the README and add a note about this. I believe this can only be properly fixed after LV v.0.15 is out.

Got a similar error with dashboards:

Updating phoenix_live_view (https://github.com/phoenixframework/phoenix_live_view.git - 597c5ddf8af2ca39216a3fe5a44c066774de3abd)
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
connection 1.0.4
cowboy 2.8.0
cowboy_telemetry 0.3.0
cowlib 2.9.1
db_connection 2.3.0
decimal 2.0.0
earmark 1.4.10
earmark_parser 1.4.10
ecto 3.5.4
ecto_sql 3.5.3
file_system 0.2.9
floki 0.29.0
gettext 0.18.2
html_entities 0.5.1
jason 1.2.2
mime 1.4.0
nimble_parsec 0.5.3
phoenix 1.5.6
phoenix_ecto 4.2.1
phoenix_html 2.14.2
phoenix_live_dashboard 0.3.6
phoenix_live_reload 1.2.4
phoenix_live_view 0.14.8
phoenix_pubsub 2.0.0
plug 1.11.0
plug_cowboy 2.4.1
plug_crypto 1.2.0
postgrex 0.15.7
ranch 1.7.1
telemetry 0.4.2
telemetry_metrics 0.6.0
telemetry_poller 0.5.1
Dependencies have diverged:

phoenix_live_view (https://github.com/phoenixframework/phoenix_live_view.git - 597c5ddf8af2ca39216a3fe5a44c066774de3abd)
different specs were given for the phoenix_live_view app:

In deps/surface/mix.exs:
{:phoenix_live_view, [env: :prod, git: "https://github.com/phoenixframework/phoenix_live_view.git", tag: "597c5ddf8af2ca39216a3fe5a44c066774de3abd"]}

In deps/phoenix_live_dashboard/mix.exs:
{:phoenix_live_view, "~> 0.14.3", [env: :prod, hex: "phoenix_live_view", repo: "hexpm", optional: false]}

Ensure they match or specify one of the above in your deps and set "override: true"
** (Mix) Can't continue due to errors on dependencies

Thanks for the extra info!
Surface is not compatible with LV v0.14.6 so I believe you'll have to add :phoenix_live_view using override: true:
  defp deps do
    [
      ...

      {:phoenix_live_view,
       git: "https://github.com/phoenixframework/phoenix_live_view.git",
       tag: "f986171cae03747366c3cd1c4f4fea047e1d7196",
       override: true},
      {:surface,
       git: "https://github.com/msaraiva/surface.git",
       tag: "b745e1322ce7b20fcbcf8503f507349da0772435"},
    ]
  end

However, the question is whether the version of phoenix_live_dashboard you're using will be compatible with the latest LV changes on master.
BTW, I'll release v0.1.0-rc.2 later today so you'll be able to use the following instead:
{:surface, git: "https://github.com/msaraiva/surface.git", tag: "v0.1.0-rc.2"}

@jburky version v0.1.0-rc.2 has just been released. Feel free to give it a try and let me know if you find any issues.

In case anyone needs to use both, Surface and Phoenix LiveDashboard, there's already a fork with the proper fixes  https://github.com/BlinkerGit/phoenix_live_dashboard/tree/0.3.6-0.15-dev.

@msaraiva That's great news. I will try and give it a go later today.

@msaraiva I just tried it out, and is seems to still fail...
Here is a reproducible project with the bug: https://github.com/happycodrz/failing_assigns_with_surface
Plain phx app + surface. Hope this helps to narrow the problem down. And: thanks for this great library, really looking to try it out on my next project!

@msaraiva Just saw the link to https://github.com/BlinkerGit/phoenix_live_dashboard/tree/0.3.6-0.15-dev -> this fork works properly!

@msaraiva are these fixes going to be coming to phoenix_live_dashboard eventually so that we don't have to rely on a fork?

@jessejanderson the fixes present in the fork are related to changes in LV's API, so yes, they must be eventually merged upstream, otherwise, it won't work with LV >= v0.15.


Fix undefined behaviour for pages with numbers as title (e.g 7)

When requesting the page "23", it is expected to receive this page: https://en.wikipedia.org/wiki/23
Expected behavior:
wiki.page("23") -> https://en.wikipedia.org/wiki/23
wiki.page(23) -> https://en.wikipedia.org/wiki/Assistive_technology
Actual behavior:
wiki.page("23") -> https://en.wikipedia.org/wiki/Assistive_technology (which has ID 23)
wiki.page(23) -> https://en.wikipedia.org/wiki/Assistive_technology
So there is no way to get the page for the number 23 or for example years like 1939, sine they will be treated as IDs.
It is checked whether an ID or a Title is given by isNaN(title) in utils.ts, the String "7" will be handled like a page ID.
Instead check the actual type of what the function receives:
return typeof title === 'string' || title instanceof String;
New behavior:
wiki.page("23") -> https://en.wikipedia.org/wiki/23
wiki.page(23) -> https://en.wikipedia.org/wiki/Assistive_technology
-> Supply a number, get a page by ID, supply a string, get the page by title

Accidentally added a version change to the pull request... My bad

I saw in the Tests that you actually test against strings as IDs. If the behavior is wanted like this, I would at least consider adding parameters to the query that define whether the string should be treated as ID or title.

Its a very interesting point @phiph-s - apologies for the late reply i was travelling last week.
You are correct in saying that currently we do consider numbers as ids particularly. Your MR makes sense to me but i think we need some more test cases. I will work on the same in the coming weekend if you are busy with something else.
Thanks for the MR @phiph-s

I'm currently quite limited in time, but maybe I'm able to write some test cases.
Another Option would be to implement the 2 functions "pageById" and "pageByTitle", in order to not change the behavior of the current implementation of "page", as this change might break some projects that use this package.


As a developer I want to use blackfire.io to profile my application

I'm new to ddev and what I have seen so far is really awesome, but I do (think I) miss something and thats one of my favorite tools at the moment: blackfire.io. Is it possible to add this feature?

Hi @haassie! I've never personally worked with blackfire.io, but it looks like from the installation instructions (https://blackfire.io/docs/up-and-running/installation) that it's mostly a series of apt-get installs. Now we could add this by default, but we would want to evaluate this solution in terms of its general need/appeal before inclusion. That said, you could adopt this approach as a post-start hook to run those commands and store it in version control so that you always had these available. See https://github.com/drud/ddev/issues/655#issuecomment-366557008 for an example on how you would go about that.

It looks like there is some relation to this issue. https://github.com/drud/ddev/issues/691

I realized that's only half of it.
The image would need
RUN export VERSION=`php -r "echo PHP_MAJOR_VERSION.PHP_MINOR_VERSION;"` \
    && curl -A "Docker" -o /tmp/blackfire-probe.tar.gz -D - -L -s https://blackfire.io/api/v1/releases/probe/php/linux/amd64/${VERSION} \
    && tar zxpf /tmp/blackfire-probe.tar.gz -C /tmp \
    && mv /tmp/blackfire-*.so `php -r "echo ini_get('extension_dir');"`/blackfire.so \
    && echo "extension=blackfire.so\nblackfire.agent_socket=\${BLACKFIRE_PORT}" > $PHP_INI_DIR/conf.d/blackfire.ini

OK, here's my end result.
docker-composer.blackfire.yaml
version: '3'
services:
  blackfire:
    container_name: ddev-${DDEV_SITENAME}-blackfire
    image: blackfire/blackfire
    ports: ['8707']
    environment:
      BLACKFIRE_SERVER_ID: d97e69fc-a4bb-4b92-b9f3-70d3bef0c2ef
      BLACKFIRE_SERVER_TOKEN: 8deefb5b9e948c1674b14778c65c5a657b8232fba754d04a3208f18e00fdaea1
  web:
    links:
      - blackfire:$DDEV_HOSTNAME

config.yaml change
webimage: drud/nginx-php-fpm-local:2018-06-20_exec-sudo

install-blackfire.sh
#!/usr/bin/env bash
wget -q -O - https://packagecloud.io/gpg.key | sudo apt-key add -
echo "deb http://packages.blackfire.io/debian any main" | sudo tee /etc/apt/sources.list.d/blackfire.list
sudo apt-get update
sudo apt-get install blackfire-php

I manually ran install-blackfire.sh, could be a hook command.

Egh, and changing blackfire.agent_socket = tcp://blackfire:8707

I'm not sure if its the project or config, but my profiles with this setup is not uploading (weird error I've never seen.)

@mglaman I've applied your configuration and the upload of the profiles works fine!

@sven-carstens @mglaman I wonder if you could do a quick update here to show how best to do it now in ddev v1.8.0, where the packages are installed?


JS console on Mac OS X (Mavericks) + Chrome

The console for JS challenges does not respond in Chrome on Mac OS X Mavericks. I tried a hard refresh with no luck. It works fine in Safari.

Could you check your JavaScript console to see whether it shows a JavaScript error? A screenshot would also be helpful. Thanks!

Refused to load the script 'http://freecodecamp.com/js/lib/jailed/_frame.js' because it violates the following Content Security Policy directive: "script-src *.optimizely.com *.aspnetcdn.com *.d3js.org https://cdn.inspectlet.com/inspectlet.js http://cdn.inspectlet.com/inspectlet.js http://beta.freecodecamp.com 'self' blob: <IP_ADDRESS> *.freecodecamp.com http://www.freecodecamp.com https://www.freecodecamp.com https://freecodecamp.com https://freecodecamp.org *.freecodecamp.org http://www.freecodecamp.org ws://freecodecamp.com/ ws://www.freecodecamp.com/ *.gstatic.com *.google-analytics.com *.googleapis.com *.google.com *.gstatic.com *.doubleclick.net *.twitter.com *.twitch.tv *.twimg.com 'unsafe-eval' 'unsafe-inline' *.bootstrapcdn.com .cloudflare.com https://.cloudflare.com localhost:3001 ws://localhost:3001/ http://localhost:3001 localhost:3000 ws://localhost:3000/ http://localhost:3000 *.ionicframework.com https://syndication.twitter.com *.youtube.com .jsdelivr.net https://.jsdelivr.net *.ytimg.com *.bitly.com http://cdn.inspectlet.com/ https://cdn.inspeclet.com/ wss://inspectletws.herokuapp.com/ http://hn.inspectlet.com/ *.googleapis.com *.gstatic.com https://hn.inspectlet.com/".

The button has been clicked at this point.

@alexbrotman this should be fixed by performing a hard refresh

closing as stale. Please feel free to reopen if issue still persists.


feat: add plugin-markdown-chart

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

103 of 104 (99.04%)  changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+2.0%) to 59.793%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

plugins/markdown/plugin-markdown-chart/src/node/markdown-it-plugins/mermaid.ts
19
20
95.0%

Totals

Change from base Build<PHONE_NUMBER>8:
2.0%

Covered Lines:
1276

Relevant Lines:
1923

💛  - Coveralls

@pengzhanbo  I have some issues with katex in markmap, helps would be appreciated.

@pengzhanbo I have some issues with katex in markmap, helps would be appreciated.

I haven't used markmap before. I read its documentation and the code of the Markmap component in the plugin, and I did not find any obvious issues. However, I noticed that markmap-view depends on d3, and I'm not sure if that has any impact. Perhaps we can refer to markmap-autoloader to adjust the loading logic of the markmap component.


fixed typo in name displayed in github workflow

Current behavior
Line 110 of the rust.yml file has a typo, as mentioned in issue #5114
Proposed changes
Fixed the Typo

@queenkatherinecodes Thank you for spending time on this, before we can merge please

[ ] Accept the Ockam Contributor License Agreement by adding your Git/Github details in a row at the end of the CONTRIBUTORS.csv file in a separate pull request to the build-trust/ockam-contributors repository.

The easiest way to do this is to edit the CONTRIBUTORS.csv file in the github web UI and create a PR, this will make the commit as verified.

Merging in #5116.
@queenkatherinecodes Thank you for an awesome first PR 🥳
I'm preparing and merging your changes in #5116.
A couple of things to note, as you explore other issues to contribute to:

We avoid any merge commits, only do rebases, and maintain a linear history in our develop branch. This makes reviewing changes a bit easier. So I removed the merge commit in #5116
We have a commit message convention to make it easier to track changes and generate release change logs, so I amended your commit message a bit but made sure we maintain your author attribution. https://github.com/build-trust/.github/blob/main/CONTRIBUTING.md#commit-messages

The changes are merged, thank you https://github.com/build-trust/ockam/commits/develop


is_professional does not modify

Updating is_professional has no effect.
Changes to is_professional are not being saved.

Error solved


Switch password hashing algorithm

When I implemented the password hashing and verification algorithm, I used older method of doing that. I don't know whether any of the current users use PHP < 5.5 but I haven't seen anything below 5.5 for a while that wasn't up-gradable to 5.5 or higher. Let me know if you need me to support PHP < 5.5 or if I can switch to password_hash method.

If you switch the hashing algorithm, will you then stop emailing passwords in clear text too? Because regarding security this is a no go. https://security.stackexchange.com/questions/17979/is-sending-password-to-user-email-secure
But regardles this is a real cool project, thank you for your work!

Mailing passwords is intended behavior right now, because it is only meant to be one time password (I'm pretty sure the email says that they should change it ASAP). This could be mitigated by having some checks in place - for example that the user would need to change password after first login. We could of course skip the mailing completely and have the admin give password to the user via a secure channel. What would be your preferred behavior? As this was at first intended to be only a school project I didn't give security much thought :slightly_smiling_face:

I see :) Yeah, the email says it should be change ASAP, but unfortunately most of the users don't...
I would recommend something, that  is seen quite often (eg. WordPress):

admin creates new user. He can decide if the user should set the password or if the admin does.
if the user sets the password a link with a token to set the password is sent to him (like the lost password function).

Just a thought... Would it not make sense to follow PHP versions end of life paradigm? 5.6 will no longer be supported after 31st of Dec 2018. PHP 7.1 will be the oldest version supported after Jan 2019, so it would be normal to only support 7.1 and 7.2 from that date on...
Ref: http://php.net/supported-versions.php

Yes that would make sense, I wanted the script to run everywhere and some hosting providers are pretty lazy to update PHP versions. But supporting 5.6+ would definitely make sense.


OpenThread build issues

As previously reported here, a clean build appears to required for projects integrating OpenThread. Moreover, some parameters (ex. : CONFIG_OPENTHREAD_NETWORK_NAME) just never make it into the final build, clean or not.
Also, the parameter CONFIG_OPENTHREAD_CUSTOM_PARAMETERS must be defined on top of already equivalent parameters such as CONFIG_OPENTHREAD_JOINER. For example:
CONFIG_OPENTHREAD_JOINER=y
CONFIG_OPENTHREAD_CUSTOM_PARAMETERS="OPENTHREAD_CONFIG_JOINER_ENABLE=1"

From what I've seen, the OpenThread project does not catch up with the Kconfig changes done after the project is generated. @tejlmand Is there any Kconfig dependency we could add to the OpenThread build stage?

@lodup29 I've submitted a fix for the OpenThread build system behaviour(https://github.com/zephyrproject-rtos/zephyr/pull/35252), it should keep up with the Kconfig changes now.

Moreover, some parameters (ex. : CONFIG_OPENTHREAD_NETWORK_NAME) just never make it into the final build, clean or not.

Parameters like CONFIG_OPENTHREAD_NETWORK_NAME are not intended for OpenThread build, they're processed by the Zephyr part of the OpenThread integration. Generall, options that affect the OpenThread build are kept in openthread/Kconfig.features (or at least that's what we want to achieve).

Also, the parameter CONFIG_OPENTHREAD_CUSTOM_PARAMETERS must be defined on top of already equivalent parameters such as CONFIG_OPENTHREAD_JOINER. For example:

This one I'm not able to reproduce. Even w/o this fix, if you set CONFIG_OPENTHREAD_JOINER in the prj.conf file, it will be recognized by OpenThread and will enable the joiner feature in the OpenThread stack. No need to use CONFIG_OPENTHREAD_CUSTOM_PARAMETERS  (this should only be used for special cases, OpenThread options that are not covered with Kconfig. Actually I'm not sure if there're any left).

@tejlmand Is there any Kconfig dependency we could add to the OpenThread build stage?

Not sure that would be needed. There is a global CMake dependency on .config so if any Kconfig option is changed, for example through menuconfig then CMake will rerun on next rebuild, and this in itself should trigger a rerun of the OpenThread CMake code, which hopefully inside OpenThread triggers correct rebuild of OpenThread source files in case OpenThread build parameters has changed.


[Backport] Add version to canonical URL

Backport of #8747 to 0.16.0-incubating.

skipping CI for doc only backport