Polar Test Flows Working/Not Working

(h/t to @jamaljsr for running these tests and documenting it all) Jamal ran through test flows on Polar to exhaustively test what is currently working with the latest vs. what is not working. Those flows and their outcomes are outlined below.
The network structure is the following with all litd nodes except for erin which is Core Lightning. I've re-tested using the latest litd branch 0-19-staging (https://github.com/lightninglabs/lightning-terminal/commit/980ee938aa1e7a45633216aff8775ca8d05c12aa)
alice --[asset]-> bob --[sats]-> carol --[asset]-> dave
                                   |
                                 [sats]
                                   |
                                   v
                                  erin (core-ln)

Working

[x]  fund asset channel using full asset balance after mint
[x] fund asset channel using partial asset balance after mint
[x] fund asset channel using full asset balance after mint + onchain send
[x] fund asset channel using partial asset balance after mint + onchain send
[x] fund asset channel using full asset balance from onchain receive
[x] fund asset channel using partial asset balance from onchain receive
[x] fund asset channel with push_sat=5000
[x] create asset invoice on dave, pay with asset on alice (multi-hop)
[x] create asset invoice on bob, pay with asset on alice (direct peer)
[x] create asset invoice on alice, pay with asset on bob (direct peer, opposite direction)
[x] create asset invoice on dave, pay with sats on bob (sats -> asset)
[x]  create sats invoice on bob, pay with asset on dave (asset -> sats)
[x] create asset invoice on alice, pay with sats on erin (cln sats -> asset)
[x] create sats invoice on erin, pay with asset on alice (asset -> cln sats)
[x] send asset keysend payment from alice to bob

Not Working

[x] create asset invoice on alice, pay with asset on dave (multi-hop, opposite direction after funding sending half asset amt. FAILURE_REASON_NO_ROUTE)
[x] create sats invoice on bob, pay with sats on alice (direct peer, sats -> sats. FAILURE_REASON_INSUFFICIENT_BALANCE) --> Solved by https://github.com/lightninglabs/taproot-assets/pull/962
[x] create sats invoice on dave, pay with sats on alice (multi-hop, sats -> sats. FAILURE_REASON_INSUFFICIENT_BALANCE) --> Solved by https://github.com/lightninglabs/taproot-assets/pull/962
[x] send asset keysend payment from alice to dave (no_route)
 [WRN] CRTR: Failed to find route for payment<PHONE_NUMBER>ce128cf60f1f807c200e78f4b5323f2c58f867870d4b7fb295b1da: destination hop doesn't understand new TLV payloads
[ ] #964
[ ] Fund a channel with a grouped asset: https://github.com/lightninglabs/taproot-assets/issues/947#issuecomment-2181026450

Let us know if there are other flows to run through!

create asset invoice on alice, pay with asset on dave (multi-hop, opposite direction after funding sending half asset amt. FAILURE_REASON_NO_ROUTE)

In this case, do all parties have enough to satisfy the normal BTC reserve?

So this can happen if the sender doesn't have a node announcement from the dest yet. In the near future, we'll start to just assume they have the bit set.

I'll try again, keeping an eye out for the node announcements. Is there anything specific I should be looking for in the logs or describegraph?

In this case, do all parties have enough to satisfy the normal BTC reserve?

Sorry, I forgot to mention that the channel was funded with push_sat=5000. The invoice payment succeed if paid by a direct peer but fails for multi-hop). Maybe this is due to the missing node announcement as well. I'll try again.

I spent some more time testing these specific failures.

send asset keysend payment from alice to dave (no_route)

So this can happen if the sender doesn't have a node announcement from the dest yet. In the near future, we'll start to just assume they have the bit set.

After noodling on this one a bit more, I'm thinking multi-hop asset->asset keysend payments over BTC channels shouldn't be possible. With an invoice payment, dave would tell carol to swap the incoming sats to the asset in order to receive on their asset channel. For a keysend payments, how would carol know what to do with the incoming payment to dave other than forward sats.
Actually, before it gets to the delivery issue, alice doesn't even know dave exists without the hophint since there's only the private asset channel between carol and dave. So dave doesn't show up in alice's view of the graph. I tested by adding a BTC channel between carol and dave. Now dave is present in alice's graph and when I sent an asset keysend payment from alice to dave, it succeeds. However, it delivers sats to dave over the BTC channel. It makes total sense to me why this happens and doesn't feel like a bug.
Correct me if I'm wrong, but I believe the original use case I was testing (asset->asset multi-hop keysend) is technically not feasible, right?

create asset invoice on alice, pay with asset on dave (multi-hop, opposite direction after funding sending half asset amt. FAILURE_REASON_NO_ROUTE)

In this case, do all parties have enough to satisfy the normal BTC reserve?

The problem with this one turned out to be that the sats channel between bob and carol didn't have enough on carol's side to make the payment in the other direction. After balancing that channel, invoice payments between alice and dave succeed consistently in both directions.
@levmi I think it's safe to remove these two or mark them as resolved in the OP.

Marked those 2 as covered!

For the direct peer sats -> sats, I think it might be the external traffic shaper getting in the way, was it a keysend payment?
Related-ish logs:
   12 2024-06-12 16:12:58.956 [DBG] CRTR: Payment cab2b64d68edef70f4632daed02d249fa73c5e1b33991a1ac55c3c7cb7300513: status=Initiated, active_shards=0, rem_value=10000000 mSAT, fee_limit=0 mSAT                                                                                11 2024-06-12 16:12:58.956 [DBG] CRTR: PaymentSession(cab2b64d68edef70f4632daed02d249fa73c5e1b33991a1ac55c3c7cb7300513): pathfinding for amt=10000000 mSAT                                                                                10 2024-06-12 16:12:58.957 [DBG] CRTR: ShortChannelID=463:1:0: external traffic shaper is handling traffic: true                                                                                9 2024-06-12 16:12:58.957 [DBG] CRTR: ShortChannelID=463:1:0: external traffic shaper reported available bandwidth: 0 mSAT                                                                                8 2024-06-12 16:12:58.957 [WRN] CRTR: Not enough outbound balance to send htlc of amount: 10000000 mSAT, only have local balance: 0 mSAT

@Roasbeef the sats -> sats payments I tested were using invoices.

I've come across another issue where litd will hang when paying invoices for a large asset amounts. I am using the flag --taproot-assets.experimental.rfq.mockoraclecentpersat=1000000 so that the mock exchange rate is 100 sats per 1 asset. The network graph is the same as above, but I minted 1,000,000 AAA assets and funded the asset channels with 400,000 AAA. I funded the sats channel with 10M sats.
I created an invoice on dave for 5,000 AAA (equivalent to 500,000 sats). When I attempt to pay this invoice on alice, the payinvoice command will hang with the output below:
litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln payinvoice -f -asset_id bc8ccb6108f132e00776d2d260997500d9223c7df7923bae71ff71257abd8f93 lnbcrt5m1pnxk9y4pp59shpvhwq7nh7nl5z30v2zy634mdmy9qauj9w6mwshhq8cq4saz4sdqqcqzzsxqyz5vqrzjq2mecn7fxjfjjgn0s284l0zpexrmq8apkakhmu93wfscpcq4dadx4xz3hkc0f2em35qqqqlgqqqqqqgq2qsp5602rr4c5r0xhqprueh5s7shn63dwlcf4c0vd6hme9mc40xt33req9qxpqysgqjrq0hmv8pvnrf5pyctnpu5fkt7jph88cndnzm6je59a7llmcaqdy09m7gfg3skhffvpceu68ff7vc6mckvc32q8wc06h8gtktcahp8spy2sjpf
Asking peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c for quote to sell assets to pay for invoice over 500000000 msats; waiting up to 60s
Got quote for 5000 asset units at 100000 msat/unit from peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c with SCID<PHONE_NUMBER>9610654351
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| HTLC_STATE | ATTEMPT_TIME | RESOLVE_TIME | RECEIVER_AMT | FEE   | TIMELOCK | CHAN_OUT        | ROUTE              |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| IN_FLIGHT  |        0.019 |            - | 62500        | 2.124 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.042 |            - | 54687.5      | 2.108 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.063 |            - | 47851.562    | 2.094 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.118 |            - | 41870.117    | 2.082 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.136 |            - | 73272.705    | 2.146 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.153 |            - | 54954.529    | 2.108 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.241 |            - | 41215.896    | 2.082 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.260 |            - | 61823.845    | 2.122 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.381 |            - | 61823.846    | 2.122 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
Amount + fee:   0 + 0 sat
Payment hash:   2c2e165dc0f4efe9fe828bd8a11351aedbb2141de48aed6dd0bdc07c02b0e8ab
Payment status: IN_FLIGHT

The command does not timeout after the default 60 seconds. I also noticed that after running this command, litd will not shutdown gracefully. Docker has to kill the process in order to stop the container.
I also created a new network then tried a direct peer payment by creating the 5,000 AAA invoice on bob. The behavior is the same in this case as well.
I have tested asset invoice payments of 1K, 2K, 3K, and 4K. Those all succeed. It hangs when I send a payment of 5K or more.
Here are the logs for all of the nodes if it is helpful.
alice.log
bob.log
carol.log
dave.log

I believe I have been able to reproduce the above issue in the litd itests by making the following changes to litd_custom_channels_test.go:

Mint 2M assets instead of 1M
Fund channel with 400K assets instead of 50K
Pay a direct peer 60K asset invoice

Note: Paying a 50K asset invoice works, but 60K and higher fails

Here's the diff of the changes I made to the itests
diff --git a/itest/litd_custom_channels_test.go b/itest/litd_custom_channels_test.go
index 63cc0ef7..01037ab9 100644
--- a/itest/litd_custom_channels_test.go
+++ b/itest/litd_custom_channels_test.go
@@ -53,7 +53,7 @@ var (
 		AssetType: taprpc.AssetType_NORMAL,
 		Name:      "itest-asset-cents",
 		AssetMeta: dummyMetaData,
-		Amount:    1_000_000,
+		Amount:    2_000_000,
 	}

 	shortTimeout = time.Second * 5
@@ -197,7 +197,7 @@ func testCustomChannels(_ context.Context, net *NetworkHarness,
 	// We need to send some assets to Dave, so he can fund an asset channel
 	// with Yara.
 	const (
-		fundingAmount = 50_000
+		fundingAmount = 400_000
 		startAmount   = fundingAmount * 2
 	)
 	daveAddr, err := daveTap.NewAddr(ctxb, &taprpc.NewAddrRequest{
@@ -410,7 +410,7 @@ func testCustomChannels(_ context.Context, net *NetworkHarness,
 	// a direct channel invoice payment with an RFQ SCID present in the
 	// invoice.
 	// ------------
-	const daveInvoiceAssetAmount = 2_000
+	const daveInvoiceAssetAmount = 60_000
 	invoiceResp := createAssetInvoice(
 		t.t, charlie, dave, daveInvoiceAssetAmount, assetID,
 	)

There's an issue which I previously reported in chat that I forgot to mention here.
After opening a channel with a capacity that's lower than the node's total asset balance, the remaining balance becomes inaccessible. tapcli assets list returns no assets and tapcli assets balance shows the asset with the pre-channel balance.

Expand for reproduction of the issue via CLI
litd@alice:/$ litcli --version
litcli version 0.13.0-alpha commit=v0.13.0-alpha-9-g980ee938aa1e7a45633216aff8775ca8d05c12aa
litd@alice:/$ tapcli --version
tapcli version 0.3.2-alpha commit=v0.3.3-0.20240609154533-2b3c60ae77a9
litd@alice:/$ tapcli assets mint --name PBUX --type normal --supply 1000
{
    "pending_batch":  {
        "batch_key":  "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3",
        "batch_txid":  "",
        "state":  "BATCH_STATE_PENDING",
        "assets":  [
            {
                "asset_version":  "ASSET_VERSION_V0",
                "asset_type":  "NORMAL",
                "name":  "PBUX",
                "asset_meta":  null,
                "amount":  "1000",
                "new_grouped_asset":  false,
                "group_key":  "",
                "group_anchor":  "",
                "group_internal_key":  null,
                "group_tapscript_root":  "",
                "script_key":  {
                    "pub_key":  "3573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886",
                    "key_desc":  {
                        "raw_key_bytes":  "031a620843e9a4a8fdfa76bd27de4d118db6e2d45b23d408802d39057109e783e4",
                        "key_loc":  {
                            "key_family":  212,
                            "key_index":  0
                        }
                    },
                    "tap_tweak":  ""
                }
            }
        ],
        "created_at":  "1718732824",
        "height_hint":  107,
        "batch_psbt":  ""
    }
}
litd@alice:/$ tapcli assets mint finalize
{
    "batch":  {
        "batch_key":  "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3",
        "batch_txid":  "19af46087e3c014a0435dc66218563231bdb4c73da9423ea681d15f51c261fda",
        "state":  "BATCH_STATE_BROADCAST",
        "assets":  [
            {
                "asset_version":  "ASSET_VERSION_V0",
                "asset_type":  "NORMAL",
                "name":  "PBUX",
                "asset_meta":  null,
                "amount":  "1000",
                "new_grouped_asset":  false,
                "group_key":  "",
                "group_anchor":  "",
                "group_internal_key":  null,
                "group_tapscript_root":  "",
                "script_key":  {
                    "pub_key":  "3573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886",
                    "key_desc":  {
                        "raw_key_bytes":  "031a620843e9a4a8fdfa76bd27de4d118db6e2d45b23d408802d39057109e783e4",
                        "key_loc":  {
                            "key_family":  212,
                            "key_index":  0
                        }
                    },
                    "tap_tweak":  ""
                }
            }
        ],
        "created_at":  "1718732824",
        "height_hint":  107,
        "batch_psbt":  "70736274ff0100890200000001cee728c17955bddb96fb686abb2dea74b50dba9690aac4cc5ed149cf6370ce0d00000000000000000002e803000000000000225120bf607a5ae2ecf3e0885fd80d0bb8d3c55c81726b9816ef14a90de3e48fee92f52b1e0f00000000002251202f3851fdcc740d758a5fd17fe3c24b0a636113966d5a3cc7792a8c06c0a965bf00000000000100de02000000000101465ab280735889735105a511dc31eaacfd60b49ca6346cb80c2b97c691a5509f0000000000fdffffff0240420f0000000000160014b71899fb99beb61413d8a1833916464b2160762fbca4f629010000001600141fad0c5c9b6959723abdc3e1d4d6c0560ce1fa0402473044022062d430d4fdfca0f3716e1b47f96d18ca38c19e17692f4ae96e4ac787c1f5a088022076c7c356ef4ea7c426db509b50e1172e80fb58c74f8e41dd6c27db202c165edd012102858ae48c2698bc5c3435e11e799637031d690e82b50c8e29176dc65374d520b36500000001011f40420f0000000000160014b71899fb99beb61413d8a1833916464b2160762f01086c0248304502210097f4e4726a42581f0c05185688eb331b10c530e6be10965f7771a5a2dd26e85e022072eb72eca3dbfe28da288575d127094d240342159ccc31c2e083010cce9feaf40121038f99fbf00e2169d3c29304b0157bf5aa0426a865ec4f2f4ece4320667fba8cb2000022020265d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf350911800000000560000800000008000000080010000000000000001052065d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf35091210765d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf35091190000000000560000800000008000000080010000000000000000"
    }
}
# 6 blocks mined here
litd@alice:/$ tapcli assets list
{
    "assets":  [
        {
            "version":  "ASSET_VERSION_V0",
            "asset_genesis":  {
                "genesis_point":  "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0",
                "name":  "PBUX",
                "meta_hash":  "0000000000000000000000000000000000000000000000000000000000000000",
                "asset_id":  "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899",
                "asset_type":  "NORMAL",
                "output_index":  0
            },
            "amount":  "1000",
            "lock_time":  0,
            "relative_lock_time":  0,
            "script_version":  0,
            "script_key":  "023573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886",
            "script_key_is_local":  true,
            "asset_group":  null,
            "chain_anchor":  {
                "anchor_tx":  "02000000000101cee728c17955bddb96fb686abb2dea74b50dba9690aac4cc5ed149cf6370ce0d00000000000000000002e803000000000000225120bf607a5ae2ecf3e0885fd80d0bb8d3c55c81726b9816ef14a90de3e48fee92f52b1e0f00000000002251202f3851fdcc740d758a5fd17fe3c24b0a636113966d5a3cc7792a8c06c0a965bf0248304502210097f4e4726a42581f0c05185688eb331b10c530e6be10965f7771a5a2dd26e85e022072eb72eca3dbfe28da288575d127094d240342159ccc31c2e083010cce9feaf40121038f99fbf00e2169d3c29304b0157bf5aa0426a865ec4f2f4ece4320667fba8cb200000000",
                "anchor_block_hash":  "22eb84a9ef36d099dfe4de67fbf2f73bd2fc91e408105c6f2c4c130b43a44d7f",
                "anchor_outpoint":  "19af46087e3c014a0435dc66218563231bdb4c73da9423ea681d15f51c261fda:0",
                "internal_key":  "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3",
                "merkle_root":  "c65a298b0ec5e8a182370b2641572055c6fccede2732cb36d3ecf2c22ca0b394",
                "tapscript_sibling":  "",
                "block_height":  0
            },
            "prev_witnesses":  [],
            "is_spent":  false,
            "lease_owner":  "",
            "lease_expiry":  "0",
            "is_burn":  false,
            "script_key_declared_known":  false,
            "script_key_has_script_path":  false
        }
    ]
}
litd@alice:/$ tapcli assets balance
{
    "asset_balances":  {
        "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899":  {
            "asset_genesis":  {
                "genesis_point":  "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0",
                "name":  "PBUX",
                "meta_hash":  "0000000000000000000000000000000000000000000000000000000000000000",
                "asset_id":  "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899",
                "asset_type":  "NORMAL",
                "output_index":  0
            },
            "balance":  "1000"
        }
    },
    "asset_group_balances":  {}
}
litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln fundchannel --asset_id 4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899 --node_key 02e61ab7bd13a3df8c8bb5aca2fa976b5bbcdc88afb1c6a8c02184cd86f0fd9615 --sat_per_vbyte 10 --asset_amount 400
{
        "txid": "ac9ec6252d772dfab1fd9969113702257561b4ab4fddf5f47df3488c2854b032"
}
# 6 blocks mined here
litd@alice:/$ tapcli assets list
{
    "assets":  []
}
litd@alice:/$ tapcli assets balance
{
    "asset_balances":  {
        "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899":  {
            "asset_genesis":  {
                "genesis_point":  "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0",
                "name":  "PBUX",
                "meta_hash":  "0000000000000000000000000000000000000000000000000000000000000000",
                "asset_id":  "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899",
                "asset_type":  "NORMAL",
                "output_index":  0
            },
            "balance":  "1000"
        }
    },
    "asset_group_balances":  {}
}
litd@alice:/$ lncli listchannels
{
    "channels": [
        {
            "active": true,
            "remote_pubkey": "02e61ab7bd13a3df8c8bb5aca2fa976b5bbcdc88afb1c6a8c02184cd86f0fd9615",
            "channel_point": "ac9ec6252d772dfab1fd9969113702257561b4ab4fddf5f47df3488c2854b032:0",
            "chan_id": "125344325632000",
            "capacity": "100000",
            "local_balance": "96920",
            "remote_balance": "0",
            "commit_fee": "2750",
            "commit_weight": "614",
            "fee_per_kw": "2500",
            "unsettled_balance": "0",
            "total_satoshis_sent": "0",
            "total_satoshis_received": "0",
            "num_updates": "0",
            "pending_htlcs": [],
            "csv_delay": 144,
            "private": true,
            "initiator": true,
            "chan_status_flags": "ChanStatusDefault",
            "local_chan_reserve_sat": "1000",
            "remote_chan_reserve_sat": "1062",
            "static_remote_key": false,
            "commitment_type": "SIMPLE_TAPROOT",
            "lifetime": "106",
            "uptime": "106",
            "close_address": "",
            "push_amount_sat": "0",
            "thaw_height": 0,
            "local_constraints": {
                "csv_delay": 144,
                "chan_reserve_sat": "1000",
                "dust_limit_sat": "354",
                "max_pending_amt_msat": "99000000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 483
            },
            "remote_constraints": {
                "csv_delay": 144,
                "chan_reserve_sat": "1062",
                "dust_limit_sat": "354",
                "max_pending_amt_msat": "99000000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 483
            },
            "alias_scids": [
                "17592186044416000000"
            ],
            "zero_conf": false,
            "zero_conf_confirmed_scid": "0",
            "peer_alias": "bob",
            "peer_scid_alias": "17592186044416000000",
            "memo": "",
            "custom_channel_data": {
                "assets": [
                    {
                        "asset_utxo": {
                            "version": 1,
                            "asset_genesis": {
                                "genesis_point": "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0",
                                "name": "PBUX",
                                "meta_hash": "0000000000000000000000000000000000000000000000000000000000000000",
                                "asset_id": "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899"
                            },
                            "amount": 400,
                            "script_key": "0250aaeb166f4234650d84a2d8a130987aeaf6950206e0905401ee74ff3f8d18e6"
                        },
                        "capacity": 400,
                        "local_balance": 400,
                        "remote_balance": 0
                    }
                ]
            }
        }
    ]
}
litd@alice:/$

I've come across an issue where litd will hang when paying invoices for a large asset amounts. I am using the flag --taproot-assets.experimental.rfq.mockoraclecentpersat=1000000 so that the mock exchange rate is 100 sats per 1 asset. The network graph is the same as above, but I minted 1,000,000 AAA assets and funded the asset channels with 400,000 AAA. I funded the sats channel with 10M sats.
I created an invoice on dave for 5,000 AAA (equivalent to 500,000 sats). When I attempt to pay this invoice on alice, the payinvoice command will hang with the output below:
litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln payinvoice -f -asset_id bc8ccb6108f132e00776d2d260997500d9223c7df7923bae71ff71257abd8f93 lnbcrt5m1pnxk9y4pp59shpvhwq7nh7nl5z30v2zy634mdmy9qauj9w6mwshhq8cq4saz4sdqqcqzzsxqyz5vqrzjq2mecn7fxjfjjgn0s284l0zpexrmq8apkakhmu93wfscpcq4dadx4xz3hkc0f2em35qqqqlgqqqqqqgq2qsp5602rr4c5r0xhqprueh5s7shn63dwlcf4c0vd6hme9mc40xt33req9qxpqysgqjrq0hmv8pvnrf5pyctnpu5fkt7jph88cndnzm6je59a7llmcaqdy09m7gfg3skhffvpceu68ff7vc6mckvc32q8wc06h8gtktcahp8spy2sjpf
Asking peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c for quote to sell assets to pay for invoice over 500000000 msats; waiting up to 60s
Got quote for 5000 asset units at 100000 msat/unit from peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c with SCID<PHONE_NUMBER>9610654351
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| HTLC_STATE | ATTEMPT_TIME | RESOLVE_TIME | RECEIVER_AMT | FEE   | TIMELOCK | CHAN_OUT        | ROUTE              |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| IN_FLIGHT  |        0.019 |            - | 62500        | 2.124 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.042 |            - | 54687.5      | 2.108 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.063 |            - | 47851.562    | 2.094 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.118 |            - | 41870.117    | 2.082 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.136 |            - | 73272.705    | 2.146 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.153 |            - | 54954.529    | 2.108 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.241 |            - | 41215.896    | 2.082 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.260 |            - | 61823.845    | 2.122 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.381 |            - | 61823.846    | 2.122 |      412 |<PHONE_NUMBER>26432 | bob->carol->03eb01 |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
Amount + fee:   0 + 0 sat
Payment hash:   2c2e165dc0f4efe9fe828bd8a11351aedbb2141de48aed6dd0bdc07c02b0e8ab
Payment status: IN_FLIGHT

The command does not timeout after the default 60 seconds. I also noticed that after running this command, litd will not shutdown gracefully. Docker has to kill the process in order to stop the container.
I also created a new network then tried a direct peer payment by creating the 5,000 AAA invoice on bob. The behavior is the same in this case as well.
I have tested asset invoice payments of 1K, 2K, 3K, and 4K. Those all succeed. It hangs when I send a payment of 5K or more.
Here are the logs for all of the nodes if it is helpful. alice.log bob.log carol.log dave.log

I think the above bug is solved by https://github.com/lightninglabs/taproot-assets/pull/959
I'll aim to get it merged and then bump the litd staging branch so that we can test once more.

Closing this issue as incomplete tasks have been rolled into their own issues.


new abbreviations added

Closes #
What new changes did you make? Tick all applicable boxes

[x] Added new abbreviation
[ ] Fixed something in the source code
[ ] Added a new feature
[ ] Fixed the docs (README.md, CONTRIBTUING.md, etc)

Describe the new changes you added.

Share a screenshot of new changes

good ones

Thank you!!

Hey there! Thank you so much for contributing to Abbreve. You've made some stellar additions 💣
I see that you may have committed changes to the package-lock.json. This may cause issues for the main branch later. To revert, you can check out this guide

Alright, I will do that. Thanks

Hi, I have revert the commit on the package-lock.json.
I hope it is okay?


VotingRegistry: Add voting systems

To future proof the abi/interface and table schemas so that we don't have to do any complicated migrations if we want to support more voting systems in the future

@sanderpick could you create a new staging deployment of the contract & all tables once this PR is approved and merged (feel free to merge if you approve it so that you don't have to wait for me merging it)? I wanna do some final testing in the staging env + create a proposal that more people can try voting on to see that everything works correctly. Related to that, can you grant my dev wallet insert permissions on the ft_rewards table for the staging deployment? That way I can setup a bunch of test wallets that anyone that wants to try voting can use that has some mumbai matic and FT, much easier to give ppl FT using the rewards table vs minting rigs & piloting


PathArray always absolute?

When I'm reading through the documentation the array property of a path element should return the segment's of it's pathdata. It also states that it returns it with the original command types:

Original syntax is M0 0 or m0 0. The svg.js syntax ['M',0,0] or ['m',0,0].

But when I use it it always seems to return the absolute versions, it transforms m into M and translates it's values into absolute values.
This is confirmed by the source:
https://github.com/wout/svg.js/blob/1286e3de26baa3d75ec6a6eafbb2eaa11305c2e1/src/patharray.js#L137
It's not really a problem for me, it just feels like the documentation is confusing or wrong.

I know this is a closed and old issue, but what is the best way to scale (increase size, or decrease size) of a path?
I am trying to avoid nesting SVGs since Adobe Illustrator does not support them (the goal is to create art which illustrator can read).
Moving works.  Is there a a method for scale?

Despite the fact, that this is the wrong topic for this issue and you should use Stackoverflow for plain questions:
Have a look here: https://github.com/wout/svg.js#size
And here: https://github.com/wout/svg.js#scale
If you want plain a plain svg without any nested svg or groups find this: https://github.com/wout/svg.js#ungroup--flatten


Camera 180 degrees rotation problem.

Hi All,
when camera is set to be rotated by 180 degrees then when you move camera to the right side, the picture moving is inverted. I think when you set camera to be rotated by 180 degrees, picture should be inverted horizontally.

Thank you for your report I will look into this as soon as I can

Resolved here:
https://github.com/CodyTolene/Flipper-Zero-Camera-Suite/pull/56
Thanks @4k3or3et!


Don't return 404 when Riak is unavailable

This is sort of in-anger change. Error reasons like <<"pr_unsatisfied,2,0">> lead to 404 HTTP status code which should be 500 or 503. This commit lets error response modules handle these miscellaneous errors.

Fake 403 in auth phase errors are badly annoying behaviour for end users also. +1 for right direction to respond 5xx :clap:
Preliminary r_t run showed 4 test failures:
put_copy_test-cs_multi_backend              : fail
list_objects_v2_test-cs_multi_backend       : fail
legacy_s3_rewrite_test-cs_multi_backend     : fail
external_client_tests-cs_multi_backend      : fail

As for title, s/404/403/ ?

The changes of this PR will conflict (logical sense) with https://github.com/basho/riak_cs/pull/1273 .
It's probably necessary to rebase on master branch.

This will address #756.

Rebased branch in #1298.


Win 11 Windows Control Overlay Stopped Working With 1.71.2

Does this issue occur when all extensions are disabled?: Yes/No
Yes it does

VS Code Version: 1.71.2
OS Version: win 11

Windows 11 adds the "SnapLayouts" feature to the maximize/restore button in the titel-bar of any window. When you hover over the button the flyout appears and let's you choose a window layout. The feature can also be activated by pressing Win + z on the keyboard.
The flyout worked as expected in 1.71.1 it started not to appear with 1.71.2 the keyboard shortcut still works.
[All the Snap-Layout features are enabled systemwide on my machine, and other windows still show the flyout as expexted.]

We call it the window controls overlay (WCO). The WCO was disabled this iteration because of https://github.com/microsoft/vscode/issues/159813. That issue has been closed, but it was closed by disabling the feature. We're currently investigating a longer-term solution to enable the WCO without causing that regression again. See PR https://github.com/microsoft/vscode/pull/159958 for details.

The described behavior is how it is expected to work. If you disagree, please explain what is expected and what is not in more detail. See also our issue reporting guidelines.
Happy Coding!

Actually, since #159813 is closed, I can keep this issue open to track enabling the WCO again.

Verification steps:

On Windows 11 with VS Code Insiders, enable the window controls overlay setting in VS Code, reloading the program if prompted.
Hover over the restore/maximize button.
A layout should appear showing tiling options for the window.


Support for pyhton3?

I hope this will be possible in the near future!

Here is a python3 version.
infogram_api_3.zip

Sorry, I've been away and with limited time for implementing & testing this, but @AntonMu has implemented it! If you want, feel free to open a PR to include this here :)


[EVM] Make account basic trait

Desc
In this PR, Make the account_basic, mutate_account_basic into one trait names AccountBasicMapping.
The reason
Recently, in order to utilize ethereum development tools, such as metamask, remix, etc. we want to achieve this via integrate frontier and evm-pallet. However, we found that the balance model in our project is different from ethereum, which will cause the balance to be displayed in metamask tool incorrectly.
To support handle such difference in runtime, make AccountBasicMapping as an associated type of pallet-evm.
For more info, please refer to:
https://github.com/darwinia-network/darwinia-common/issues/339

AccountBasic is supposed to be only internal representations right now. The actual balance is handled by an actual currency module. The other thing is that the new "native runner" will also not need any mutations of account basic, so part of what this PR will not apply.
@AsceticBear is the problem only related to balance decimals? Then maybe we can just fix that. Do you think this would be better?

AccountBasic is supposed to be only internal representations right now. The actual balance is handled by an actual currency module. The other thing is that the new "native runner" will also not need any mutations of account basic, so part of what this PR will not apply.
@AsceticBear is the problem only related to balance decimals? Then maybe we can just fix that. Do you think this would be better?

As far as I have known, this problem only exists when using eth-related balance manipulation tools. I read the source code of native runner and evm. I guess that native runner and stack runner will coexist for a long time in the future, and users can make choices according to their needs.  Is it possible the change of this pr can only be applied to stack runner?

@AsceticBear Are you using Frontier or just pallet-evm?

@AsceticBear Are you using Frontier or just pallet-evm?

We use a forked Frontier with substrate master pallet-evm crate now.

@AsceticBear Okay I see the problem.
Would you mind to change this PR to only operate on balance instead of AccountBasic. In other words, create an BalanceMapping trait that converts BalanceOf<T> to and from U256. Then library users can customize this trait to define the decimal to be whatever is needed.
Or if you don't mind, I can write a separate PR of what I meant by this, but please let me know if it's okay for you!

@sorpaas In addtion to the problem of convert the balance type between u128 and U256, there is actually another issue behind it. How to deal with the remaining decimal after mod conversion. Obviously,  directly discarding is an inappropriate approach. We also need to open up storage to store this part of the data. So,  I think that just providing a balance conversion trait can not completely solve this problem.

@AsceticBear Why directly discarding it being an inappropriate approach? We only lost the reminder, right? Those funds will just not be accessible from the EVM. That is the only consequence.

Ah I see, you mean if the EVM has higher decimal than Substrate.

Ah I see, you mean if the EVM has higher decimal than Substrate.

Yes. Our substate chain balance type is u128(maybe many chain as well). EVM use u256.

@AsceticBear What is your EVM decimal settings and Substrate decimal settings right now?

@AsceticBear What is your EVM decimal settings and Substrate decimal settings right now?

The EVM account is 18(the same as ethereum), the substrate account is 9.
An example of this:
If one evm account in metamask with balance 123_456_789_000_000_009, However,  the corresponds substrate account balance displayed in substrate balance system is 123_456_789

@AsceticBear The only two way I can think of to properly solve this issue is to:

Use native runner, so that you can customize the transfer function. Either define it to reject without 9-zeros, or to truncate the value.
Modify the EVM decimal.

It's honestly quite difficult to do this with the stack runner.

Maybe we can deep more into this issue and have more discussion.

Use native runner, so that you can customize the transfer function. Either define it to reject without 9-zeros, or to truncate the value.

impl<'vicinity, 'config, T: Trait> Handler<'vicinity, 'config, T> {
       ...
      fn transfer(&self, transfer: Transfer) -> Result<(), ExitError> {
		let source = T::AddressMapping::into_account_id(transfer.source);
		let target = T::AddressMapping::into_account_id(transfer.target);

		T::Currency::transfer(
			&source,
			&target,
			transfer.value.low_u128().unique_saturated_into(),
			ExistenceRequirement::AllowDeath,
		).map_err(|_| ExitError::OutOfGas)
	}
}

According to the current native runner implementation in the substrate-frontier branch, if we want to customize our own transfer function,  we still need a fork, am I right? It would better if it could be customized without fork.

Modify the EVM decimal.

If change the EVM decimal,  it will not be compatible with the tools of ethereum, like the remix, matamask. This probably not a good idea for this problem.


[ZEPPELIN-550] Let interpreter define function for angular js

What is this PR for?
This allow interpreter define a function that callable from front-end.
What type of PR is it?
Feature
Todos

[x] - implement AngularFunction
[x] - add tests
[x] - add docs

Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-550
How should this be tested?
Check screenshot
Screenshots (if appropriate)

Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

@Leemoonsoo Closed?

This PR is fully functioning with test and docs. However, i think https://github.com/apache/incubator-zeppelin/pull/591 provides much simpler way to use angular display system.
While this PR still adds some complexity into the code base, i'd like to not merge this code.
What do you think?

@Leemoonsoo I think #591 is easier indeed.
Just asking in case closing it was a mistake :)


outdated ccxt version

It would be nice if we have the option to update ccxt as the version used by ccxt-rest is outdated

Updated the ccxt version in https://github.com/franz-see/ccxt-rest/pull/16


support for elm 0.17.x

the title says it all, anybody working on this already?

Just came across some spare beach time on the Mediterranean, so I'll be working on it over the weekend.

Fixing this should fix #32.

great, enjoy the 🏖

I'm going to keep a bit of a journal of the process here, both to help others that may have to do a port like this in the future, and certainly to keep things straight in my head right now.

@seliopou I started trying to do some of the migration myself... but I was having difficulty turning signals .
There was also some conflict as to using Native code (as you have chosen) or Ports.

@seliopou any progress?

I was hoping to leverage your awesome library to showcase Elm 0.17 with some work related stuff. Do you think you could provide an ETA on this? I'd prefer the library rather than peddle with ports :)

Sorry for the radio silence. I did some digging into the current state of Elm's native interfaces, and the situation is not great with respect to a speedy port to 0.17. Essentially, everything about the runtime has changed, which means that this will likely be a significant lift. That will be a bit of coding for sure, but but even before getting to that I have to identify how surface native events to application code, and also providing a custom rendering function. I'll look a bit more tonight, but being realistic here, I wouldn't expect a quick turnaround.
If anybody has any pointers, I'm all ears.

Thank you for the update - helps me prioritize stuff on my end.

@seliopou Can you outline the "Elm's native interface" changes that affect elm-d3? I.e. what are the tasks required to migrate up?

@hoelzro wrote a great article about a migration from 0.16 to 0.17
@jadbox, I think @seliopou is referring to how elm-d3 uses the JavaScript files located in this directory
It seems that the native API changes very quickly and it is not recommended

As long as the work is more or less mechanical / testing I can also help out.

@francisdb the mechanical part (syntax changes, dependency versions update) was already done by @MonsieurCactus, as I see, these are the steps required:

Change the native code to ports (requires understanding of both)
Change the uses of Signal to Commands (this was a conceptual change introduced in 0.17)

0.18?

Looking to use elm + d3 for a project i'm about to begin work on... Is there any status update on this?
I'm also happy to help w/ any mechanical / testing work to be done.

@clevinson what would you like to see in a port from Elm to D3?  Elm is not at v0.18 and D3 is not at v4.   Therefore I might open a new issue.
I noticed I am approximate many of D3's canvas and SVG features purely within Elm.  Not 100% of systematically.  I have considered going back to work on this.


macOS environment versioning

Describe the bug
I've got a concern with the latest update to macos-latest environment:
How do I version which environment I'm using? Effectively I don't see any docs on the new macos-latest. I assume since it's catalina then it would be macos-10.15, but it seems it can only be referenced as macos-latest. I'd like to have an option to use the same environment in the future to have reproducible builds instead of chasing bugs that are related to GitHub updates of the environment.
For example a gentleman reached out to me via action's issues.
The issue is that he's using the macos-latest environment and he mentions that tr command is not available in his version of the macos environment. When I try to do the same I don't see any problem. I suspect that latest is now pointing to the new version. This is prone to random build fails because obviously we can't test all of the env snapshots against all the possible actions on github. So having a way to specify an exact snapshot (git sha of the source version of the snapshot since it's probably automated?) would be awesome.
Virtual environments affected

[x] macOS 10.15
[ ] Ubuntu 16.04 LTS
[ ] Ubuntu 18.04 LTS
[x] Windows ?

Expected behavior
I expect to have a predictable environment version that gives reproducible builds
Actual behavior
Uncontrollable changes of environment lead to confusion of action users and end-users of actions, inability to debug issues and bad experience overall.

Use macos-latest
Wait until there is invisible change to the environment
Find something broken
Another change
Try to reproduce. Find another issue
Repeat

This does feel like an already solved problem. Docker for example allows latest tag, but it's highly discouraged from actual usage. The only 2 concerns that I see here are:

Increased storage usage for all the different environments
Harder to have a pre-warmed instances for execution for all the different versions of environments there might be

Thanks,
Anton

Huge +1, see also #78 which is bit related to this.

+1
See also https://github.com/actions/virtual-environments/issues/79.

There is a similar issue for Windows, windows-2016 was deprecated in https://github.com/actions/virtual-environments/issues/68, but for some reason also windows-2019 disappeared from the docs, and now only windows-latest is available.

+1

Adding another use case for this: There are quite a lot of tools (take e.g. PyInstaller) that require you to compile with the oldest OS version that needs to be supported.
If GitHub Actions only supports -latest, we can't serve users on older yet still supported macOS versions.

Hmm, the docs say macos-latest is macOS Catalina 10.15 but it looks like Python 3.5-3.8 are still on macOS 10.13 High Sierra and only PyPy3 is on macOS 10.15 Catalina.
From checking the logs of a recent run:
$ grep -ril "macOS.*10.13" .
./10_macOS-latest Python 3.5.txt
./macOS-latest Python 3.5/9_Test.txt
./macOS-latest Python 3.5/8_Build.txt
./macOS-latest Python 3.5/12_After success.txt
./7_macOS-latest Python 3.8.txt
./8_macOS-latest Python 3.7.txt
./macOS-latest Python 3.7/9_Test.txt
./macOS-latest Python 3.7/8_Build.txt
./macOS-latest Python 3.7/7_Install macOS dependencies.txt
./macOS-latest Python 3.7/12_After success.txt
./macOS-latest Python 3.8/9_Test.txt
./macOS-latest Python 3.8/11_Docs.txt
./macOS-latest Python 3.8/8_Build.txt
./macOS-latest Python 3.8/7_Install macOS dependencies.txt
./macOS-latest Python 3.8/12_After success.txt
./macOS-latest Python 3.6/9_Test.txt
./macOS-latest Python 3.6/8_Build.txt
./macOS-latest Python 3.6/7_Install macOS dependencies.txt
./macOS-latest Python 3.6/12_After success.txt
./9_macOS-latest Python 3.6.txt

$ grep -ril "macOS.*10.14" .

$ grep -ril "macOS.*10.15" .
./6_macOS-latest Python pypy3.txt
./macOS-latest Python pypy3/9_Test.txt
./macOS-latest Python pypy3/8_Build.txt
./macOS-latest Python pypy3/12_After success.txt

Please could someone confirm what is what and make it clearer in the docs and logs?

Hey everyone, thanks for the feedback! I'm one of the Product Managers on the Actions team. I agree this is inconsistent. Part of the inconsistency is in the product, and part of it is in the docs.
In the product, we do not support using macos-10.15 as a label to reference the macOS virtual environment. We're going to get that fixed. We do, however, support the use of the label windows-2019 to reference the windows virtual environment.
In the docs, the only version label for a latest environment we document is ubuntu-18.04. We'll update the docs to make sure it's clear you can use windows-latest or windows-2019, and that you can use macos-latest or macos-10.15.

@github-actions Please keep open

Related: for some reason recent macOS build started failing whenever I try to use jq. Created issue here: https://github.com/actions/virtual-environments/issues/526
Would be great to have versioning to rule out github updates from being a source of build failures.

@kceb we've already implemented the versioning! You can find an image version in the $ImageVersion environmental variable.

@miketimofeev that's awesome to hear! now that we have visibility, how do we request a specific environment version for our builds?

@Malinskiy unfortunately, only one version is available at the moment. This variable helps us identifying issues

@miketimofeev I noticed there are multiple versions used by my runner, is it possible I can lock to a version? ex: lock to runner version: 2.165.2?
I believe it is currently 2.168.x, but I noticed this recent version for some reason slows down certain steps in my build 5x

@miketimofeev I noticed there are multiple versions used by my runner, is it possible I can lock to a version? ex: lock to runner version: 2.165.2?
I believe it is currently 2.168.x, but I noticed this recent version for some reason slows down certain steps (like the setup step) in my build 5x. When macOS build time is so expensive, it's worth optimizing wherever possible.

Hey @kceb, to be clear the runner version actually refers to the small client that runs in the VM and manages the workflow. The code for the runner is at actions/runner. The virtual-environment version is actually in the ImageVersion variable. That's the one that indicates what version was build from this repo. It's a bit confusing since the hosted runners run the runner but hope this helps 😄. Thanks!

I came to this issue because I found that the only two values accepted are "macos-10.15" and "macos-latest", which are both are same thing.
I run tests with Selenium 3 on Safari 11, which in turn requires "macos-10.13" to exist. Running on "macos-10.15" means I'd have to use Selenium 4 with Safari 13, but all my other tests would still have to use Selenium 3.
That's inconvenient, but next to that it also means I'm not able to test any older browsers that I still need to support.
I guess it's a logistics issue, but for me this is extremely important. The only option I currently have is to contact Browserstack and ask them for an open source license, so that I can run tests on Safari 11 with their servers as host and github actions as client. And while Browserstack is awesome, it does introduce a performance penalty by having to send literally everything over the internet.

Hello @StephanBijzitter ,
Currently,

GitHub Actions supports only macos-10.15
AzDO supports macos-10.14 and macos-10.15

Unfortunately, we don't have plans to add macos-10.13 because it was removed (deprecated) from Azure DevOps recently (when 10.15 was added)

Hello everyone, I am going to close this issue.
Currently, the following macOS images are available:

GitHub Actions supports only macos-10.15 and macos-11.0
AzDO supports macos-10.14 and macos-10.15

We update every image on weekly basis and for now, there is no way to freeze specific image version and avoid such updates.
Please let us know if you have any additional questions