Abandoned channel (later force closed by peer) settled balance not showing up in LND wallet

Background
Hi,

Oct 21: I opened a channel with WoS, edge here: https://amboss.space/edge/705994x2429x1
Oct 23: After aprox 1-2 days, my node was not able to set a policy for the channel. So decided to abandon the channel with lnd lncli abandonchannel --funding_txid 9a75c78fb9d1925555bfb6ba8f4b678091111df6baf1f0b6c999a140adc85ad1 --output_index 1 --i_know_what_i_am_doing
Contacted WoS to make them aware and ask for a Force Close on their side.
Oct 24: WoS force closed channel on their side succesfully. Closing Tx id confirmed Note: As I abandoned the channel my node didn't know about the closing Tx but got it from peer.

I waited 3 weeks expecting the settled amount to reach my LND wallet, but no Tx related to this closed channel can be seen in my node wallet. So the whole funding amount is lost.

Nov 9: I have restarted the node with reset-wallet-transactions=true" on lnd.conf. No luck.

Peer (WoS) has been very cooperative and we have been assessing the issue together, we conveyed it was the time to rise the issue to LND team as we don't have a clue on what's going on. Apparently my LND wallet cannot recognize the receiving address of the closing Tx or it cannot scan properly past events.
Side note: it could be that my node was offline shortly when the closing Tx got confirmed, but if it was it was for less than 5 minutes.
Your environment

version of lnd: v0.13.3-beta
which operating system: umbrel on pi, 0.4.6
version of btcd, bitcoind, or other backend: 22
any other relevant environment details: probably not related but on same period this happened I have been experiencing strange force closes from active channels, either triggered by my node or the other side with no apparent reason for forcing it.

Steps to reproduce
I really don't want to reproduce it again :=) but I can extract logs or provide configuration details by request if needed.
Links to opening and closing Tx provided above.
Expected behaviour
My node LND wallet should have received the settled amount already as per confirmed Tx but is not there or my LND wallet cannot "see it".
Actual behaviour
Missing ~4Msats from settled onchain tx, closing Tx not showing in Tx list. Funds missing.
Thanks in advance,

Abandoning a channel means giving up on it. It really should only be used if the funding transaction never made it to the chain. Did you read the help text? I assume yes, since you provided the --i_know_what_i_am_doing flag.
lnd will never pick up those funds automatically since it removed all information about the channel. You'll need to use chantools genimportscript --lndpaths to derive the private keys for the funds.

Thanks for the prompt reply @guggero , please don't blame me for using the --i_know_what_i_am_doing flag...., clearly I was not enterly sure about the full effects of doing it, but doc mentioned something about it keeping track of closures still, so I went ahead. won't do again. Lesson learnt.
Thanks for the tip, I'll go through it carefully.
Just a couple of questions,

what's the difference between reset-wallet-transactions=true I did recently and what chantools will do?
Considering my environment (umbrel), do I need to take any precaution before running chantools genimportscript --lndpaths (currently going through chantool docs).

Thx again!

The command keeps a summary of the closed channel (what the funding TX and the remote peer was) but lnd won't look at the chain anymore for an abandoned channel. Maybe we should make the doc even clearer.
Currently lnd doesn't track the derivation path where those remote force closed funds go to automatically. Only when you use an SCB. That's why reset-wallet-transactions will not pick them up either. This is a bug and should eventually be fixed.
chantools just generates a list of all private keys, including the derivation path that lnd doesn't look at automatically. You'll need to import the private key into electrum or something similar to get the funds out.
You can install chantools on your local desktop computer, it doesn't need to be in umbrel. But it will ask you for your seed, so make sure you have that ready.

Okay, thanks for the clarity. I think I got it now, let's try step by step. I'll return with the results (or more questions..).
thx a million

Ok, unfortunatelly it didn't work..., what I did:

Generated the private keys with chantools genimportscript --lndpaths --format electrum. Went OK.
Imported keys into Electrum. Electrum desktop connected to my node via Tor. Sync went OK, I can see all Tx for the wallet.

But...

The missing Tx or receiving address did not appear, and the balance is the same I can see in the node's LND wallet.

Should I consider this funds lost forever?  would it be useful if I send some sats to that address? would that make the node recognize the address and maybe scan back for more Tx's¿¿
You mentioned there was a bug causing reset-wallet-transactions to not pick up these tx. Should I wait till this is solved? Do you have link to that bug report?
Totally lost at this point. Any further tips or actions I can take are really appreciated. Would you please reopen the case if possible as this is not solved.
Thx

Attached what electrum can tell about the Tx in question after importing the keys from chantools output.

"Transaction unrelated to your wallet"
LockTime: 545278228 (1987-04-13 04:10:28) ?¿

Ah, it was an anchor channel. There the funds are in a script output, not a simple pubkey hash output.
You'll need to run chantools sweepremoteclosed --recoverywindow 100 --feerate 5 --sweepaddr <some_address_of_your_wallet_to_send_to> --publish to get the funds out.

This is what I get after running chantools sweepremoteclosed --recoverywindow 100 --feerate 5 --sweepaddr <some_address_of_your_wallet_to_send_to> --publish and entering the seed words:

Error: found 0 sweep targets with total value of 0 satoshis which is below the dust limit of 600

found it! previous commend ran well but with a --recoverywindow 500  since I hade some more than 100 onchain Txs. Was published succesfully and it inmediately appeared on the node wallet.
 Thanks!

Cool! Wasn't sure how many channels your node had in its lifetime, so apparently 100 was too small.
No tip necessary. But I'm always happy about a GitHub star on the chantools repo ;-)


Glossary page: FQDN

Glossary entry for fully qualified domain name.
Description of the change:
Reason for the change:
Link to original source:

Closes #

Hello @hmbrown64,Thanks again for the PR! It's HACKTOBERFEST! We want to show our appreciation by sending                                              you some special Hacktoberfest swag. If you have not already, could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


Can't throw logs inside AsyncTask

I have logging properly configured and I can write yii::trace logs throughout my application. But inside the execute method of my asynctask, everything looks fine, the jobs are being executed, but any log I throw doesn't get written to log file.
Here's my confs, I have placed the same conf both inside common, console, and frontend just to ensure that configuration is not the case (I have already mentioned that I can write to log file when throwing logs on controllers, for example)
        'log' => [
            'targets' => [
                [
                    'class' => 'yii\log\FileTarget',
                    'levels' => ['error', 'warning'],
                ],
                [
                    'class' => 'yii\log\FileTarget',
                    'levels' => ['trace'],
                    'categories' => ['push'],
                    'logFile' =><EMAIL_ADDRESS>
                    'maxFileSize' => 1024 * 2,
                    'maxLogFiles' => 50,
                    'exportInterval' => 1
                ],
                [
                    'class' => 'yii\log\FileTarget',
                    'levels' => ['trace'],
                    'categories' => ['queue'],
                    'logFile' =><EMAIL_ADDRESS>
                    'maxFileSize' => 1024 * 2,
                    'maxLogFiles' => 50,
                    'exportInterval' => 1
                ],
            ],
        ],

And here's my execute command inside AsyncTask child class:
class NotificationQueue extends AsyncTask
{
    public $args;

    public static $queueName = "notifications";

    public function execute()
    {
        try {
            Notification::sendNotification($this->args);
            if (YII_ENV_DEV) {
                \Yii::trace(print_r($this->args,true),'queue');
            }
            return true;
        } catch(\Exception $e){
            \Yii::trace($e->getMessage(),'queue');
            return false;
        }
    }

I'm not sure if this happens because inside the asynctask yii can't figure out what is the context being executed, I assume is console?
Can anyone throw a bone on this one?

I've figured out what was missing...


upgrading to 3.5.0

Issue describing the changes in this PR
resolves #issue_for_this_pr
Pull request checklist

[x] My changes do not require documentation changes

[ ] Otherwise: Documentation issue linked to PR

[x] My changes should not be added to the release notes for the next release

[ ] Otherwise: I've added my notes to release_notes.md

[x] My changes do not need to be backported to a previous version

[ ] Otherwise: Backport tracked by issue/PR #issue_or_pr

[x] I have added all required tests (Unit tests, E2E tests)

Additional information
Additional PR information

/check-enforcer evaluate

@shrohilla Changes look good, but would it be possible to update the PR and commit descriptions to clarify this is an update to Kafka? Thanks!

@shrohilla Changes look good, but would it be possible to update the PR and commit descriptions to clarify this is an update to Kafka? Thanks!

@fabiocav Updated in the description section

@shrohilla Can you make a copy of release_notes.md from https://github.com/Azure/azure-functions-dotnet-worker/tree/main/sdk location and paste it in the root of the kafka extension and update the release notes as needed(Include information about this 3.5.0 release).

@shrohilla Changes look good, but would it be possible to update the PR and commit descriptions to clarify this is an update to Kafka? Thanks!

@fabiocav Updated in the description section

Could you please update the PR title as well to include the extension information?

Please update the PR title to be more elaborate.

Done

@shrohilla Changes look good, but would it be possible to update the PR and commit descriptions to clarify this is an update to Kafka? Thanks!

@fabiocav Updated in the description section

Could you please update the PR title as well to include the extension information?

Done

@shrohilla Can you make a copy of release_notes.md from https://github.com/Azure/azure-functions-dotnet-worker/tree/main/sdk location and paste it in the root of the kafka extension and update the release notes as needed(Include information about this 3.5.0 release).

Added that please share your comments in-case it's not upto the mark

/check-enforcer evaluate

/check-enforcer evaluate


Unable to resolve dependency

Unable to resolve dependency for ':app@debug/compileClasspath': Could not resolve com.github.PierfrancescoSoffritti:SlidingDrawer:0.11.

Try with com.github.PierfrancescoSoffritti:SlidingDrawer:0.12


[Feature] Support @staticmethod and @classmethod

We need to add a flag to functions to support @staticmethod and @classmethod.

For some reason, this issue is a bit hard to be solved. I will postpone the impl of this feature.


NestedScrollView take a extra space from the bottom with TabBarView

Steps to reproduce

Create TabBarView with NestedScroll view, in one the tabs add Container with short height
Try to scroll down

Expected results
Widget with dynamic height
Actual results
Widget with dynamic height, but with extra space
Code sample
Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: false,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: DefaultTabController(
        length: 2,
        child: NestedScrollView(
          headerSliverBuilder: (context, _) {
            return [
              SliverOverlapAbsorber(
                handle:
                    NestedScrollView.sliverOverlapAbsorberHandleFor(context),
                sliver: SliverSafeArea(
                  top: false,
                  sliver: SliverToBoxAdapter(
                    child: Column(
                      children: [
                        Container(
                          height: 100,
                          color: Colors.red,
                        ),
                        Container(
                          height: 100,
                          color: Colors.deepOrange,
                        ),
                        Container(
                          height: 100,
                          color: Colors.green,
                        ),
                      ],
                    ),
                  ),
                ),
              ),
            ];
          },
          body: Column(
            children: [
              const TabBar(
                tabs: [
                  Text(
                    'Tab 1',
                    style: TextStyle(color: Colors.black, fontSize: 20),
                  ),
                  Text('Tab 2',
                      style: TextStyle(color: Colors.black, fontSize: 20)),
                ],
              ),
              Expanded(
                child: TabBarView(
                  children: [
                    Builder(builder: (context) {
                      return CustomScrollView(
                        physics: const NeverScrollableScrollPhysics(),
                        slivers: [
                          SliverOverlapInjector(
                            handle:
                                NestedScrollView.sliverOverlapAbsorberHandleFor(
                                    context),
                          ),
                          SliverList(
                            delegate: SliverChildListDelegate(
                              [
                                Container(
                                  height: 300,
                                  color: Colors.red,
                                ),
                              ],
                            ),
                          )
                        ],
                      );
                    }),
                    Builder(builder: (context) {
                      return CustomScrollView(
                        physics: const NeverScrollableScrollPhysics(),
                        slivers: [
                          SliverOverlapInjector(
                            handle:
                                NestedScrollView.sliverOverlapAbsorberHandleFor(
                                    context),
                          ),
                          SliverList(
                            delegate: SliverChildBuilderDelegate(
                              childCount: 10,
                              (context, index) {
                                return Container(
                                  height: 100,
                                  color: Color.fromRGBO(
                                    index + 10,
                                    index + 20,
                                    index + 30,
                                    1,
                                  ),
                                );
                              },
                            ),
                          )
                        ],
                      );
                    }),
                  ],
                ),
              ),
            ],
          ),
        ),
      ),
      // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.16.3, on macOS 14.2 23C64 darwin-arm64, locale ru-US)
    • Flutter version 3.16.3 on channel stable at /Users/donierbek/Documents/flutter_version/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b0366e0a3f (6 weeks ago), 2023-12-05 19:46:39 -0800
    • Engine revision 54a7145303
    • Dart version 3.2.3
    • DevTools version 2.28.4

[!] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/donierbek/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 15.1)
    • Xcode at /Users/donierbek/Downloads/Xcode.app/Contents/Developer
    • Build 15C65
    • CocoaPods version 1.14.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] Connected device (3 available)
    • iPhone 15 Pro (mobile) • B9A238C4-D56C-43EF-87E9-9F5F7C8F07AC • ios            • com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 14.2 23C64 darwin-arm64
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 120.0.6099.234
    ! Error: Browsing on the local area network for iPhone (Aziz). Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Thanks for filing this @denihero.
I can reproduce as in screenshot shared but this seems to me to be WAI, can you share more information preferably screenshots/screenrecords on the expected behaviour?


Prompt-Image correspondence

In order to add a new prompt in test_prompts.txt, I need to first check the alphabetic order of the image filename, and insert it in the corresponding line? Any better way? Thanks.

Hi. Thanks for your comments.
"I need to first check the alphabetic order of the image filename, and insert it in the corresponding line" EXACTLY.
I think a better way is to reimplement the load_prompts and load_data_prompts in inference.py and reformat the test_prompts.txt file.
For example, the prompt txt can be in the format of
"[filename]"+"[prompt]"


Azure - ID Parsing Utilities

+            resource_group = id.split('/')[4]
+            name = id.split('/')[8]

These two extractions happen periodically.  Can we get them in a common utilities class or in some appropriate base class that we have everywhere and then go bulk replace any usages?

This looks similar to some of the ARN generation that happens for AWS resources, possibly add a function in c7n.utils for that.
reference: https://github.com/capitalone/cloud-custodian/blob/master/c7n/utils.py#L314-L326

Oh yep we should just copy that model and have a utils.py under Azure plugin so with things that are not applicable to core c7n.

this is done


specrunner.html not working

specrunner.html in test folder not working.

Works for me. What error are you getting?
Just to check, did you:

Run npm install
Compile the source and set up watch with gulp test-browser
Open the spec runner in your browser

Hi, @jmeas. Thanks for quick reply. Yes, I did all this things, and ended up with this message:
"before each" hook
global.sinon is undefined
[3]</</module.exports/<@http://localhost/babel/tmp/__spec-build.js:44:9
and
"after each" hook
this.sandbox is undefined
[3]</</module.exports/<@http://localhost/babel/tmp/__spec-build.js:54:9
Probably nothinghting to worry about :+1:

Interesting, @rickardjanson . I'll take another look tomorrow to see if I can reproduce it!

The connection to ws://localhost:35729/livereload was interrupted while the page was loading.
livereload.js (line 74)
Firefox can't establish a connection to the server at ws://localhost:35729/livereload.

 I went to the store and baught a new laptop and re-installed. This was with windows. It happened again.
Then I got frustrated, so I went back to the store and bought a mac. And re-installed. And whole!
Problem seems to be with fsevents(mac). Problems happened on windows.
Very expensive experience this one, but at least. This is my conclusion.

I don't have a PC to test on, so there's not much I can do : [
If you diagnose what the problem is, do let me know. Maybe once I get around to updating the dev dependencies it will automatically fix itself.

Hi. Updating the dev dependencies will cause issues I guess. The path for sinon seems to have changed in a newer version. To the /lib folder now. And I have no idea how I solved this issue on my own, but I ended up with upgrading the packages and discovered this. And after changing the path, it works now.

@jmeas I forgot to mention... I also renamed the gulpfile.js to gulpfile-babel.js  and re-wrote it into ES6. Something new you can do with newest babel version. I have no idea if this too helped me solve the issue, but I just mention it here just in case.

Awesome! Could you do a pull request with those changes?

@megawac I have no time this week, but it's very easy task if you think about the gulpfile.
Take a look at this Twitter post: https://twitter.com/wesbos/status/608987962370424832
For the sinon issue look at this line:  https://github.com/babel/babel-library-boilerplate/blob/master/test/runner.html#L14
What I did was changing pkg to lib after upgrading to a newer version.

Gonna close this as this seems to be related to updating sinon.

Hm, I took a look, and the pkg directory is what the sinon library exports. The file in lib should not work in the context of a browser...
For this reason, I think I'm going to keep it referencing pkg in the runner, unless I'm missin' something obvious here :question:


NIFI-1180: Modify PutS3Object to enable encryption

This adds a property to set server side encryption on PutS3Object. In addition, it adds s3.sseAlgorithm to FetchS3Object so users of that processor will know if object encryption is turned on.

I did not see any tests for this new functionality. I wrote a very simple validation test for this new option here: PutS3ObjectTest.groovy
How much integration testing did you do for this feature?

@alopresto Thanks for the comments! You're right that I should write some integration tests. It worked correctly when I tested it manually, but let me get back to you.

@alopresto I looked up the inheritance chain and there aren't any other processors that follow the name/displayName convention. They all set name only. Is this convention documented somewhere? I couldn't find it in the developer guide, contributor guide, or mailing list...
Also, doesn't changing the name attribute break backwards compatibility for existing attributes? The name field is used in flow.xml.gz, so changing it would cause a processor to "forget" the setting, unless I'm mistaken...
I pushed another commit with integration tests for the server side encryption attribute.

@adamonduty You are correct that the .name/.displayName convention is not documented well. I pinged @joewitt to look for a canonical source for the convention and we need to improve the documentation.
In this case, I am ok with not changing the existing PropertyDescriptors, as it would break backwards compatibility as you noted. Unfortunately, this is a side-effect of the attributes not being set "properly" in the first place -- the displayName value is visible to the end user through the UI and may need to be changed, while the name value is used for object resolution in the application and so should not need to change if the messaging changes (imagine future internationalization where the displayName value is language-dependent). However, I do think in this case, the new PropertyDescriptor should follow the correct convention and a comment added noting the reason for the difference with other PropertyDescriptors in the class.

When a PropertyDescriptor does not have a .displayName() set, then the .name() is used in the UI.  There was a time when .displayName() did not exist and .name() displayed in the UI, so that's why you see many standard processors that do not use displayName.  displayName was added so that you could change the name used in the UI without breaking backward compatibility in the flow.xml.
So the original comment from @alopresto represents the latest best practice for NiFi processor development.

For what its worth, I already made the change to server-side-encryption in 33805e2e7d849ec3d1cd48b4e0d63c5c7bfc0792. Sorry if that wasn't clear.

@adamonduty that looks good. I understand the confusion around this undocumented convention.
My only outstanding request would be to combine the two commits into a single commit (via rebase --interactive and push --force) to condense the PR and then one of the committers should be able to merge it. Feel free to include my linked simple validation tests as well.

@alopresto Done! Let me know if anything else needs addressed.

I do not have AWS S3 credentials to run the integration tests myself but the code looks good to me. +1

@alopresto What do you think about merging this one with your new committing superpowers? :)

@adamonduty Will do in the morning. Don't want to miss merging into both branches now that 1.0 and 0.x have split. (Provided docs on that on the wiki. ) Those commits will also be GPG-signed now that GitHub supports it.
Thanks for reminding me.

Awesome, thanks @alopresto!!


Add DebugBear

Add a link to the free DebugBear speed test.
Pull Request Template
Please ensure all items below are checked before creating a pull request:

[x] My changes were made in the resources folder, not in the auto-generated README.md file or db folder
[x] My submission is formatted according to the guidelines in the contributing guide
[x] My submission is ordered alphabetically based on the resource name
[x] My submission has a useful description
[x] I have searched the repository for any relevant issues or pull requests

Thank you!