no_docstrings not working for stdlib

Short description:
As the title says, docstrings are apparently not removed from the stdlib files even though no_docstrings is specified.
Example
Source file:
"""This docsting will be removed as expected. """

if __name__ == "__main__":
    print("Hello World!")

Nuitka Command Line:
python -m nuitka docstring-test.py --standalone --onefile --python-flag=no_site,no_asserts,no_docstrings --windows-company-name Zedia --windows-product-version 4096

Result:
The resulting executable does not contain the docstring from the source file, but it does contain docstrings from the stdlib files. One specific example I found in the executable is "Heap queue algorithm (a.k.a. priority queue)" [...] - a docstring from the beginning of the file [pythondir]/Lib/heapq.py.
The issue remains if I add the flag --follow-stdlib (hoped the C compiler would maybe remove it).
Haven't tested if this issue concerns other imports, too.
Additional Info

Nuitka version, full Python version and Platform (Windows, OSX, Linux ...)

Tested with both stable and current development versions - same issue.
<IP_ADDRESS>
Python: 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
Executable: D:\Dokumente\PycharmProjects\Scratchbook\venv\Scripts\python.exe
OS: Windows
Arch: x86_64

0.6.12rc5
Python: 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
Executable: D:\Dokumente\PycharmProjects\Scratchbook\venv\Scripts\python.exe
OS: Windows
Arch: x86_64

How did you install Nuitka and Python

using pip within the venv of my PyCharms project.

The specific PyPI names and versions

Actually, I have quite a lot of packages installed in this venv. However in this specific example I'm not using anything except the stdlib so I'd argue the list is not important for this issue. Please correct me if I'm wrong.

That's actually a good catch, I doubt that docstrings are any use in there, although there might be code relying on it.
Our difficulty here, will be the following:
            elif part in ("-O", "no_asserts", "noasserts"):
                _python_flags.add("no_asserts")
            elif part in ("no_docstrings", "nodocstrings"):
                _python_flags.add("no_docstrings")
            elif part in ("-OO",):
                _python_flags.add("no_docstrings")
                _python_flags.add("no_asserts")

As you can see, no_docstrings is a Nuitka specialty, which doesn't exist in Python. And when we compile bytecode, we might not have the ability to do only one and not both, I will check it.
It's also noteworthy, that some OSes, have stopped including source code for some of standard library, Fedora 33 is something to consider there, in that case, we will only be able to do it, by patching the bytecode itself.

I checked the compile built-in that is currently used to turn source code to bytecode there, and it doesn't consider any of the flags.
What would have to the done, is to use ast.parse, which is going to give us a chance to modify the tree, e.g. setting all doc attributes to None. The passing that tree to compile is going to work nicely. Doing it on the resulting bytecode object from direct compile may make it more likely to work in all cases. The ast.parse is known to fail with recursion errors on too complex code.

Deleting the docstrings causes segfaults to Python, and setting them to None caused errors on future imports, so setting them to empty string is the safest way. Since that's a shared value too, it doesn't matter much I guess. Not doing it for demoted to bytecode modules now, not until I have a good test case that shows that we can do a fallback to primitive bytecode compilation, should that ast.parse fail.

Thanks for the quick reaction!
I have to say, I can't follow you a 100% but then I'm not that experienced with Python (ast definitely is something I'd need a LOT more time to understand).
However, I can tell you what my original goal was. When I compile the hello world example from above into a standalone exe I get a 24MB (!) file. That seemed crazy to me. Adding --follow-stdlib reduced it to about 7MB - much better but still not exactly small. So I had a look at the exe and found tons of (doc)strings.
Since I can't judge how complicated it is to implement this (and hence how long it might take) I'd like to know if there is something I could do myself to get rid of that unecessary data in the exe?
Kind regards,
Max

With the --follow-stdlib there should have no docstrings all along, can you confirm or deny that?
I have the change ready to be pushed on factory, but currently a pre-release is building and I don't want to mess with that yet, I will let you know.
With that, docstrings for standard library should be gone either way. As an aside, you can also try the no_site, which prevents including things that site.py loads (and never get used), which somtimes can be a lot too.
I am not sure if --follow-stdlib ought to reduce it that much, or rather increase it. But that's a clear hint that we need to have it on by default, even if we demoted it later again. But right now, before caching of result of Nuitka frontend compilation, I don't want to go there, it would just take too much of my time in this order.

Also, keep in mind, your program may say hello world only, but you asked to bring a Python installation along. These are not small.

With the --follow-stdlib there should have no docstrings all along, can you confirm or deny that?

That option does reduce the file size drastically but it does not remove the docstrings from the stdlib files as states in the initial post.
I can try to look a little more into what causes the drastic size difference of --follow-stdlib. However, I don’t know if I’ll find anything meaningful since I’m not experienced at all with how Nuitka works and thus what I should expect or not.

I have the change ready to be pushed on factory

Awesome, thank you!

@MMMZZZZ can you provide evidence of the doc string not being reduced from the module or a function? I would need an example to work with. From what I saw, it did it. But the visitor might miss some things of course.

I'm afraid I can't quite follow you (as I said, not very experienced in this field). What evidence or example do you need...? First post contains the command line, the source that's been compiled and an example of a docstring that has not been removed. All I can think of that's missing would be the .exe file(s) themselves?
Regards,
Max

I am missing the docstring that is included and you think it should be removed, with a pointer to the source of the standard library where it lives. If you see data in the binary, it doesn't have to be doc strings, it can be regular strings too. At this time I don't know your example of a docstring that survives.

Quote from the opening post:

Result:
The resulting executable does not contain the docstring from the source file, but it does contain docstrings from the stdlib files. One specific example I found in the executable is "Heap queue algorithm (a.k.a. priority queue)" [...] - a docstring from the beginning of the file [pythondir]/Lib/heapq.py.
The issue remains if I add the flag --follow-stdlib (hoped the C compiler would maybe remove it).

To rephrase it using quotes from your post:

the docstring that is included and you think it should be removed

"Heap queue algorithm (a.k.a. priority queue)" (the string is much longer but this snipped is enough to identify it).

a pointer to the source of the standard library where it lives

docstring from the beginning of the file [pythondir]/Lib/heapq.py.

If you see data in the binary, it doesn't have to be doc strings, it can be regular strings too.

Of course. But I'm pretty sure the given example is a docstring. And it's not the only one; it just happened to be the first one I found.
I can only assume you overlooked the given example. Otherwise I'm sorry but I'll need more explanations what information is missing in the example and how to provide a more useful one
Regards,
Max

We might be talking past each other. The change is part of the 0.6.12 release and previously was on factory, which I forgot to state, did you have it occur using that version, because it seems you think I dispute your original report, which I didn't mean to. I cannot reproduce your example string in either Python2 not Python3, so I am assuming we are just confused (and it's my fault definitely ?

Oh okay. Yeah I really didn't realize you released a new versions that's supposed to fix it. Unfortunately I have bad news: the new version didn't fix it. Here's what I did:

Installed Nuitka v<IP_ADDRESS>
Deleted all the folders with cached files from previous runs.
Used the command line and example file I posted at the top
Result: heap queue docstring and many others still present in the exe
Added --follow-stdlib, compiled again
Result: heap queue docstring and many others still present in the exe

Aaaand the heap queue docstring is still contained in the exe. Others, too.
Regards,
Max

Ok, I was using my development version, but I don't think it should make a difference, and could find it with either vi nor strings on the executable, i saw a lot of things that looked like docstrings though. I didn't check though if "heap q" which is what I looked for is actually in the source files, which I now did.
May I suggest you check if you get into compileSourceToBytecode of nuitka.Bytecodes module and see what it does, and report if it doesn't remove things for you or doesn't get asked? Maybe your installation has no source code for that standard library module, and you don't take that code path?


Add Dark Mode/Light Mode Toggle Feature

Description:
The Dark Mode button is not working.
Currently, the website lacks a Dark mode/Light mode toggle, which would enhance user experience by allowing users to switch between dark and light themes as per their preference. I propose implementing a toggle button that enables users to easily switch between the two modes.
Features:

Toggle Button: Add a button in the navigation bar or settings menu to switch between Dark and Light modes.
Theme Persistence: Save the user’s theme preference using local storage, so it remains consistent across
sessions.
Smooth Transitions: Ensure smooth CSS transitions when switching themes.
Accessibility: Make sure both modes are visually accessible, with appropriate colour contrasts.

Benefits:

Improved user experience with theme customization.
Reduced eye strain during night-time browsing.
More modern and engaging UI design.

Tech Stack:
HTML, CSS, JavaScript

@vishanurag  Please assign this issue to me.


General improvements

This Pull Request includes:

Use argparse on CLI.
Add optional positional argument to MemoryBackend to create instance with file path.
Proxy register_blueprints(), init_backend(), and set_config() to the Flask instance passed.
Documentation on how to build custom backends, how to override auth, how to add a users_databases.
Allow to dynamically load backends. This way, one can define custom backends and attach them in an easier fashion.
Add logging capability to medallion
Fix status responses to add successes, failures, etc.
Removed unused method stubs and relocated create_bundle() to common
Formatting changes
Update README to represent the latest changes

Addresses:

closes #13
closes #14
closes #24
closes #1

Codecov Report

Merging #26 into master will decrease coverage by 1.52%.
The diff coverage is 74.37%.

@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
- Coverage   88.54%   87.01%   -1.53%
==========================================
  Files          16       15       -1
  Lines         716      755      +39
==========================================
+ Hits          634      657      +23
- Misses         82       98      +16

Impacted Files
Coverage Δ

medallion/scripts/run.py
0% <0%> (ø)
:arrow_up:

medallion/views/collections.py
88.23% <100%> (ø)
:arrow_up:

medallion/views/manifest.py
90.9% <100%> (ø)
:arrow_up:

medallion/views/discovery.py
90.47% <100%> (+0.47%)
:arrow_up:

medallion/test/test_memory_backend.py
100% <100%> (ø)
:arrow_up:

medallion/views/objects.py
85.71% <100%> (ø)
:arrow_up:

medallion/backends/memory_backend.py
85.13% <80%> (-0.79%)
:arrow_down:

medallion/utils/common.py
85.96% <81.81%> (+1.34%)
:arrow_up:

medallion/__init__.py
87.5% <86.84%> (+7.5%)
:arrow_up:

medallion/filters/basic_filter.py
93.05% <0%> (+1.38%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93c05df...7fddd0b. Read the comment docs.


 This is due to bad box sizing:

This PR adds box-sizing: border-box, which fixes this issue. After this PR:

This PR also fixes the float-right text (i.e. "# replies") in post headers being cut off; it was caused by essentially the same issue (bad box sizing).

Seems good :+1:


Enable RHEL 8 packages

Signed-off-by: Jaymala Sinha<EMAIL_ADDRESS>
Description

Related Issue

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] I have run the pre-merge tests locally and they pass.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] All commits have been signed-off for the Developer Certificate of Origin.

https://buildkite.com/chef/chef-chef-dk-master-omnibus-adhoc/builds/6

https://buildkite.com/chef/chef-chef-dk-master-omnibus-adhoc/builds/4 already passed


html

Hello!


o365 autograb email anyone?

DO NOT ASK FOR PHISHLETS.
DO NOT ASK FOR HELP CREATING PHISHLETS.
DO NOT ASK TO FIX PHISHLETS.
DO NOT ADVERTISE OR TRY TO SELL PHISHLETS.
EXPECT A BAN OTHERWISE. THANK YOU!
REPORT ONLY BUGS OR FEATURE SUGGESTIONS.
office autograb email in link, is it possible?

DO NOT ASK FOR PHISHLETS.
DO NOT ASK FOR HELP CREATING PHISHLETS.
DO NOT ASK TO FIX PHISHLETS.
DO NOT ADVERTISE OR TRY TO SELL PHISHLETS.
EXPECT A BAN OTHERWISE. THANK YOU!
REPORT ONLY BUGS OR FEATURE SUGGESTIONS.
office autograb email in link, is it possible?

You guys should be careful of this guy demanding for money. He's a scammer and a ripper. Will claim he has phislet and start demanding for money claiming nothing is free.
stop-robot<EMAIL_ADDRESS>Unsubscribe | stop-robot<EMAIL_ADDRESS>Unsubscribe | 4:07 AM (1 hour ago)
stop-robot<EMAIL_ADDRESS>Unsubscribe
to kgretzky/evilginx2, me, Author | to kgretzky/evilginx2, me, Author
to kgretzky/evilginx2, me, Author
<EMAIL_ADDRESS>target="_blank" style="color: rgb(17, 85<EMAIL_ADDRESS>stop-robot<EMAIL_ADDRESS>Unsubscribe
4:07 AM (1 hour ago)
to kgretzky/evilginx2, me, Author
Add me on ICQ @MAX1N3WT0N
or skype<EMAIL_ADDRESS>

Hello! If you were already able to resolve your doubts and achieve your goals, close the issue so that we know which ones are pending.
Thank you!

If you have edited working o365 that grab email and remove bots please dm @xfoxhack icq


Update netmap from luigirizzo

The latest code from Luigi Rizzo is needed for netmap to work with the latest kernel from CentOS.

👍  : verified on my development VM - rpm installed correctly.

@john-gress : for verifying this do you suggest to install your PR from NMBaseInstall first the. This rom then rebuilding/bootstrapping the rest of our repos?
Or is it enough to just install the rpm and testing that it works (& no errors)?

@KjellKod, you need to install the latest rpms in my pull request on NMBaseInstall first, then reboot; then rebuild netmap. After installing netmap, I recommend rebuilding all other repos and installing those rpms. After you have completed all that, then run NetMon and configure the Engine to use a netmap interface. Verify that the latest netmap correctly reads packets from the interface. Verify no unexpected messages in /var/log/messages from netmap.


Added Finished Project

Finished Task 1-4, added all required assets, sent a text file to answer the Task 4 questions.

HI @ProfessorVincere love what you did here. Just one more thing, I have another task for you to complement this assessment.
Can you add Unit Testing on this projects for all the scripts you made? I also wanna know if you can produce a well tested Unity app via code level.


[SPARK-45068][SQL] Make math function output column name displayed in upper-cased style

What changes were proposed in this pull request?
The pr aims to Make math function output column name displayed in upper-cased style.
Why are the changes needed?
1.Although It sees there is not a criterion of the style. Functions with the same functionality have different aliases, with one lowercase and one uppercase when displayed, which looks weird.
https://github.com/apache/spark/blob/60d8fc49bec5dae1b8cf39a0670cb640b430f520/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L427-L428
Before:
scala> spark.sql("SELECT sign(-1), SIGN(-1), signum(-1), SIGNUM(-1)").show
+--------+--------+----------+----------+
|sign(-1)|sign(-1)|SIGNUM(-1)|SIGNUM(-1)|
+--------+--------+----------+----------+
|    -1.0|    -1.0|      -1.0|      -1.0|
+--------+--------+----------+----------+

After:
scala> spark.sql("SELECT sign(-1), SIGN(-1), signum(-1), SIGNUM(-1)").show
+--------+--------+----------+----------+
|sign(-1)|sign(-1)|signum(-1)|signum(-1)|
+--------+--------+----------+----------+
|    -1.0|    -1.0|      -1.0|      -1.0|
+--------+--------+----------+----------+

2.The display of function names differs in case of one parameter and two parameters
scala> spark.sql("SELECT ceil(-0.1), ceil(-0.1, 1), floor(-0.1), floor(-0.1 ,1)").show
+----------+-------------+-----------+--------------+
|CEIL(-0.1)|ceil(-0.1, 1)|FLOOR(-0.1)|floor(-0.1, 1)|
+----------+-------------+-----------+--------------+
|         0|         -0.1|         -1|          -0.1|
+----------+-------------+-----------+--------------+

scala> spark.sql("SELECT ceil(-0.1), ceil(-0.1, 1), floor(-0.1), floor(-0.1 ,1)").show
+----------+-------------+-----------+--------------+
|CEIL(-0.1)|CEIL(-0.1, 1)|FLOOR(-0.1)|FLOOR(-0.1, 1)|
+----------+-------------+-----------+--------------+
|         0|         -0.1|         -1|          -0.1|
+----------+-------------+-----------+--------------+

Does this PR introduce any user-facing change?
Yes.
How was this patch tested?

Pass GA.
Manual testing.

Was this patch authored or co-authored using generative AI tooling?
No.

cc @HyukjinKwon @zhengruifeng @cloud-fan

given the amount of changes, I'm rethinking about what's the right behavior. Looking at Expression#prettyName, it does seem like we prefer lower-cased names for functions. But we can't say that it's wrong to use upper-cased names for math functions.
Shall we just follow the existing naming style? Math functions prefer upper-cased names and we should update UnaryMathExpression#prettyName instead.

given the amount of changes, I'm rethinking about what's the right behavior. Looking at Expression#prettyName, it does seem like we prefer lower-cased names for functions. But we can't say that it's wrong to use upper-cased names for math functions.
Shall we just follow the existing naming style? Math functions prefer upper-cased names and we should update UnaryMathExpression#prettyName instead.

Okay, that's fine. Let's show all the mathematical functions in uppercase form, which is also reasonable. We need to maintain consistency.
Let me to do it!

Although this change involves a lot of files, I think it should be made, such as:

Let me double check again.

Done.

please rebase after https://github.com/apache/spark/pull/42869 get merged

#42869

Okay, let me wait for it.

please rebase after #42869 get merged

Done.


Use HttpMethods.IsGet()

Use HttpMethods.IsGet() instead of a string comparison.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 90.41%. Comparing base (91e61c5) to head (91734ee).

Files
Patch %
Lines

...washbuckle.AspNetCore.Swagger/SwaggerMiddleware.cs
50.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2971      +/-   ##
==========================================
+ Coverage   88.26%   90.41%   +2.15%
==========================================
  Files          95       74      -21
  Lines        3271     2870     -401
  Branches      560      452     -108
==========================================
- Hits         2887     2595     -292
+ Misses        384      275     -109

Flag
Coverage Δ

Linux
90.41% <50.00%> (+2.23%)
:arrow_up:

Windows
?

macOS
90.41% <50.00%> (-0.04%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Certain Biome Names Do Not Have Proper Assigned Text

When flying around through biomes, noticeably certain biomes will not have proper text assigned.
Minecraft Version: 1.20.1
Fabric Version: 0.15.7
Eden Ring Version: 0.6.17
Sodium Version: 0.5.8
Fabric API Version: 0.92.0

I Haven't Entered The Translation Keys Yet.

This Is Fixed In 0.6.18


Sample to perform http to https Redirect in CloudFormation

I have been trying to find a way to create an "HTTP to HTTPS" redirect method utilizing an AWS::ElasticBeanstalk::Environment in CloudFormation. The samples include documentation for doing this via a "Custom Resource" (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-format-resources-eb.html) but I'm unable to find a way to accomplish the same thing via CloudFormation.

@colinbjohnson I have the same issue, have you managed to find it meanwhile?


[CLOSED] Integrate tiled parser

 Issue by Sean-Nishi
Sunday Feb 05, 2023 at 22:06 GMT
Originally opened as https://github.com/Sean-Nishi/M1Wengine/issues/20

None

 Comment by owalk
Monday Feb 06, 2023 at 22:11 GMT

We will need to check out how the linter looks at the new map format.
For now I won't add any map specific exceptions into the linter

 Comment by owalk
Saturday Feb 11, 2023 at 14:50 GMT

The Tiled software has a nice CSV export. We just need a good tileset to use with it first.
After exporting, it will be a CSV of numbers that are mapped to an image out of the tile set.
The draw function for levels will need to hold an enumeration of numbers to images within the tileset. It should be flexible to use with any CSV. All CSV tilesets should use the same width/height in pixels.

 Comment by owalk
Wednesday Aug 09, 2023 at 23:27 GMT

This should have functional camera logic for the new tile integration


Create Login and Signup pages

I will work on this @consci210

Sure

Sure

The theme color should be blue white and black right

I agree with the black and white , I am not sure about the blue though . Have a look at the other pages and if you think the three colors would be a good match to the rest of the content then sure you can go with those .

I agree with the black and white , I am not sure about the blue though . Have a look at the other pages and if you think the three colors would be a good match to the rest of the content then sure you can go with those .

Ok Thank you

I have approved your PR. Good Job .

Thank you, I really appreciated

@Mublin  🎉 I have added all the required functionalities to the login and signup forms . This means that any user can now create an account and login using their email and password 📧🔐 or simply use their Google account to authenticate.


Add $j discussion and move $t before it. Fixes #15

Add a discussion about $j to the "Extensions to the Metamath Language"
section.  Move the $t discussion into that same section
(just before the $j section), since the two share a lot in common
syntactically.
Signed-off-by: David A. Wheeler<EMAIL_ADDRESS>

@nmegill @digama0 - This is my try at a description about $j.  There's a lot I don't know, so instead of trying to nail down everything, I tried to nail down the syntax and then include some examples.  I think it gets the point across and - more importantly - it's a big improvement.
Suggestions welcome.

Note: I have to merge this before I can move the old $t stuff from appendix A.  We should have all the information of how to write a $t section in one place.

@nmegill - This is holding up my work to clean up Appendix A.  Okay to merge, and then we can improve the discussion of $j later if we can find a way to improve it?

I just noticed that @digama0 approved the changes earlier (I don't know how I didn't notice that).  So I plan to merge this soon unless someone says otherwise...!

I'm ok with this.


Add tracing and metrics

Experimental tracing and metrics support for the service-api.

Leaving this in draft for the future where we have a tracing backend.


mc-a2m-mapper should send only anomalies back to the metrics topic

Currently mc-a2m-mapper is sending all classifications back to the metrics topic. This is neither necessary nor desired. We want to send only weak and strong anomalies back.

Done: b80a0d66bce56ce28fc6f0141be5d602ca2b635e


ci: Add release-please-action

Imagine a world.... where bots did all our work for us...
That world.... is now!

Our other "release" actions might conflict with the releases created by release-please. I'll probably have to mess around on master a bit (and kill opam publishes before they submit).

Our other "release" actions might conflict with the releases created by release-please. I'll probably have to mess around on master a bit (and kill opam publishes before they submit).


Add credentials parameter

Expected Behavior

I would like to be able to use specific credentials when running commands in this module.
Current Behavior

Currently the current user credentials are passed through.
Possible Solution

add a parameter to allow for credentials to be passed through to invkoke-command
Steps to Reproduce (for bugs)

Context

Your Environment

Version used:
Environment name and version (e.g. Chrome 39, node.js 5.4):
Operating System and version (desktop or mobile):
Link to your project:

@thefirstnoe1 Would this need to run under these credentials locally too?
If you were on server1 and that is the server you were querying..
Running as user1, but providing creds for user2, you would want, locally, to run under user2 context?
My assumption is that if you are RDP'd into the server you are running this against, then you would already be signed in as the correct user, but providing credentials for all remote queries would be easy.
Thoughts?

Personally I never use this locally on a box, it is all remote. I am with you on the thinking that if it is being run locally, you probably already have the permissions to run using the logged in user.

Check out this branch 9a5626c14ad9fc3867646e5792a9d7eca4b32473 to see if you like / agree with this implimentation


feat: 76 user story channel implementing closeinit handshake

Description:
As a user, I want to be able to initiate a channel close handshake with another user so that we can safely close the channel and settle any pending transactions.
Commit Message
feat: implementing closeinit handshake for channel

see the guidelines for commit messages.
Checklist:

[x] I have performed a self-review of my own code
[x] I have documented my code in accordance with the documentation guidelines
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have run the unit tests
[ ] I only have one commit (if not, squash them into one commit).
[x] I have a descriptive commit message that adheres to the commit message guidelines

Please review the CONTRIBUTING.md file for detailed contributing guidelines.

Codecov Report

Merging #196 (c035d15) into main (f27e901) will increase coverage by 0.84%.
The diff coverage is 78.40%.

@@            Coverage Diff             @@
##             main     #196      +/-   ##
==========================================
+ Coverage   51.05%   51.89%   +0.84%
==========================================
  Files          64       65       +1
  Lines        5602     5723     +121
==========================================
+ Hits         2860     2970     +110
- Misses       2742     2753      +11

Impacted Files
Coverage Δ

...w-ibc-core/src/ics04_channel/handler/close_init.rs
76.47% <76.47%> (ø)

...smwasm-vm/cw-ibc-core/src/ics04_channel/handler.rs
49.34% <78.87%> (+17.26%)
:arrow_up:

...osmwasm-vm/cw-ibc-core/src/ics04_channel/events.rs
98.89% <100.00%> (ø)

... and 2 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Fix for Bash env script injection on RPM family distros

Resolves #1187 - this is a quick fix I hacked up that piggybacks on the existing add_install_dir_to_path function.
It works well in a Fedora container, but shell scripting is not my strong suit - I'll gladly cook up something different if there's a footgun I overlooked.

Oh wow, I super appreciate the research you did into this!

Hmm interesting it looks like rustup currently in fact doesn't create a bash file if they're all missing, so maybe your original commit should be desired behaviour? (It does unconditionally create .profile)
https://github.com/rust-lang/rustup/blob/fe99f782cf362a11609768d58ea27183ae685532/src/cli/self_update/shell.rs#L136-L156

I can definitely see an argument for that. Should we make the other shells behave similarly, in that case? I did notice that the Atuin installer was creating a .zshrc in my home despite zsh not even being installed on my system

This is really interesting, it seems like rustup has tightened up how hard it shotguns since last i checked. It means that someone who installs zsh later won't see cargo on path, hmmm...

(Gonna consult with coworkers on what we wanna do here)

Thanks for this PR! It's a big help.

Most people will (I'm assuming) be using non-login shells

macOS's terminal uses login shells by default, so we do want to plan around it. (I don't even have a .bashrc on my system!)
I think, reading up on what rustup does, that their logic makes some sense. Plus, if we stop creating a file by default, we can completely dodge the "login shell or not login shell" question.
The "should we create .profile" question is kinda sorta separate. The answer "this is the only one standardized by POSIX, so we should always handle it" makes a certain amount of sense, but in practice absolutely nothing on my system has ever interacted with it (and only cargo-dist has ever put stuff in there, so it's a graveyard of test data). It doesn't hurt to make that the default file we create, but I'm not convinced it's actually going to affect very many users.

Interesting, I didn't know that about macOS.
With respect to .profile - I think it makes sense to keep it, even if just for the token POSIX acknowledgement. Best effort is better than no effort.

Sounds good! In that case, how about we keep what you've written except we change the order so the "write to the first in the list" is .profile?

Sure! Do you think we need to alter the behavior for the other shells at all? I'd need to double check, but I think ZSH etc. all match rustup.

Double checked, and the existing logic for POSIX/Zsh/fish seems to be valid and equivalent to rustup's implementation.


"Făget", "Bolovăniș", "Făgetu de Sus", "Ghimeș", "Răchitiș", "Tărhăuși" are villages in "Ghimeș-Făget" common, "BACĂU" county

Description : When user is typing the villages in the search bar, the locations should appear in the dropdown menu correctly spelled and with localities information (as in Law nr. 2 din 16 februarie 1968)
Preconditions : the website (https://peviitor-ro.github.io/ui.orase/) is up and runing.
After every step delete the text you typed from the search bar.

Steps 1: Click on the search bar and write "Făget"
Expected result: In the drop down menu you find "Făget, Bacău (Ghimeș-Făget)"

Steps 2: Click on the search bar and write "Bolovăniș"
Expected result: In the drop down menu you find "Bolovăniș, Bacău (Ghimeș-Făget)"

Steps 3: Click on the search bar and write "Făgetu de Sus"
Expected result: In the drop down menu you find "Făgetu de Sus, Bacău (Ghimeș-Făget)"

Steps 4: Click on the search bar and write "Ghimeș"
Expected result: In the drop down menu you find "Ghimeș, Bacău (Ghimeș-Făget)"

Steps 5: Click on the search bar and write "Răchitiș"
Expected result: In the drop down menu you find "Răchitiș, Bacău (Ghimeș-Făget)"

Steps 6: Click on the search bar and write "Tărhăuși"
Expected result: In the drop down menu you find "Tărhăuși, Bacău (Ghimeș-Făget)"


Installing pytest from info bar doesn't trigger test discovery

Steps to Reproduce

Open project with pytests, pytest configured (but not installed)
Info bar appears, click to install pytest

Result: tests don't appear after pytest is installed
Expected behavior
Tests are discovered

This should be handled by https://github.com/microsoft/PTVS/issues/5424


Fix infinite loop caused by URL loading

Loads the argument URL received during WebView initialization only once.
How to reproduce the issue.
ContentViewState.swift
- let requestURL = URL(string: "https://cybozu.github.io/webview-debugger")!
+ let requestURL = URL(string: "https://www.google.com")!

ContentView.swift
WebViewReader { proxy in
    VStack {
        // ...

-       WebView(configuration: viewState.configuration)
+       WebView(url: viewState.requestURL, configuration: viewState.configuration)
            .uiDelegate(viewState)
            .navigationDelegate(viewState)
            .allowsInspectable(true)
            .allowsBackForwardNavigationGestures(true)
            .allowsLinkPreview(false)
            .refreshable()
            .border(Color.gray)
    }
-   .onAppear {
-       proxy.load(url: viewState.requestURL)
-   }
+   // .onAppear {
+   //     proxy.load(url: viewState.requestURL)
+   // }
}

We would like to use testing to ensure that the infinite loop issue does not occur.
Is it possible to guarantee this with a unit test?

We would like to use testing to ensure that the infinite loop issue does not occur. Is it possible to guarantee this with a unit test?

It would be difficult to write an unit test that runs ViewRepresentable's updateUIView() method.


[CODEGEN][CUDA]: fix cuda half math function is undefined: herf

I got an "identifier herf undefined" error when converting a model into TVM, which can be reproduced by the following code
import torch
import numpy as np
import tvm
from tvm import relay

class ErfTest(torch.nn.Module):
    def forward(self, data):
        return torch.erf(data)

def run(data):
    # convert pytorch to tvm
    traced = torch.jit.trace(ErfTest(), torch.from_numpy(data).cuda())
    mod, params = relay.frontend.from_pytorch(traced, [('data', data.shape)])

    # compile
    with tvm.transform.PassContext(opt_level=3):
        relay_graph, relay_lib, relay_params = tvm.relay.build(mod, target='cuda', params=params)
        relay_model = tvm.contrib.graph_runtime.create(relay_graph, relay_lib, tvm.context('gpu', 0))

    # run
    relay_model.set_input('data', data)
    relay_model.run()
    return relay_model.get_output(0)

data = np.random.rand(3, 4).astype(np.float32)
print(torch.erf(torch.Tensor(data)))
print(run(data))
print(run(data.astype(np.float16)))

I fixed this in the same way as #6225 and the results seem ok

cc @vinx13 Could you help to review this?

Could you add test cases for half math of erf to verify the accuracy?

I have added a test case for half erf, but ci failed. It seems that this condition #if defined(__CUDA_ARCH__) && (__CUDA_ARCH__ >= 530) is not met in the ci environment, and herf is still undefined. Should I just remove the test case?

You can add a check of cuda compute capability and skip the test if not supported

@kongroo @vinx13 please followup


Use new CodeGenModel trait for generation to allow extensibility

Baased on https://github.com/qir-alliance/pyqir/pull/46, only the final few commits are new.
Other tools may wish to use the same code generation utilities but with a different "SemanticModel", this trait allows for that.
As there are accessors for all members SemanticModel members are made private.
Instruction code gen utilities are made public for reuse.
These changes are used here:  to protoype QIR code generation from the tket JSON IR.

This was accidentally closed by GH when the PR was merged.

Closing as this is not a change critical to my development and may not align well with more recent changes to main.


Devices in config file

Hi.
zigbee2mqtt add-on version (if edge, please report commit hash):
<IP_ADDRESS>
Operating environment (HassOS, Virtual Machine, Device/platform running Hass.io):
Armbian, Hass.io running on Rock64
Description of problem:
I want to give it a friendly-name.
I add in config this string:
 "devices": "{'0x00158d00025ee533': {'friendly_name': 'AqaraTemper', 'retain': False}}",
Config is OK, zigbee2mqtt restart successfully, but my friendly-name does not applycated (
In logs I see:
zigbee2mqtt:info 9/2/2019, 12:41:36 PM MQTT publish: topic 'zigbee2mqtt/0x00158d00025ee533', payload '{"temperature":17.3,"linkquality":84,"humidity":67.78,"pressure":974.3,"battery":100,"voltage":3025}'
If I add this code:
'0x00158d00025ee533':
  friendly_name: 'AqaraTemper'
 retain: false

in device.yaml file - all is OK, friendly_name is working.
But I want use configuration.yaml only. That I doing wrong?
Thanks

Device-specific configuration is only supported via a separate file as the README says. I'm surprised your configuration isn't rejected by the hassio schema validation. If you include your entire configuration, that would help.

Daniel, my zigbee2mqtt configuration file:
{
  "data_path": "/share/zigbee2mqtt",
  "devices": "{'0x00158d00025ee533': {'friendly_name': 'AqaraTemper', 'retain': False}}",
  "groups": "groups.yaml",
  "homeassistant": false,
  "permit_join": true,
  "mqtt": {
    "base_topic": "zigbee2mqtt",
    "server": "mqtt://<IP_ADDRESS>:1883",
    "user": "***",
    "password": "***"
  },
  "serial": {
    "port": "/dev/ttyACM0",
    "disable_led": true
  },
  "advanced": {
    "pan_id": 6754,
    "channel": 11,
    "network_key": [
      1,
      3,
      5,
      7,
      9,
      11,
      13,
      15,
      0,
      2,
      4,
      6,
      8,
      10,
      12,
      13
    ],
    "availability_blacklist": [],
    "log_level": "debug"
  },
  "ban": [],
  "whitelist": [],
  "queue": {}
}

It's work but without friendly-name )

Im just struggeling with the same, I have them in a seperated devices.yaml. But HA doesn't pick up on them. Where does the file need to be located? I the /share folder I guess? And does it need to be json or yaml? (Tried both without success anyway)

Hello.
My device.yaml located in /usr/share/hassio/share/zigbee2mqtt/
In yaml. My devices:
'0x00158d00025ee533':
friendly_name: 'AqaraTemp'
retain: false
From: mason-xx<EMAIL_ADDRESS>Sent: Tuesday, September 10, 2019 6:28 PM
To: danielwelch/hassio-zigbee2mqtt<EMAIL_ADDRESS>Cc: Бочегов Александр Владимирович<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [danielwelch/hassio-zigbee2mqtt] Devices in config file (#220)
Im just struggeling with the same, I have them in a seperated devices.yaml. But HA doesn't pick up on them. Where does the file need to be located? I the /share folder I guess? And does it need to be json or yaml? (Tried both without success anyway)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/danielwelch/hassio-zigbee2mqtt/issues/220?email_source=notifications&email_token=AKSKFAQXWVP5LJCGEE2WQZLQI6OGJA5CNFSM4IS5TRRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LCRUA#issuecomment-529934544, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKSKFATCIFDJ2S7X4RK2L33QI6OGJANCNFSM4IS5TRRA.

Gotcha. So this passes the schema check because you've included the JSON as a string, but the string is just supposed to be a path to your device-specific configuration file. I'm going to close this now, as what you're trying to do isn't supported, but it may be in a future release if the schema restrictions from Hass.io change.


Can not perform this action after onSaveInstanceState

Using version 0.4.3. I am sometimes seeing this crash in production:
Fatal Exception: java.lang.IllegalStateException: Can not perform this action after onSaveInstanceState
at android.app.FragmentManagerImpl.checkStateLoss(FragmentManager.java:1433)
at android.app.FragmentManagerImpl.enqueueAction(FragmentManager.java:1451)
at android.app.BackStackRecord.commitInternal(BackStackRecord.java:687)
at android.app.BackStackRecord.commit(BackStackRecord.java:663)
at android.app.DialogFragment.show(DialogFragment.java:230)
at pub.devrel.easypermissions.helper.BaseFrameworkPermissionsHelper.showRequestPermissionRationale(BaseFrameworkPermissionsHelper.java:30)
at pub.devrel.easypermissions.helper.PermissionHelper.requestPermissions(PermissionHelper.java:73)
at pub.devrel.easypermissions.EasyPermissions.requestPermissions(EasyPermissions.java:183)
at pub.devrel.easypermissions.EasyPermissions.requestPermissions(EasyPermissions.java:141)
at com.xxx.xxx.xxx.xxx.BaseActivity.requestPermission(BaseActivity.java:333)
Device: Galaxy J5(2016)
OS: 6.0.1

@jnam90 that means you are trying to request the permission after the activity is no longer visible. My guess is that you have the permission request is an async callback?

@jnam90 Just another thought, are you calling super.onActivityResult() in your activity?

@SUPERCILEX  Sorry I saw last comment too late.


filtering.include/exclude throw a TypeError when any class attribute uses a Factory default

The hash_ method added by _add_hash in attr._make throws TypeError: unhashable type: 'Factory' if any of the class's attributes uses an attr.Factory default. So for example, below no longer works.
@attr.s
class C(object):
    a = attr.ib()
    b = attr.ib(default=attr.Factory(list))

attr.asdict(C(1), filter=attr.filters.include(attr.fields(C).a))

Sorry for that, will be fixed in the upcoming release.

No worries. attrs is a great library.


make apply

Summary
Run make apply
Ticket Link
Follow up on https://github.com/mattermost/mattermost-plugin-confluence/pull/79

Merging to fix master.


Kubeflow Pipelines Standalone Deployment

https://www.kubeflow.org/docs/components/pipelines/installation/standalone-deployment/

https://www.kubeflow.org/docs/components/pipelines/installation/standalone-deployment/


Add Light and Dark Variants

I was messing around with Telescope's colorscheme switcher and found it really nice, but with how gruvbox is distributed currently, I could not find a nice way to directly switch to the light and dark variants (#297).
I'm not sure if this is best way to approach this problem (and I am very new to the neovim ecosystem, so forgive me if I missed something obvious) - but I found that this was a pretty straightforward method of implementing the schemes. Your thoughts?
Here's how it looks on my machine

hey @sf1tzp thanks for the PR, however, I think this is a very edge-case scenario that you can probably achieve by creating your own personal command for it. Have you tried that approach?

I did, but wasn't able to come up with something that worked how I wanted (eg :colorscheme gruvbox-light). I saw that other colorscheme plugins add variants like this, so I thought I would propose this here.

Actually, apparently I didn't try adding exactly this to my nvim/colors 😅


CursorHold show_cursor_diagnostics keeps the window open

Describe the bug
After updating after quite some time, using an autocommand for cursor diagnostics keeps the diagnostic window open even after moving to another line.
Steps to reproduce
    vim.api.nvim_create_autocmd("CursorHold", {
        group = general,
        pattern = "*",
        command = 'Lspsaga show_cursor_diagnostics ++unfocus',
    })

On lsp-setup config
Expected behavior
Moving to another line should close the floating window.
Neovim version (nvim -v)
0.8.3 and nightly
lspsaga commit
397201abffa681419a610ecbfd5d3c0fed45d20a
Terminal name/version
alacritty

can't reproduce with min config

I have the same issue

It's hard to tell what triggers it, so far I can confirm that if the float is open and I switch to another window it will stay there, and any new float also stays on screen.
My config for the float
  vim.cmd [[autocmd CursorHold * Lspsaga show_cursor_diagnostics ++unfocus ]]

Terminal: WezTerm
NVIM v0.8.3
Installed via packer:
     use {
        'glepnir/lspsaga.nvim',
        branch = 'main',
      }

have a split win right?

In my case I had in the on_attach function in the lsp-setup. When I moved it outside of the function it worked fine.

no need in attach. if not lsp it does not popup.


Allow to provide Post Body as an Array

I've been learning Guzzle for the last week, and figured out a lot of stuff, but can`t make my service description to use the passed parameters array and take the parameters tagged for location 'body' and combine into an string, using postFiled for location, almost do it, but interpolates the & symbol, it would be nice to pass the body parameters just like the request body option array.
http://docs.guzzlephp.org/en/latest/clients.html#request-options
Thank you, i'm eager to know about the production ready tag.

I will close this issue. If your problem still exist with latest versions of Guzzle and Guzzle Service, feel free to reopen it.


Explain below you fucked it. reupload the original. i had to regrade my system and i don't have version one. please reupload.

bro what just explain what's wrong with new version

Hello @KDot227 nice work so far man, really great work… but I got a complain. Whenever I try to obfuscate or rather, after running the obfuscation, the resulting obf bat file, just keeps running blank in the cmd after which I get these errors that has -6 at the beginning, -1, -5… and all.
I’m guessing you know about this already so how can I get over it please?
it has always been my problem but I’m certain your app works perfectly but I don’t know what I’m doing wrong 🤦🏽♂🤦🏽♂

Hello @KDot227 nice work so far man, really great work… but I got a complain. Whenever I try to obfuscate or rather, after running the obfuscation, the resulting obf bat file, just keeps running blank in the cmd after which I get these errors that has -6 at the beginning, -1, -5… and all. I’m guessing you know about this already so how can I get over it please? it has always been my problem but I’m certain your app works perfectly but I don’t know what I’m doing wrong 🤦🏽♂🤦🏽♂

I honestly have never heard of that issue before. My discord is kdot_227 please pm me on there or send me a video

ok i think even having all options set to off and still there's bloat. it's not clear. in the last upload version i could at least tweak the settings so it worked and then convert to exe but now impossible!!!

do u mean bloat as in the file is big and the weird utf characters were added or bloat as in when u open the file its all in the weird text?

thank you both for making issues, I can only make it so good without people telling me wtf is wrong with my programming lol.

haha sorry yeah i really like this project

Hey man, sorry I wasn't able to clearly state what's wrong.
It's a bit glitchy. Here's my question. What would happen if I turn all settings to false?
Also, can you explain what require wifi means. Thanks

Hey man, sorry I wasn't able to clearly state what's wrong.
It's a bit glitchy. Here's my question. What would happen if I turn all settings to false?
Also, can you explain what require wifi means. Thanks

Turning everything to false just means you don't want any extra options. Wi-Fi check makes sure the user has internet available before they run the script. This is broken as of right now I believe but it will be re added next update. I have A LOT of very cool things coming next update and would consider it potentially somalifuscstors second biggest update (V2 being the biggest)

If you have any other questions let me know or add me on discord kdot_227

sent a discord friend request. i think the main issue i'm trying to address here is that some of the options for obfuscation techniques or levels are not clearly noted as to what they do and without the right combination of settings the obfuscated script may not run or have other unintended attributes (I'm not sure exactly as it's been a while since I used it)


Oauth2 merge with kaffe-dev updates

Removes the oauth2 conflicts from kaffe-dev, thanks @malmers

@Kaffe-work Will you fix this?


[web]: Fetch Data Example throws "Failed assertion: boolean expression must not be null" error

I am on firefox 68.0 and the error I get when I run webdev serve is "Failed assertion: boolean expression must not be null" which is caused by a snapshot.hasData being null and not boolean.
name: leadership_coach

environment:
  # You must be using Flutter >=1.5.0 or Dart >=2.3.0
  sdk: '>=2.3.0-dev.0.1 <3.0.0'

dependencies:
  flutter_web: any
  flutter_web_ui: any
  http: ^0.12.0+2

dev_dependencies:
  build_runner: ^1.0.0
  build_web_compilers: ^2.0.0

dependency_overrides:
  flutter_web:
    git:
      url: https://github.com/flutter/flutter_web
      path: packages/flutter_web
  flutter_web_ui:
    git:
      url: https://github.com/flutter/flutter_web
      path: packages/flutter_web_ui

Where is snapshot.hasData coming from?

It's part of a FutureBuilder. See the example code below in which I just replaced the flutter with flutter_web.
import 'dart:async';
import 'dart:convert';

import 'package:flutter_web/material.dart';
import 'package:http/http.dart' as http;

Future<Post> fetchPost() async {
  final response =
      await http.get('https://jsonplaceholder.typicode.com/posts/1');

  if (response.statusCode == 200) {
    // If the call to the server was successful, parse the JSON.
    return Post.fromJson(json.decode(response.body));
  } else {
    // If that call was not successful, throw an error.
    throw Exception('Failed to load post');
  }
}

class Post {
  final int userId;
  final int id;
  final String title;
  final String body;

  Post({this.userId, this.id, this.title, this.body});

  factory Post.fromJson(Map<String, dynamic> json) {
    return Post(
      userId: json['userId'],
      id: json['id'],
      title: json['title'],
      body: json['body'],
    );
  }
}

void main() => runApp(MyApp(post: fetchPost()));

class MyApp extends StatelessWidget {
  final Future<Post> post;

  MyApp({Key key, this.post}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Fetch Data Example',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: Scaffold(
        appBar: AppBar(
          title: Text('Fetch Data Example'),
        ),
        body: Center(
          child: FutureBuilder<Post>(
            future: post,
            builder: (context, snapshot) {
              if (snapshot.hasData) {
                return Text(snapshot.data.title);
              } else if (snapshot.hasError) {
                return Text("${snapshot.error}");
              }

              // By default, show a loading spinner.
              return CircularProgressIndicator();
            },
          ),
        ),
      ),
    );
  }
}

@testphys Have you tried running this code using flutter run -d chrome? I'm curious if the bug is only in the fork or in the upstream Flutter.

This is not caused by the FutureBuilder, this issue is raised when from the http Response we are accessing the body property.
Failed assertion: boolean expression must not be null dart_sdk.js:19110:15
dart:sdk_internal 17552:33                                                                            matchAsPrefix
package:string_scanner/src/packages/string_scanner/src/string_scanner.dart 169:25                     matches
package:string_scanner/src/packages/string_scanner/src/string_scanner.dart 126:19                     scan
package:http_parser/packages/http_parser/src/media_type.dart 48:14                                    <fn>
package:http_parser/packages/http_parser/src/utils.dart 13:16                                         wrapFormatException
package:http_parser/packages/http_parser/src/media_type.dart 46:12                                    parse
package:http/src/packages/http/src/response.dart 84:39                                                _contentTypeForHeaders
package:http/src/packages/http/src/response.dart 77:66                                                _encodingForHeaders
package:http/src/packages/http/src/response.dart 27:51                                                get body
package:webcvportfolio/data/services/packages/webcvportfolio/data/services/medium_service.dart 13:32  <fn>
dart:sdk_internal 25433:9

A workaround for anyone having this error in Firefox is to run the release version webdev serve -r, I didn't have more problems with it.

I had the same problem in Dart 2.6.0 and webdev 2.5.3, and Chrome 78. I started getting these errors after I upgraded Chrome and Dart.
Instead of accessing the body property from http Response I just accessed bodyBytes.
Instead of doing this: json.decode(response.body)
I did this: json.decode(utf8.decode(response.bodyBytes))
It seems like a temporary solution to this problem.

The dart:sdk_internal 17552:33 from the stack trace does not provide enough information. We recently fixed stack-traces in profile web builds. Could you please reproduce this again and provide the new stack trace?

Sure, it's a pleasure to help. Is this information enough or do you need the full stack trace?
EXCEPTION: Failed assertion: boolean expression must not be null
STACKTRACE:  dart:sdk_internal                                    decode package:http/src/response.dart 27:42                 get body package:element/controller/service/users.dart 31:41  fetch package:stack_trace                                  <fn> package:angular/src/core/zone/ng_zone.dart 144:18    <fn> dart:sdk_internal                                    runUnary package:angular/src/core/zone/ng_zone.dart 141:18    [_runUnary] dart:sdk_internal                                    <fn> package:angular/src/core/zone/ng_zone.dart 117:11    <fn> package:stack_trace                                  <fn> package:angular/src/core/zone/ng_zone.dart 132:18    <fn> dart:sdk_internal                                    run package:angular/src/core/zone/ng_zone.dart 129:18    [_run]

@kaede28 You seem to be using a different web framework. It says "angular" in the stack trace, but this is Flutter :)

Oh, sorry about that. As I came here by researching the same problem, I thought my input could help. I assumed Flutter shared some code base with Angular.

Are folks still seeing this?  As @yjbanov said above, stack traces have been improved since the original bug was filed, so a new trace would be helpful.

i'm also getting the same issue :
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel unknown, v1.15.18-pre.1, on Mac OS X 10.15.2 19C57, locale en-GB)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✗] Xcode - develop for iOS and macOS
    ✗ Xcode installation is incomplete; a full installation is necessary for iOS development.
      Download at: https://developer.apple.com/xcode/download/
      Or install Xcode via the App Store.
      Once installed, run:
        sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
        sudo xcodebuild -runFirstLaunch
    ✗ CocoaPods not installed.
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To install:
        sudo gem install cocoapods
[✓] Chrome - develop for the web
[✓] Android Studio (version 3.6)
[!] IntelliJ IDEA Community Edition (version 2019.3.1)
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
[✓] VS Code (version 1.43.2)

The error reporting is really bad and the stacktrace shows error in some totally different line in a different file than the one where the error happens.

For me , the issue was that I was assigning a null value to the Visibility widget's visible property . But it was showing the error in some whole another unrelated page.

webdev serve is no longer supported. Use flutter run -d chrome for debugging instead. If there are issues related to stack traces that haven't been filed yet, please open a new issue. I'm going to close this one.


docs(readme): needed

:tada: This PR is included in version 1.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Optifine - chopped logs are unlit

https://www.curseforge.com/minecraft/mc-mods/treechop?comment=518

Probably fixed with #167


41+ checks: Table grows too large into the credits!

I don't know how to describe the issue better, but with a large amount of entries in the table, it grows into the credits at the bottom:
http://i.yt.gl/get/44ecc/sheex.png
It's only cosmetics, but it looks somewhat broken ;)
Would appreciate if you can take a look at this issue!

Use Firebug and Fix it using css


Fix Falcon bug

Describe your changes
This addresses the raised in https://github.com/0xPolygonMiden/miden-base/issues/616
The problem was due to the order of the secret key basis.
Checklist before requesting a review

Repo forked and branch created from next according to naming convention.
Commit messages and codestyle follow conventions.
Relevant issues are linked in the PR description.
Tests added for new functionality.
Documentation/comments updated according to changes.

Question: do you think this would have been caught if we had #293 implemented?

It might have been discovered in the process of implementing #293 . The problem was basically just the order and sign of the basis element of the secret key. I think this was messed up when we were reverting the key recovery mode changes.
In any case, I think having #293 implemented will be good to get more confidence in the implementation. I got some ideas for how to do this while debuging the current issue, I will add them to #293 for discussion soon.


Extension doesn't work when file type is "helm-template"

I use tim-koehler.helm-intellisense  extension for helm and VScode recognise values.yaml as "helm-template" type of file. Sops extension seems to be not enabled then (I assume it works only for yaml file type). Would be nice to use both extensions the same time.

I tried to find the problem, however, I was unsuccessful.
I'm using the mentioned extension as well. But the file type of those HELM templates is detected by VSCode as standard YAML files, not the helm-template. Maybe it's because you are using one more extension which changes the file type to helm-template.
Additionally, the HELM templates are usually not a valid YAML files. They use a lot of template specific characters, which makes YAML file unparsable by standard tools. I guess the SOPS Mozilla tool cannot work with those invalid files as well. Tell me if I'm wrong.
So could you provide me more info on this issue? Some example files and related code extension list would be great.

Hi @misak113
Ok I see it now - the helm-template file is detected and changed by Kubernetes extension that provides helm template intellisense:
ms-kubernetes-tools.vscode-kubernetes-tools


Transform value.toString() converting (blank) values in a cell to a string with 'null' as the value

When reporting a bug please provide the following information to help reproduce the bug:
When reviewing open issues this #332 seems to be the closest I could get to, but what is being described is not the same thing I am running into.
Version of OpenRefine used (Google Refine 2.6, OpenRefine2.8, an other distribution?):
2.8 and 3.0-beta
Operating Systems and version:
Windows 10 Version 1803
Browser + version used - Please note that OpenRefine doesn't support Internet Explorer but works OK in most cases:
Chrome Version 66.0.3359.181 (Official Build) (64-bit), using the API via python 2.7 and 3.6 (only http request)
Steps followed to create the issue:
Started new project with the attached file.
louisiana-elected-officials.zip
In the Zip Code column clicking transform. I use the following code
value.toString()[0,5]
What happens is it converts the previously (blank) rows into 'null' strings
If you are allowed and are OK with making your data public, it would be awesome if you can include the data causing the issue or a URL pointing to where the data is (if your concerned about keeping your data private, ping us on our mailing list):
Current Results:
value.toString() on nulls is turning them into strings with the value 'null'
Expected Results:
null / blank rows  kept as null / blank values in the row

Hi !  We just released 3.0 Beta of OpenRefine.  Please try it out, as we have add several new features to help you with NULL values, such as new Facets and a new Toggle mode for displaying NULL cells.  You can read about the new features and download the newest version on our releases page https://github.com/OpenRefine/OpenRefine/releases
We would appreciate any feedback you can give us on how we can make things even easier for your needs !

Sorry, WHAT are you trying to perform ?  So you have some blank Zip Codes on 891 rows... they basically look like this ""  ... so what do you want to do with that empty string (also called a blank) ?

@dbutlerdb
The short explanation is that the behaviour of 'toString' on a null cell is deliberately set to result in the string null.
The short solution:
Use the 'Facet on blank' option on the column to remove all rows with blank and/or null values before applying the 'toString' transformation on the column
More information:
probably worth giving some background on this first - apologies if some/all of this you may know or is obvious
Each cell in an OpenRefine project contains a value of a specific 'type'. The types supported are:

String
Number
Date
Boolean
error
null

The first four are different types of data. The data type will determine what you can do with the value. For example, if you want to sum two values by adding them together, they must both be of Number type.
error is when the cell is storing an error generated during a transformation in OpenRefine
null is a special value which basically means 'this cell has no value' (see https://techterms.com/definition/null for a slightly fuller definition).
You can convert between types within some limits (e.g. you can't turn the string "asdfsd" into a Date or a Number, because OpenRefine has no way of knowing how to make a Date or Number from a random collection of letters).
The 'toString' option will work on any of the value types and gives a String version of the value. This works in different ways depending on what type you are converting from. The deliberate decision in OpenRefine is that using 'toString' on null results in the string "null"
When you see an empty cell in an OpenRefine project there are broadly three possibilities:

the cell contains the null value
the cell contains a string value of zero length
the cell contains a string value that only contains whitespace

When you are using the toString() function, any null cells will get coverted to "null"
In OpenRefine 3.0 upwards we've introduced some more options for handling the difference between null and zero length strings, although there are probably more improvements to be made, and we probably can't get it right for everyone at the same time.
These improvements include:

ability to view where there are null values in your project by using the menu All->View->Show/Hide null values in cells
option to "facet by null" as well as "facet by blank" (the latter was previously the only option and tests for both null and empty strings) (in the "Customised Facets" drop down menu)
option to convert cells "to null" or "to empty string" (in the "Common Transforms drop down menu)

Owen,
Great writeup actually !  I've added this info to our Wiki under 2 pages:
https://github.com/OpenRefine/OpenRefine/wiki/Cell-Editing#what-is-a-cell-
https://github.com/OpenRefine/OpenRefine/wiki/GREL-String-Functions  <--we
were missing the toString() function, and was found only on Date page
https://github.com/OpenRefine/OpenRefine/wiki/GREL-Date-Functions
Feel free to edit more and improve.
-Thad

Hi Owen,
I'm having a hard time understanding why the default action when converting a numeric cell with no values into a string is to add 'null' into the cell. If operations like .toString[0,2] didn't lop off part of the 'null' string it would make more sense to me.  I am not the expert here like you guys so I'm sure there is a reason that I am missing.
A work around it to exclude all nulls before making a transformation but that can be easily forgotten or unknown to a new user. Nulls/blanks were not handled this way in version 2.5 and lower.

@ostephens We wanted toString() to be able to convert any Data Type to a string, that included null.  Sorry, but how does this hurt ?  Can you give me a scenario where this causes some grief ?

Oh yes, I completely agree on those points.  And I'm not sure why Tom did that...other than visibility of null's.  But I always had the idea of simply handling that via display...and not Data Type conversions as he did.
+1  Please revert toString to old style.  Knowing that we have new ways to make null visible now.

Fixed by #1650

Thanks to @dbutlerdb for raising this - new behaviour will be in 3.0

Thank you for changing the functionality!


Errors during execution on TrueNAS SCALE 22.12-BETA.2

Hello,
I tried to use the script on the latest version of TrueNAS Scale, but without success. The disks do not react and I get errors when running:

Do you have any idea of the cause?
Thank you!

Hi,
please try to use only the drive names (e.g. sda) instead of the full path (e.g. /dev/sda) with the -i option:
/root/spindown.sh -t 10 -p 5 -i sdd -i sda
You can also have a look at the examples in the README file. Hope this helps :)

Indeed I made a mistake in the syntax. I didn't understand correctly the README, the /dev/ ommision seemed weird to me.
Sorry and thanks for the answer !

After checking, I can't spindown correctly on this version of TrueNAS Scale. It seems that reading SMART attributes (temperature?) every 5 minutes resets the counter on unused drives.
Here is a picture of one of the tests,

SDA is my boot-pool (with system dataset on it), SMART disabled because it is a virtual drive. Used every 5 sec by TrueNAS.
SDB & SDC are two different unused hard drives (just with empty datasets), SMART on.
SDD is an SSD permanently used by external services, SMART on.

No matter what APM value I set (disabled, 1, 127, 128), or whether I enable/disable the SMART system service (with Never, standby nor sleep option), the SDB and SDC drives never spindown.
The counter is reset every 300 seconds, which is 100% coincident with the periodic SMART reading.

You reported that "No matter [...] whether I enable/disable the SMART system service [...] drives never spindown.".
If I understand it correctly, the drives do not spin down, even if you disbale S.M.A.R.T.. Therefore, S.M.A.R.T don't seem to be the problem here.
Might there be any other services running that causes periodic disk reads? Possible candidates can be:

Running VMs/Jails: https://github.com/ngandrass/truenas-spindown-timer/issues/13
k3s service: https://www.truenas.com/community/threads/prevent-frequent-reads-from-waking-up-hdds.93176/page-3#post-721803

You could try to disable the k3s service by issuing: systemctl disable --now k3s.service

By "SMART service" I meant the system service, which is triggered every 30 minutes. I played with each of its parameters without success. At the moment it is completely disabled.

![image](https://user-images.githubusercontent.com/26553241/199925713-71b0fd13-a0b9-48da-a1ad-beb43f7702d1.png)

On the drive side, the goal is to leave SMART enabled so that at least a manual check can be activated. It is this option that, in my opinion, generates the data flow seen on the graphs.

With "by-drive" SMART deactivated for SDB :

Thanks for your detailed problem description! :+1:
Your settings for the S.M.A.R.T service as well as the drives individual settings look perfectly fine to me. Sidenote: You are not alone with the K3S problem. I found many people complaining about it consuming a huge amount of idle CPU...
One idea that comes to my mind: Reading the S.M.A.R.T. values should NOT cause Disk I/O. However, self tests will. Can you please post a screenshot of your scheduled S.M.A.R.T. tests? You can find them under Tasks > S.M.A.R.T. Tests - All explicitly specified tests will prevent the drives from sleeping.

Thanks for the confirmation for my settings!
I think the same, I don't really understand IO generated by tests.
The original TrueNAS timer works on some disks, even with this permanent IO caused by tests (or temperature reading). But the problem is that it depends on the model of the disk because some interpret it as usage.
I have also seen some feedback for K3S, it's still a pity especially since I haven't found any solution even temporary. On my side, it's about 15-20% of permanent use on a 80W TDP CPU. The bill could be more expensive than the 0-1% normally consumed by TrueNAS in IDLE.
Regarding scheduled tests, I think we don't have control over what happens every 5 minutes. In the "data protection" tab, I have the task setup, which has just one simple SMART test per week, so it must not be related :

An idea came to me. Before I found your script, I modified another script I found on the forums to manage the auto-shutdown of my backup server.

I was looking to accomplish this:

Monday 9:55am: WakeOnLan of my "TrueNAS BACKUP SERVER"
Monday 10:00am : boot done, SMART tests are executed + scrub pools
Monday 10:05am : start backup from my main TrueNAS to "TrueNAS BACKUP SERVER
Monday, later : Backup done
Monday, later + 15 minutes of IDLE : Auto-shutdown of the "TrueNAS BACKUP SERVER"

Despite SMART tests enabled on 4 disks in 2 pools, this server shuts down perfectly after 15 minutes of IDLE, despite SMART tests every 5 minutes.
When I discovered your script I fired mine, but it didn't work so without thinking I put my script back to work.
I now understand that the logic we used is different between your script and the one I adapted for my use. (my bash skills are limited, I don't understand everything in your script)
Yours uses IOSTAT by disks, while mine uses IOSTAT on pools directly. If this is what is happening, then it is "logical" to have found that my shutdown script works, when yours did not (in my case and with trueNAS scale beta).

Ok, your scheduled tasks also look absolutely fine.
Differences in how drives handle the S.M.A.R.T. reads may be an issue though. All systems I looked at up until now didn't show such behavior. This, however, does not mean that it is impossible. Disk controllers are quite a pain when it comes to uniformity of their S.M.A.R.T. interfaces...

I was unaware of the possibility to detect I/O on a pool level using zpool iostat. It should be possible to differentiate between "ZFS pool" and "disk drive" operation mode within the script. This would take some work and proper testing though.
Taking this idea a little further: Can you think of any case in which a drive is NOT part of a zfs pool but needs to be spun down anyway? If not, we could theoretically fully replace disk level I/O detection with the pool based approach. However, I'm not very familiar with ZFS¹ and not aware which operations are factored in by zpool iostat and which are not... I'm a little worried to overlook some cases that might lead to missing important I/O, e.g. during resilvering, scrubs, or some other low-level operations, at this point.
Using ONLY the pool based approach should also allow us to replace the differentiation between TrueNAS Core and SCALE (camcontrol vs hdparm) as a positive side effect :+1:
¹: I'm using TrueNAS only within my basic 8 drive home setup and have not much experience with more sophisticated setups.

One case that comes to my mind is that you might want to hold spare uninitialized drives and like to spin them down until they are added to a pool (e.g. as a replacement for a failed drive). This might be useful for a remote setup that you cannot access regularly.
I'm not sure how many people would do something like this though...

I'm seeing a lot of the same I/O issues as @jojolll on the 22.12.0 SCALE release. I didn't see the same issues on CORE (with the exact same drives)
I wonder if this is some kind of bug in SCALE.

Also having the same issues as @jojolll and @sebirdman on 22.12.0 Scale.

@jojolll @sebirdman @gpatkinson Does you still experience the described problems?
If so I'd try to introduce an option to switch between disk and pool operation mode. This, however, would require quite some adjustments to the script. I could take this as an opportunity to more or less refactor the whole script and get rid of some things that bother me since day one like the quite unintuitive CLI...

Hey everyone,
I created a version that is able to work with both plain device identifiers (i.e. on a per-disk basis) and zpool names (i.e. on a zfs pool basis). You can find it in the feature/zpool-iostat-mode branch.
I was able to successfully test it on my own TrueNAS CORE setup but I'm missing more tests, especially on SCALE. Therefore I would like to ask you to test if this version works for you and fixes the above described problems.
As the README still has to be updated here is a quick summary of the changes:

A new CLI argument was introduced to switch between disk and zpool operation mode: -u MODE
When no operation mode is explicitly given, the script works in disk mode. This completely ignores zfs pools and works as before.
When operation mode is set to zpool by supplying -u zpool, the script now operates on a per-zpool basis. I/O is monitored for the pool as a whole and disks are only spun down if the complete pool was idle for a given number of seconds. ZFS pools are either detected automatically or can be supplied manually (see help text for -i and -m).
Drives are referenced by GPTID in zpools (at least on CORE :pray:. Cloud become a problem on SCALE or other systems since I'm not sure how reliable GPTIDs are used...). Therefore the script creates a mapping from GPTIDs to device identifiers using glabel to be able to spin down the drives of a pool using camcontrol (CORE) or hdparm (SCALE)

Here is an example of translating a disk-based script invocation to a zpool based one:

disk: ./spindown-timer.sh -v -t 3600 -p 600 -i da0 -i ada7
zpool: ./spindown-timer.sh -v -t 3600 -p 600 -u zpool -i freenas-boot -i ssd

Keep in mind that you can always test the script in dry run mode (-d) to circumvent a large number of unwanted spindowns during testing :)
Thanks in advance for your help with testing! :+1:

Hi!
It's been a while since I've thought about it, I had finally let my disks run 24/24...
On this version, it seems that TrueNAS scale does not have the prerequisites installed. An error appears in the logs, and there is no spindown happening. I think it does not detect physical disks (in zpool mode)
root@truenas[~]# ./v2.sh -v -t 30 -p 10 -u zpool -i boot-pool -i SSD
[2023-02-18 16:19:13] Running HDD Spindown Timer version 2.1.0
[2023-02-18 16:19:13] Operation mode: zpool
[2023-02-18 16:19:13] Using disk parameter tool: hdparm
./v2.sh: line 168: glabel: command not found
[2023-02-18 16:19:13] Detected GPTID to disk identifier mappings:
[2023-02-18 16:19:13] Ignoring zfs pool: boot-pool
[2023-02-18 16:19:13] Ignoring zfs pool: SSD
[2023-02-18 16:19:13] Detected zfs pool: HDDNAS
[2023-02-18 16:19:13] Detecting disks in pool: HDDNAS
[2023-02-18 16:19:13] Monitoring drives with a timeout of 30 seconds:
[2023-02-18 16:19:13] I/O check sample period: 10 sec
[2023-02-18 16:19:13] Drive timeouts:
[2023-02-18 16:19:23] Drive timeouts:
[2023-02-18 16:19:33] Drive timeouts:
[2023-02-18 16:19:43] Drive timeouts:
[2023-02-18 16:19:53] Drive timeouts:
[2023-02-18 16:20:03] Drive timeouts:
[2023-02-18 16:20:13] Drive timeouts:
[2023-02-18 16:20:23] Drive timeouts:

The disk-based version seems to detect correctly, despite the error also present on the absence of glabel.

root@truenas[~]# ./v2.sh -v -t 30 -p 10 -i sda2 -i sda1 -d
[2023-02-18 16:24:49] Running HDD Spindown Timer version 2.1.0
[2023-02-18 16:24:49] Performing a dry run...
[2023-02-18 16:24:49] Operation mode: disk
[2023-02-18 16:24:49] Using disk parameter tool: hdparm
./v2.sh: line 168: glabel: command not found
[2023-02-18 16:24:49] Detected GPTID to disk identifier mappings:
[2023-02-18 16:24:49] Detected drive sda as ATA device
[2023-02-18 16:24:49] Detected drive sdb as ATA device
[2023-02-18 16:24:49] Detected drive sdc as ATA device
[2023-02-18 16:24:49] Monitoring drives with a timeout of 30 seconds: sda sdb sdc
[2023-02-18 16:24:49] I/O check sample period: 10 sec
[2023-02-18 16:24:49] Drive timeouts: [sda]=30 [sdb]=30 [sdc]=30
[2023-02-18 16:25:00] Drive timeouts: [sda]=20 [sdb]=20 [sdc]=30
[2023-02-18 16:25:10] Drive timeouts: [sda]=10 [sdb]=10 [sdc]=30
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[2023-02-18 16:25:20] Drive is already spun down: sda
[2023-02-18 16:25:20] Would spin down idle drive: sdb. No spindown was performed (dry run).
[2023-02-18 16:25:20] Drive timeouts: [sda]=30 [sdb]=30 [sdc]=20
[2023-02-18 16:25:30] Drive timeouts: [sda]=20 [sdb]=20 [sdc]=10
[2023-02-18 16:25:40] Would spin down idle drive: sdc. No spindown was performed (dry run).
[2023-02-18 16:25:40] Drive timeouts: [sda]=10 [sdb]=10 [sdc]=30

Thanks for the quick test! Could you provide the output of zpool list -v so that I can determine which device identifiers are used if glabel is absent? Thanks!
In theory we could skip the whole GPTID detection for disk operation mode since it does not look up those values anyway.
Regarding the SG_IO message: This seems to originate from https://github.com/ngandrass/truenas-spindown-timer/blob/feature/zpool-iostat-mode/spindown_timer.sh#L406. Could you try to execute the following command for your drives sda, sdb, and sdc: hdparm -C /dev/sdX?

Ok, it seems that GPTID is used on BSD (CORE) only. SCALE seems to identify the disks inside the pool by partuuid.
I updated the script accordingly. With commit 1d1bed03b77704abb2a0285db5a9dfc22a8669a4 the zpool operation mode should (or is it more appropriate to say "could" :pray:) work on SCALE. Please try again :)

Thanks for your reply, the new version work in both modes 🥇

The error message actually comes from the SDA disk, the boot-pool. It's a virtual disk in my case (TrueNAS is virtualized by Proxmox). I think Hdparm can't read its status correctly beacause it's a virtual disk. However, the script seems to ignore the error as it is. In any case I don't see why to ask the script to include this disk, once excluded there is no more error.

root@truenas[~]# hdparm -C /dev/sda
/dev/sda:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 drive state is:  standby
root@truenas[~]# hdparm -C /dev/sdb

/dev/sdb:
 drive state is:  active/idle
root@truenas[~]# hdparm -C /dev/sdc

/dev/sdc:
 drive state is:  active/idle
root@truenas[~]# hdparm -C /dev/sdd

Awesome! I'll wait for the test results from @gpatkinson, update the README and release a new version when no further error arise. Thanks for testing :tada:
Regarding the virtual disk: Yeah, hdparm will fail for virtual disks and cannot spin them down.

Wow, great work @ngandrass ! Mine appears to work as well. In this first test, sdb has per-disk SMART off and sde has it on. The script does not care either way:

And here's the non-dry-run, per-disk SMART now enabled on both disks of the pool:

Just for clarity:

trunas scale bluefin 22.12.0 on bare metal
no VMs running
k3s and all containers stopped manually

This script and new iostat zpool functionality appears to be working. Will be testing soon how k3s and running containers affect the script. @jojolll -- have you tested anything with applications / containers at this point? (not running on the pool to spin down, obviously, I seemed to be having issues with just having k3s running on the system)

Awesome! Nice to hear that everything works well for you :+1: Thanks for testing!
I updated the README and created #18 to prepare for release. If no show stoppers arise I could release the new version later today. Any objections?

Wow, great work @ngandrass ! Mine appears to work as well. In this first test, sdb has per-disk SMART off and sde has it on. The script does not care either way:

And here's the non-dry-run, per-disk SMART now enabled on both disks of the pool:

Just for clarity:
* trunas scale bluefin 22.12.0 on bare metal

* no VMs running

* k3s and all containers stopped manually

This script and new iostat zpool functionality appears to be working. Will be testing soon how k3s and running containers affect the script. @jojolll -- have you tested anything with applications / containers at this point? (not running on the pool to spin down, obviously, I seemed to be having issues with just having k3s running on the system)

On my side, SMART is a problem on my recent disk (Seagate IronWolf 4Tb). If I leave the option activated in the disk menu, even if the script works well, the disk spins up after 5 minutes maximum of spin down (when TrueNAS reads the SMART). On another disk I don't have the problem, so it seems to be only related to the disk behavior.
I have not tested with K3S, but there is no reason why there could be a data exchange on disks not used by the services :)
When I enabled it on my SSD, I never heard my HDD working, which means there was no IO.
I'll see if I can find a way for SMART not to wake up my disk, and if not I'll leave SMART disabled and only run a manual test every now and then :(

I configured my S.M.A.R.T. daemon to execute tests every sunday night. It wakes the drives only for this task and leaves them sleeping for the rest of the time. At least on CORE... There seem to be a lot of behavioral differences between CORE and SCALE :disappointed:
Just to make sure: Note that having an active shell session inside a directory that resides on a disk can cause I/O and therefore reset the spindown timer.

t to make sure: Note that having an active shell session inside a directory that resides on a disk can cause I/O and therefo

In my case, the timer is never reset. There is no I/O on the pool even on SMART read. I think it is the temperature reading on the disk, every 5 minutes, that causes this.

I will try to add options to SMART.

@ngandrass @jojolll one thing i hadn't noticed until this morning is that using the Truenas GUI to read the disk settings (not even hitting the 'edit' button) pulls a disk out of standby/spin-down. i had one disk of my mirrored pool spun down this morning and the other active. As soon as I used the GUI to look at the settings on the spun-down drive, it became active again. I guess this makes sense as it needs to read these settings from the drive itself, I just hadn't considered it and this could confound testing in the future, FYI.

pulls a disk out of standby/spin-down. i had one disk of my mirrored pool spun down this morning and the other active. As soon as I used the GUI to look at the settings on the spun-down drive, it became active again. I guess this makes sense as it needs to read these settings from the drive itself, I just hadn't considered it and this could confound testing in the future, FYI.

I've noticed that too, but in my case it's something else since even after leaving everything for 30 minutes, if the script puts my HDD to sleep, it will only stay there for a short time.

This is rather interesting. I tried to reproduce this behavior but weren't able to. Even reading the S.M.A.R.T. data of a drive via smartctl -a /dev/ada0 did not wake it.
I'm using primarily Western Digital Red drives on TrueNAS CORE. IMHO it's possible that this behavior depends on both whether CORE or SCALE is used and how the controller of the respective drive handles S.M.A.R.T. reads...

I'm using primarily Western Digital Red drives on TrueNAS CORE. IMHO it's possible that this behavior depends on both whether CORE or SCALE is used and how the controller of the respective drive handles S.M.A.R.T. reads...

I think you're right. I bet there's some low-level BSD vs Linux difference that is causing the difference

Okay, I've been doing some more basic testing around the potential k3s / apps issues with spinning down drives. I've basically been able to show that running the k3s service / apps panel does not spin up my zpool on my 22.12.0 SCALE system and that all containers that don't touch the -u zpool that i've specified are fine to be running. I suspected this would be the case but I had read some threads elsewhere that suggested there could be a problem here.
the '-u zpool' option is really great in that it solves the problem of my sdX values rotating randomly with reboots ... a POSTINIT script would have been trickier to run in my setup otherwise.
question for either of you @ngandrass @jojolll -- what's your preferred method to track the last XX hrs of a drive's idle status? I want to leave this running for a number of hours to make sure that I am not having intermittent spin-ups that I'm just not catching ...

question for either of you @ngandrass @jojolll -- what's your preferred method to track the last XX hrs of a drive's idle status? I want to leave this running for a number of hours to make sure that I am not having intermittent spin-ups that I'm just not catching ...

I was using a rather old script that was quite tailored to my specific setup, hence not flexible enough to publish here. Instead, I introduced a check mode to the spindown timer (-c) to display the current power state of all monitored drives every poll interval. See README.md > Using the check mode for more details.
I hope this helps :)

 I will try to add options to SMART.

For your information, after multiple tests the different options in this field seem... ignored 👎 my HDD starts every 5 minutes at exactly the same second (2:11, 7:11, 12:11, etc.) if I spin it down.
My only option seems to be to disable SMART by disk, which disables this uncontrollable I/O every 5 mins.
Even if I choose not to spin down my disks, I hear a slight movement of the disk's read head every 5 minutes (at the same second), so it is understandable that if the same command is issued to it in standby then it comes out of standby.
This situation irritates me a little :(

If you notice I/O and even physical disk head movements this cannot be due to the temperature reads. There must be more to it...
Have you double checked for running VMs, Services, k3s, ...? Have you checked the configuration of the S.M.A.R.T. service under Services > S.M.A.R.T. as well as all kinds of scheduled Tasks under Tasks > *?
And just to be sure: Which POLL_TIME have you set for the spindown timer script? If no explicit time is given via -p <POLL_TIME> 600 seconds will be used. I just want to make sure that this is no inference from the script.

I'm experiencing 5 minute reads identical to @jojolll on TrueNAS-SCALE-22.12.0.
The script does appear to ignore these 5 minute reads, suggesting that they are S.M.A.R.T. related. When a drives is spun down, however, it is inevitably spun back up when the 5 minute interval elapses.
It was mentioned here that collectd checks the temperature every 5 minutes via S.M.A.R.T... perhaps that has something to do with it.
My S.M.A.R.T. configuration for the service under Services > S.M.A.R.T. has Check Interval = 30 and Power Mode = Standby. My only test under Data Protection > S.M.A.R.T. Tests is a Long test once per month.

@joelightshoe Have you confirmed that your system dataset is placed on a pool that is independent from the disks you are trying to spin down? See System Dataset Placement
I found some people that experience similar problems [1], [2]. There is also a Jira issue [3], but it was closed without any changes to TrueNAS.
And here are my configurations of the S.M.A.R.T. service and the disks for reference:

Maybe this helps...

[1]: https://www.truenas.com/community/threads/reads-every-5-minutes-but-not-on-core.91396/
[2]: https://www.truenas.com/community/threads/prevent-frequent-reads-from-waking-up-hdds.93176
[3]: https://ixsystems.atlassian.net/browse/NAS-109628

Thanks @ngandrass, yes I have my system dataset on the boot-pool. I also have my Apps (aka ix-applications dataset) on a completely separate ssd.
As I mentioned, the hdd's in the zpool I have configured to spin down have no I/O when I add the SMART extra options -n standby -d ata, but they still spin up on the 5 minute interval. The only difference I can tell between our disk configurations is that my drives are Seagate Barracuda.

I'm going to try to completely disable temperature monitoring and see what happens, perhaps by setting the HDD Standby value to something like 330 minutes, or maybe there's a way to manually disable it. I'll let you know what I find out.

Adding n standby d ata seems to have solved the issue of disks spinning back up immediately for me on Bluefin.

Thanks for reporting back @Brightwater !
I'm trying to derive general instructions from your findings to add them to the README. If I see it correctly, you are manually adding -n standby -d ata to every disk, right?

The -d ata option overwrites the auto detected device type and should therefore not be related to the spindown issues in general (but can be on your particular system nonetheless!)
The -n standby option excludes all drives from smartd that are either in STANDBY or SPINDOWN mode. These two modes are used to signal spindown on most disks. Adding this option most likely suffices to fix the spindown issues.

So far, so good. However, the -n option can also be set globally via the smartd config (See smartd.py#L48). This would simplify the solution significantly. On TrueNAS Core this setting is located under Services > S.M.A.R.T. > General Options.
Could you please try if specifying "Power Mode: Standby" globally solves the spindown issues on SCALE?
If so, we could make this info more visible within the README. Thanks a lot !!

@ngandrass Just tried out that option (setting "Power Mode: Standby" globally) on my scale setup. Looks like at least for my case it did not work.

I added that setting and removed the extra smart options from my sde and sdf drives. Those 2 drives spun back up after a few minutes, while the other 3 drives that still had the n standby d ata set remained spun down.
Another thing to note is that when setting the global standby setting, it removes temperature monitoring from all the drives even when they are not spun down. It also gives a note that smart tests are only ran if the setting is set to "never". I will have to see if setting the options manually for each drive affects the smart test, but I can confirm it does at least retain the temperature monitoring while the drives are not spun down.
One thing that may help with setting this more easily is at least on scale there is an option to bulk edit the settings for all disks. So if you just select each disk you can apply the extra smart values for all of them at the same time.

Note at least on scale it does not allow you to put -n standby -d ata it will only accept the options without "-"s. Like n standby d ata

@Brightwater Thanks for conducting the tests! Reading your findings I'm a little worried by the fact that SCALE does not allow you to add dashes to the S.M.A.R.T. extra options. Which gave me an idea...

According to the SCALE docs, the S.M.A.R.T. Extra Options field allows to "Enter additional smartctl(8) options".
According to smartctl(8) these options do need to contain dashes.
These options are simply prepended to the config line (see smartd.conf(5)) for the specific disk in the generated smartd.conf. The config file is generated by smartd.py::get_smartd_config(). On line 48 the disk-specific args are added:

48: config = f"{args} -n {disk['smart_powermode']} -W {difference}," \
49:              f"{informational},{critical}"

A valid smartd.conf could look like this:
/dev/sda -n standby -W 4,35,40
/dev/sdb -n standby -W 4,35,40
/dev/sdc -n standby -W 4,35,40

However, specifying the S.M.A.R.T. Extra Options for a single drive (/dev/sda in the following example) will produce an invalid config:
/dev/sda n standby d ata -n standby -W 4,35,40     # <- This row is invalid
/dev/sdb -n standby -W 4,35,40
/dev/sdc -n standby -W 4,35,40

This will most likely cause smartd to fully exclude /dev/sda from the set of monitored drives, resulting in a complete disabling of S.M.A.R.T. checks for the specific drive.

Taking considerations further, the -W switch, always added by smartd.py::get_smartd_config() regardless of config options, will enable temperature monitoring. This might be what prevents your specific drives from spinning down. Unfortunately the behavior of this option is fully undocumented by smartd(8), if I'm not mistaken, but at least mentioned in the ArchWiki.

So... What are we left with now? There is still the -n {disk['smart_powermode']} part of the smartd.conf line generation. disk['smart_powermode'] is directly sourced from smart.py. This corresponds to the Power Mode setting located under System Settings > Services > S.M.A.R.T. (See TrueNAS SCALE docs for S.M.A.R.T. Service Screen).
Sadly, this was reported to not work and kept waking up drives :(

We now know, that smartd will wake up drives when the -W option (temperature monitoring) is supplied, regardless of the set power mode. I checked the source code of smartd, but it seems to honor the supplied power mode before performing any checks (See smartd.cpp#L3474).
However, it could still be possible that your device returns an non ATA compliant power mode value, resulting in ignoring of the -n directive (See smartd.cpp#L3551). This would produce a critical message in the smartd log. Can you check it on your system for any such messages? Thanks!

To conclude: This currently leaves us with two potential actions and one very dirty hack:

Filing a TrueNAS bug report that there is a problem with smartd not respecting the power mode. Maybe it is not passed correctly to smartd after setting it under System Settings > Services > S.M.A.R.T.? Also it would be worth to point out that the disk specific S.M.A.R.T. Extra Options do not allow to enter dashes, resulting in broken config files.
If it turns out that your drive indeed returns non ATA compliant value you could try to reach out to the developers of smartd.

And the very dirty hack would be to completely disable S.M.A.R.T. for specific disks by intentionally breaking the smartd config file. But this is... well... not very nice :/

-n and -d cannot be specified in SMART extra options because these are set automatically by TrueNas. (This is not a global issue for "-" as you can e.g. add -q.
IMO the -n value in /etc/smartd.conf should always be set to -n standby rather than always set to -n never and I have raised a TrueNas ticket for this: https://ixsystems.atlassian.net/browse/NAS-124148