End-to-end tests for terraform modules

Creating a draft for early review

Checklist

I applicable, I acknowledge that I have:

[ ] Read the contributing guide
[ ] Ran terraform fmt on all modified files
[ ] Regenerated the relevant README.md files using tools/tfdoc.py
[ ] Made sure all relevant tests pass

nit: can you use a local branch next time? :)

Waiting impatiently for this to land, let me know if you want me to help in any way. :)

Ok, this is ready for final review.
What I left for next PR, is integrating tests/examples_e2e/README.md into CONTRIBUTING.md

🥳
I know where my weekend is going, now...


Dark mode

Hello. Thank you very much for great addon. I am using it to generate screenshot of some cards and send to telegram.
Is it possible to add dark mode to browser setting or some arguments for browser ?
Find something about that, but i don't where to change that on my server)
async function renderUrlToImageAsync(browser, pageConfig, url, path) {   let page;   try {     page = await browser.newPage();     await page.emulateMediaFeatures([       {         name: "prefers-color-scheme",         value: "light"       }     ]);

@dmutpuul please check the latest version, the dark mode feature is now available via #97

@deenter Hello! Have no updates so i reinstalled addon. I have preferred color option in config, but it doesn't work( still no dark screenshots...

Seems to be an issue when using it directly as an addon via HACS,
using the latest docker image everything works as expected for me.


Refactoring logic from Function into the Service and Core projects

fixes #838

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

16 of 115 (13.91%)  changed or added relevant lines in 3 files are covered.
101 unchanged lines in 8 files lost coverage.
Overall coverage decreased (-4.2%) to 72.124%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/DevOpsMetrics.Service/Controllers/DORASummaryController.cs
3
102
2.94%

Files with Coverage Reduction
New Missed Lines
%

src/DevOpsMetrics.Service/Controllers/DORASummaryController.cs
2
7.38%

src/DevOpsMetrics.Core/ChangeFailureRate.cs
3
94.44%

src/DevOpsMetrics.Core/Models/Common/Build.cs
3
90.77%

src/DevOpsMetrics.Core/Models/Common/ChangeFailureRateModel.cs
5
70.83%

src/DevOpsMetrics.Core/Models/Common/ProjectLog.cs
9
84.0%

src/DevOpsMetrics.Core/DataAccess/LeadTimeForChangesDA.cs
15
49.49%

src/DevOpsMetrics.Core/DataAccess/TableStorage/AzureTableStorageDA.cs
18
82.52%

src/DevOpsMetrics.Core/DataAccess/ChangeFailureRateDA.cs
46
58.24%

Totals

Change from base Build<PHONE_NUMBER>:
-4.2%

Covered Lines:
2282

Relevant Lines:
3094

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

18 of 114 (15.79%)  changed or added relevant lines in 5 files are covered.
148 unchanged lines in 13 files lost coverage.
Overall coverage decreased (-3.5%) to 72.82%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/DevOpsMetrics.Service/Controllers/DORASummaryController.cs
3
99
3.03%

Files with Coverage Reduction
New Missed Lines
%

src/DevOpsMetrics.Core/Models/AzureDevOps/AzureDevOpsBuild.cs
1
26.67%

src/DevOpsMetrics.Core/Models/AzureDevOps/AzureDevOpsPRCommit.cs
2
0.0%

src/DevOpsMetrics.Core/Models/Common/Committer.cs
2
0.0%

src/DevOpsMetrics.Core/ChangeFailureRate.cs
3
94.44%

src/DevOpsMetrics.Core/DataAccess/TableStorage/PartitionKeys.cs
3
91.67%

src/DevOpsMetrics.Core/Models/AzureDevOps/AzureDevOpsPR.cs
4
0.0%

src/DevOpsMetrics.Core/Models/Common/ChangeFailureRateModel.cs
5
70.83%

src/DevOpsMetrics.Service/Controllers/DORASummaryController.cs
5
7.38%

src/DevOpsMetrics.Core/Models/Common/ProjectLog.cs
9
84.0%

src/DevOpsMetrics.Core/DataAccess/APIAccess/AzureDevOpsAPIAccess.cs
11
75.0%

Totals

Change from base Build<PHONE_NUMBER>:
-3.5%

Covered Lines:
2303

Relevant Lines:
3094

💛  - Coveralls


New css from prototypes

The following commit is an aggregate of all changes currently at http://github.com/codeforamerica/rva-screening-ui-prototypes in order to prepare for our upcoming trip to RVA.
Major changes include:

custom CSS (removed bootstrap)
updated search/create patient workflow, now "Search First" (located at index.html)
functionality for filter patients has been combined with search into index.html. Filters are going to be statistical aggregates of patients, such as (3 patients have been shared with your clinic today) > clicking that button will show you which patients.
JS now broken into unique files for simpler editing, also exposed globally as major constructors
main navigation is mobile "swipe" friendly (but needs some major work and testing)

TODO before this should be merged:

[x] re-introduce bootstrap forms.css so we can still have nice looking forms on the patient details page
[x] update .container in some pages that are currently missing
[ ] re-add prescreening styles from old _base.scss file.
[ ] style login page
[ ] make "View Patients" filters more mobile friendly - probably turn into a dropdown menu instead of buttons on mobile

@esmithayer I added the styles back from the original site for the prescreener, but I'm not able to run python add_screening_criteria_data.py successfully, so I can't really see any changes. Let me know if you're able to take a look!
Very likely that it won't look great :stuck_out_tongue_closed_eyes:

Oh yeah, that needs the new create_app configuration. One sec.

Fixed in #182.

Boom! :boom:

I've added back the styles for the prescreener and results. Also included a new sidebar navigation (breadcrumb style) just as an experiment.

You can also use the sidebar breadcrumbs as a way to progress the prescreener app:

I've also added styles for .errors per @esmithayer's addition in #178


Rename MODULE_TEXT to TEXT_HEAP.

Summary
Rename MODULE_TEXT to TEXT_HEAP as the latter is more generic.  Also renames other variations like modtext, etc.
Impact
The usage of CONFIG_ARCH_MODULE_TEXT.  It has to be changed to CONFIG_ARCH_TEXT_HEAP.
Testing
esp32-devkitc:elf esp32-devkitc:sotest esp32-devkitc:module

This will fail in ESP32-C3 checks.  I'll wait for https://github.com/apache/incubator-nuttx/pull/3945 then I'll rebase and change the names for C3 as well.

i have no problem with the new name.

@Ouss4
LGTM but it seems that there were build errors.

Error: chip/esp32c3_modtext.c:50:6: error: function declaration isn't a prototype [-Werror=strict-prototypes]
void up_module_text_init()

@Ouss4 the new name looks good, please rebase your change, #3945 just merged.

@Ouss4 the new name looks good, please rebase your change, #3945 just merged.

Thanks, just did.
Regarding the previous build failures, that was expected.  I didn't update ESP32-C3 until #3945 was merged.


Can not import component

Can not import component to lb4 application.Please help

@dangmiracle can you share the error/issue you are facing, possibly with a screenshot or stacktrace?

@dangmiracle share the LB4 version you are using along with package version you are using.

facing same issue

Can anyone of @dangmiracle, @HarshalNathe or @abjagtap share the version of @loopback/context used

@akshatdubeysf I'm using the ^3.15.0 version of @loopback/context

@HarshalNathe the latest version of loopback4-helmet works with 3.18.0, and there seems to be some typing incompatibility between these two versions from the Loopback package, you can try updating your context package to 3.18.0

I confirm : Doesn't work with @loopback/context 4.0.0. Work better with 3.18.0

Is there a solution, or at least a plan for fixing this?

HI @markus-hsk , the helmet package has been updated, you can try the latest version, which works with @loopback/context 5.0.0.

Thank you @akshatdubeysf , works!

Hi @akshatdubeysf
I have to configure loopback4-helmet in my application, when I am trying to add component in application.js, I am facing same issue and I have tried all the solution which you discussed above. Please help me on this
versions:-
"@loopback/boot": "^3.4.2",
"@loopback/context": "^5.0.0",
"@loopback/core": "^2.16.2",
"@loopback/repository": "^3.7.1",
"@loopback/rest": "^10.0.0",
"@loopback/rest-explorer": "^3.3.2",
"@loopback/service-proxy": "^3.2.2",
"loopback-connector-rest": "^4.0.1",
"loopback4-helmet": "^4.2.0",
Error: Argument of type 'typeof Loopback4HelmetComponent' is not assignable to parameter of type 'Constructor'.
Types of construct signatures are incompatible.
Type 'new () => Loopback4HelmetComponent' is not assignable to type 'new (...args: any[]) => Component'.
Construct signature return types 'Loopback4HelmetComponent' and 'Component' are incompatible.
The types of 'bindings' are incompatible between these types.

@Rahulgojra make sure you are using correct version of @loopback/context


Create an error when trying to create a room name that is already in use

I am working on a solution for this.

working on this one as well

will work on a different ticket.

#36 also addresses the following

Should also error when trying to take an existing 2N hardcoded URL, such as /streamer.


Documentation incorrect for GET /users/{username}/activity/current

The documentation specifies that the GET request to /users/{username}/activity/current always specifies that the started field is in UTC time (by including the Z at the end):
{
    "started": "YYYY-MM-DDTHH:MM:SS.ssssssZ",
    "duration": "10",
    "heartbeat": {
        "language": "c",
        "hostname": "hostname1",
        "editor_name": "Neovim",
        "project_name": "cool_project22"
    }
}

However, in reality that "Z" is not included. Here is an example response that I got:
{
  "started": "2023-12-17T12:03:20.938417739",
  "duration": 240,
  "heartbeat": {
    "project_name": "testaustime-frontend",
    "language": "typescript",
    "editor_name": "vscode",
    "hostname": "ideapad"
  }
}

I believe the same problem exists for all other timestamps that are returned from any endpoint. We already have an issue in the frontend for this, but I think it would be best to fix in the backend.
Also, it seems that the duration field is a number instead of a string.

The easiest thing would be just to use DateTimes that include a timezone (most likely just UTC). This could be achieved with minor database refactoring.


ttfautohint in Windows XP - icons disappear

The icons disappear when I change or enable/disable any CSS properties in Chrome DevTools. It happens in Win XP only. What could it be?

Hi!
I have the same problem on Windows XP in Firefox, Chrome, Opera and Yandex.Browser (it's based on Chromium).
I had created a virtual machine in the VirtualBox and then I had installed the last versions of these browsers. Then I had created the simple HTML page with jQuery, Bootstrap and custom Fontello font (from SVG source). When I open that page in the browsers, icons sometimes disappear. If I
repeatedly press Ctrl+F5 in the browsers, icons sometimes disappear and then appear. But Bootstrap Glyphicons are displayed always!
What can I do to solve this problem?
P. S. Demo page also works wrong.

I had got problem on four different real PC and on one virtual PC. Bootstrap Glyphicons always work properly.

Could you help to create a minimal demo with instructions? Desired:

Font with one simple glyph (zip from fontello), if possible.
Link to virtual machine (i guess, from MS site), where this bug can be reproduced.

If you have a time - try to resave font via fontforge or any other font editor, and dump via ttx utility from fonttools package to see difference (if resave provides forking font).
One more notice - disable hinting in fontello setting, to make sure that's not ttfautohint bug.

Edited title - problem is specific to XP, not to Chrome only.

Okay....
I downloaded and started the virtual machine "IE8 on XP" for VirtualBox from this url: https://dev.windows.com/en-us/microsoft-edge/tools/vms/windows/.
Then I created font fontello-1d1dbd1b with only one symbol and fontello-bc676347 with four symbols (for my project). Then I simplify my project and put all files into fontello_bag.zip.
So you must copy the archive to VM and open custom.html in IE or Firefox (you must download it before). Press repeatedly Ctrl+F5 until the icons will have disappeared.
Example of issue is on .

Sorry, I forgot to disable hinting. When I disabled it, the problem disappeared.

Sorry, I forgot to disable hinting. When I disabled it, the problem disappeared.

Thanks a lot for info! I'll try to rebuild a new version.

http://www.freetype.org/ttfautohint/
Could you try to apply ttfautohint 1.5 manually?

build font without hinting
apply ttfautohint https://github.com/fontello/fontello/blob/master/internal/fontello/font_build/_lib/worker.js#L152-L155
leave TTF file only (delete another formats)

Does 1.5 version help?

I can try it only after weekend. Sorry.

Okay.

I took the last build without hinting (fontello_without_hint.ttf - 2.1kB).
I applied ttfautohint: ttfautohint --no-info --windows-compatibility --symbol fontello_without_hint.ttf  fontello_with_manual_hint.ttf
I got the font with hinting (fontello_with_manual_hint.ttf - 5.6kB).
I specified new font in my CSS and removed other font types.

...and the problem came back.

@vykulakov thanks for info.
Reported to Werner Lemberg (developper of ttfautohint).

Thank you for the wonderful service!

Where can I download fontello_without_hint.ttf?

@lemzwerg settings menu -> advanced font settings -> hinting checkbox. It defines should ttfautohint be applied or not.
Then select some icons & download. If more help needed - let me know.

Where can I download fontello_without_hint.ttf?

You can find it in the archive fonts.zip.

The problem is a bug in ttfautohint – thanks for the report :-)
A quick fix is to add -f latn to the ttfautohint command line arguments.

@vykulakov could you check? Updated fontello code.

I can not reproduce the problem with your changes -  the problem disappeared. I will try to use new font in my project.

@vykulakov can we close this issue?

No, I do not have the button "Close Issue".

Close as solved


driver adaptor

Fixes #
Implementation of Driver adaptor
This pull introduces/changes:
*
*
Pings:
@klobuczek
@amitsuryavanshi

Coverage decreased (-44.2%) to 41.979% when pulling d178bfea0690b7eb596311837f67bcee34234089 on driver into 11b5b2dc727b70f8e717302f7fa9eee4c470a126 on master.

Coverage decreased (-44.2%) to 41.979% when pulling d178bfea0690b7eb596311837f67bcee34234089 on driver into 11b5b2dc727b70f8e717302f7fa9eee4c470a126 on master.


Not working whatsoever

It used to work a few days ago, but now it's not working at all.
Here's the Problem Signature.
Problem signature:
Problem Event Name:	CLR20r3
Problem Signature 01:	Unity Studio.exe
Problem Signature 02:	<IP_ADDRESS>
Problem Signature 03:	5650f254
Problem Signature 04:	System.Xml
Problem Signature 05:	4.6.1064.2
Problem Signature 06:	565f9c84
Problem Signature 07:	b79
Problem Signature 08:	27
Problem Signature 09:	IOIBMURHYNRXKW0ZXKYRVFN0BOYYUFOW
OS Version:	6.3.96<IP_ADDRESS>.768.101
Locale ID:	1033
Additional Information 1:	5861
Additional Information 2:	5861822e1919d7c014bbb064c64908b2
Additional Information 3:	1318
Additional Information 4:	13181ae637ac2592fac0ca2dd662cdcc
Read our privacy statement online:
http://go.microsoft.com/fwlink/?linkid=280262
If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt

Use: https://github.com/Perfare/UnityStudio


For Personal Pages

Not and issue, More a feature request,
is it possible for the package to allow for posting to a user instead of a page on facebook?
great package btw.
Bests

That would be great!


Revise the implementation of set_activation_checkpointing

Context
What is the purpose of this PR? Is it to

[x] add a new feature
[x] fix a bug
[x] update tests and/or documentation
[ ] other (please add here)

set_activation_checkpointing is a bit restrictive in that it only allows policies to be taken in via a defined set instead of allowing users to also author policies. We'll soon be improving our activation checkpointing to make llama3 training more memory efficient, which will require a more custom policy this PR will enable. More context in https://github.com/pytorch/torchtune/issues/781. Also improving some of the documentation around the API and removing allowing passing of None into the policy since this does not make sense
Changelog

Improve documentation
Stop allowing None passed as the policy
Allow custom policy to be passed in, in addition to sets
Add unittests

Test plan
Please make sure to do each of the following if applicable to your PR. (If you're not sure about any one of these just ask and we will happily help.)
pytest test_memory.py -v

[ ] run pre-commit hooks and linters (make sure you've first installed via pre-commit install)
[x] add unit tests for any new functionality
[ ] update docstrings for any new or updated methods or classes
[ ] run unit tests via pytest tests
[ ] run recipe tests via pytest tests -m integration_test
[ ] manually run any new or modified recipes with sufficient proof of correctness

[ ] include relevant commands and any other artifacts in this summary (pastes of loss curves, eval results, etc.)

Codecov Report
Attention: Patch coverage is 44.44444% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 27.00%. Comparing base (a978956) to head (f5b2095).
Report is 3 commits behind head on main.

Files
Patch %
Lines

tests/torchtune/utils/test_memory.py
45.45%
12 Missing :warning:

torchtune/utils/memory.py
40.00%
3 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #866       +/-   ##
===========================================
- Coverage   66.39%   27.00%   -39.40%
===========================================
  Files         155      156        +1
  Lines        6484     6592      +108
===========================================
- Hits         4305     1780     -2525
- Misses       2179     4812     +2633

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Starting VS Code from conda environment fails to activate that environment

Type: Bug

Behaviour
When starting Visual Studio Code from the terminal with a conda environment activated does say that environment is selected in the status bar (and with the 'run' button, scripts run using that environment's interpreter) but that environment is not activated in the terminal.
Steps to reproduce:

Open a terminal and activate an environment:conda activate my_env

Start Visual Studio Code:code .

See that that my_env is active, according to the statusbar.
See that pressing the run button does run a Python script using my_env's Python interpreter.
Open an integrated terminal.
See that the prompt gives no indication my_env is activated.
Run:conda info

and see that only the base environment is activated.
Alternatively, start VS Code from a fresh terminal using:
conda run -n my_env code .

with the same results.

Diagnostic data
NOTE: the environment I tested with was actually called cli-board-manager. Sorry about that.

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2024-06-07 15:43:46.313 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-06-07 15:43:46.313 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-06-07 15:43:46.313 [info] Experiment 'pythonTestAdapter' is active
2024-06-07 15:43:46.345 [info] Default formatter is set to null for workspace /Users/david/tmp/test-code-env
2024-06-07 15:43:46.352 [info] Test server listening.
2024-06-07 15:43:46.352 [info] VS Code was launched from an activated environment: 'cli-board-manager', selecting it as the interpreter for workspace.
2024-06-07 15:43:46.352 [info] Python interpreter path: /opt/homebrew/Caskroom/miniconda/base/envs/cli-board-manager/bin/python
2024-06-07 15:43:46.363 [info] > pyenv which python
2024-06-07 15:43:46.363 [info] cwd: .
2024-06-07 15:43:46.492 [info] > conda info --json
2024-06-07 15:43:46.492 [info] shell: zsh
2024-06-07 15:43:46.897 [info] > conda run -n cli-board-manager --no-capture-output python ~/.vscode-insiders/extensions/ms-python.python-2024.4.1/python_files/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2024.4.1/python_files/printEnvVariables.py
2024-06-07 15:43:46.897 [info] shell: zsh
2024-06-07 15:43:47.313 [info] Starting Pylance language server.
2024-06-07 15:43:47.315 [info] > /opt/homebrew/bin/python3.12 ~/.vscode-insiders/extensions/ms-python.python-2024.4.1/python_files/printEnvVariables.py
2024-06-07 15:43:47.315 [info] shell: zsh
2024-06-07 15:43:47.361 [info] Setting environment variable CONDA_SHLVL in collection to 4 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.361 [info] Setting environment variable CONDA_PROMPT_MODIFIER in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Setting environment variable _CE_CONDA in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Setting environment variable CONDA_PREFIX_3 in collection to /opt/homebrew/Caskroom/miniconda/base {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Setting environment variable CONDA_PREFIX_2 in collection to /opt/homebrew/Caskroom/miniconda/base/envs/cli-board-manager {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Setting environment variable _CE_M in collection to  {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Setting environment variable XPC_SERVICE_NAME in collection to 0 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-07 15:43:47.362 [info] Send text to terminal: /opt/homebrew/bin/python3.12 /Users/david/.vscode-insiders/extensions/ms-python.python-2024.4.1/python_files/printEnvVariablesToFile.py /Users/david/.vscode-insiders/extensions/ms-python.python-2024.4.1/python_files/deactivate/zsh/envVars.txt
2024-06-07 15:43:52.355 [info] > conda info --json

Output for Python Debugger in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python Debugger)

2024-06-07 15:43:46.494 [info] Name: Python Debugger
2024-06-07 15:43:46.494 [info] Module: debugpy
2024-06-07 15:43:46.494 [error] sendDebugpySuccessActivationTelemetry() failed. [Error: Extension 'ms-python.debugpy' CANNOT use API proposal: debugVisualization.
Its package.json#enabledApiProposals-property declares: portsAttributes, contribIssueReporter but NOT debugVisualization.
 The missing proposal MUST be added and you must start in extension development mode or use the following command line switch: --enable-proposed-api ms-python.debugpy
	at E (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:140:50959)
	at Object.registerDebugVisualizationTreeProvider (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:162:43818)
	at t.registerDebugger (/Users/david/.vscode-insiders/extensions/ms-python.debugpy-2024.6.0-darwin-arm64/dist/extension.js:2:880949)
	at e.activate (/Users/david/.vscode-insiders/extensions/ms-python.debugpy-2024.6.0-darwin-arm64/dist/extension.js:2:1128292)
	at W.jb (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:13382)
	at W.ib (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:13092)
	at /Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:158:11114
	at async c.n (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:142:6231)
	at async c.m (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:142:6194)
	at async c.l (/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:142:5651)]

Extension version: 2024.4.1
VS Code version: Code - Insiders 1.88.0-insider (Universal) (be210b3a60c7d60030c1d3d92da00d008edf6ab9, 2024-03-15T05:49:14.505Z)
OS version: Darwin arm64 23.5.0
Modes:

Python version (& distribution if applicable, e.g. Anaconda): 3.12.3
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

Installed Extensions

Extension Name
Extension Id
Version

JavaScript Debugger
ms-vscode.js-debug
1.87.0

JavaScript Debugger Companion Extension
ms-vscode.js-debug-companion
1.1.2

Pylance
ms-python.vscode-pylance
2024.6.1

Python
ms-python.python
2024.4.1

Python Debugger
ms-python.debugpy
2024.6.0

Table Visualizer for JavaScript Profiles
ms-vscode.vscode-js-profile-table
1.0.8

System Info

Item
Value

CPUs
Apple M2 Pro (12 x 24)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
2, 2, 3

Memory (System)
16.00GB (0.12GB free)

Process Argv
. --crash-reporter-id 1571364c-c8c2-454c-9aea-30d78628cd73

Screen Reader
no

VM
0%

A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
vsaa593cf:30376535
py29gd2263:31024238
c4g48928:30535728
a9j8j154:30646983
962ge761:30841072
pythongtdpath:30726887
welcomedialog:30812478
pythonidxpt:30768918
pythonnoceb:30776497
asynctok:30898717
dsvsc013:30777762
dsvsc014:30777825
dsvsc015:30821418
pythontestfixt:30866404
pythonregdiag2:30926734
pythonmypyd1:30859725
pythoncet0:30859736
2e7ec940:31000449
pythontbext0:30879054
accentitlementst:30870582
dsvsc016:30879898
dsvsc017:30880771
dsvsc018:30880772
cppperfnew:30980852
pythonait:30973460
jchc7451:31067543
showvideot:31016890
chatpanelt:31014475
0ee40948:31013168
a69g1124:31018687
dvdeprecation:31040973
pythonprt:31036556
dwnewjupyter:31046869
impr_priority:31057980
26j00206:31048877
nativerepl2:31063928

Turn off auto activation. To do this, disable these two settings, ideally only for the project you are working on:

Exit VS Code.
Open Anaconda Prompt, navigate to your project.
Activate the environment that you want.
Run code . or code project.code-workspace to launch VS Code.

This will give you an activated environment, and the extension will not have to try and activate every time a terminal is created.
Also, look for the inherit env setting and see if toggling it helps.

I disabled both settings and tried both variations of "inherit env". Whenever I run VS Code from an activated environment, only the base environment is activated in the integrated terminals. So, unfortunately, this did not work.

BTW, this is certainly related to #22879 since before that, it worked just fine.

So, I guess the solution would be to execute steps 2 and 3 (see above) no matter when you're running from an activated environment or not. If I manually apply those two steps after creating a terminal, that terminal is fixed.


Questions about SEP102: Adjustment of Used Gas

gas_used = gas_limit - gas_remained
if gas_used * 4 < gas_limit then /*the estimated gas consumption is more than four times of real gas consumption*/
    gas_used = gas_limit /* no gas fee will be returned*/
else if gas_used * 2 < gas_limit then /*the estimated gas consumption is more than two times of real gas consumption*/
    gas_used = (gas_used + gas_limit) / 2
else then /*the estimated gas consumption is no larger than two times of real gas consumption*/
    /* return the gas fee just as ethereum*/
endif
returned_gas_fee = (gas_limit - gas_used) * gas_price

This penalty can be bypassed with a small piece of code, which doesn't really make much sense. And hope to improve the accuracy of the estimated gas.
uint gasLimit = gasleft() + 21200;
...
while (true) {
    if ((gasleft() * 100) / gasLimit < 50) break;
}

I can not fully understand your pseudo code. Can you elaborate on it?
Can you show us a demo whose gas is badly estimated? We need test cases.

https://www.smartscan.cash/block/2989379
I think this is more typical.

We had to add the following code to ensure that gas was sufficient.
const gasLimit = await contract.estimateGas.func();
const tx = await contract.func({ gasLimit: gasLimit.toNumber() * 1.2 });

I think smartBCH is not able to correctly estimate the gas of this function.
// SPDX-License-Identifier: UNLICENSED

pragma solidity >=0.8.0;

contract testGas {
    function ignorePenalties() external   {
        uint gasLimit = gasleft() + 21200;
        while (true) {
            if ((gasleft() * 100) / gasLimit < 50) break;
        }
    }
}

Some txs sent to your email, it is sent by protonmail. Maybe in the trash mail.

Got the mail. We'll study it.

https://www.smartscan.cash/block/3183469    Is this block normal?

This low-level bug, made me execute 61 reverted trades today, until I manually raised the gaslimit. What a bad day!

Please make your question more specific. Why is this block abnormal?
SEP102 is not meant to make arbitrage easy and gas-efficient. We do not view it as a bug.

Is it normal for too many transactions to fail because of insufficient gas? Now, let's say the gas estimate is 100, if you set it to 100, it may fail due to out of gas, if you set it to 120, it will execute successfully, and only consume 80.

Well, if normal DApp's users have such a problem, we'd better figure out a way to optimize it. If only arbitrage transactions suffer from this, then it is not the first priority to optimize it. For arbitrage, fairness is more important than gas efficiency. If you have such a problem, other arbitragers have it too. So it is fair.

I see, the goal of SmartBCH is that the normal dApp user can execute one or two simple transactions without problems. If some complex transactions go wrong, it doesn't really matter.
SmartBCH does not promise that complex contracts will be executed correctly, please write some simple contracts.
If everyone has a problem, it's not a problem, it's fair. That's a good point. Stay hungry, stay foolish, and everyone will feel fair.
There really shouldn't be any quality requirements for personal projects. Anyway, good job!

Now complex contracts are executing correctly on smartBCH. They work just as the specifications, including SEP102. Sorry, we cannot take your words as the specification.
Feel free to reopen this issue when you find some real bugs. And hope your arbitrage goes smoothly in the future.

Thanks, classifying one transaction as a normal dApp transaction or an arbitrage transaction to determine if there is a bug is the most interesting thing I learned about EVM.

We had to add the following code to ensure that gas was sufficient.
const gasLimit = await contract.estimateGas.func();
const tx = await contract.func({ gasLimit: gasLimit.toNumber() * 1.2 });

Thank you for this information. I was having the same problem.


Analyzing compatibility failed

Hi, I'v ran the console version on a WPF application (debug build, anycpu where 64bit preferred, .NET framework 4.7.2) and I'm getting the following error:
Detecting assembly references                      [Done]
Analyzing compatibility                            [Failed]

There was an unknown error attempting to reach the service.  Please try again.

Any idea what that is supposed to mean and what can have caused it? The target folder contains third party libraries which are obfuscated if that matters.

This may be related to #697 fixed by #699

Actually, that's probably not the case. Are you behind a proxy?

Hi,
no proxy / no VPN.

Gave it another try after a great while, this time I've used the visual studio addon (latest) and it worked out of the box. I'm closing this issue then, either its fixed already or it was a problem with standalone version.


toString not located in Qt5Core.dll

I build on win7x64
cmake version 3.9.0-rc6
Qt version: 5.10.1
build is fine, but when launch the *.exe it will popup error window message like:
?toString@QJsonValue@@QEBA?AVQString@@XZ could not be located in the Qt5Core.dll

It looks like you have another distribution of Qt on your system and it gets dynamically linked to your executable.
Check other software using Qt.
See also https://forum.qt.io/topic/66447/qfiledialog-problem-entry-point-not-found/3
Regards,
Dmitry

Any news?


Can I avoid connecting to a NATS cluster?

Describe the bug
I'm following the LDAP example using docker-compose. I'm currently stuck because OCIS tries to connect to a NATS cluster. I have many services in my Docker setup but none is a NATS cluster, nor I see why I should have one since it's a simple homelab setup. Apparently, the LDAP example doesn't mention NATS either.
Steps to reproduce

Have this compose.yaml file:version: "3.7"

services:
  ocis:
    image: owncloud/ocis:latest
    restart: unless-stopped
    networks:
      - authentication
    entrypoint: /bin/sh
    command: [ "-c", "ocis init --insecure yes || true; exec ocis server" ]
    environment:
      OCIS_JWT_SECRET: "${MY_JWT_SECRET}"
      OCIS_LDAP_BIND_DN: "cn=admin,dc=${MY_3LD},dc=${MY_2LD},dc=${MY_1LD}"
      OCIS_LDAP_BIND_PASSWORD: "${MY_LLDAP_ADMIN_PASSWORD}"
      OCIS_LDAP_URI: "ldap://lldap:3890"
      OCIS_RUN_SERVICES: "web,webdav"
      MICRO_REGISTRY: "memory"
      OCIS_CACHE_STORE: "memory"
      OCIS_PERSISTENT_STORE: "memory"
      PROXY_ENABLE_BASIC_AUTH: "true"
      OCIS_URL: "https://${MY_DOMAIN}/ocis"

Note that the lldap service is defined in another file and works correctly.
Those variables set to "memory" are an attempt to clear all default references to nats-js-kv.

Expected behavior
OCIS starts.
Actual behavior
OCIS exits with {"level":"error","error":"error connecting to nats cluster ocis-cluster: error connecting to nats at <IP_ADDRESS>:9233 with tls enabled (false): nats: no servers available for connection","time":"2024-08-17T13:04:02Z","caller":"github.com/cenkalti/backoff@v2.2.1+incompatible/retry.go:24","message":"can't connect to nats (jetstream) server, retrying in 1m9.42635782s"}.

Oh, I think I found the solution. It seems "nats" is mandatory, at least for "web" and/or "webdav". This variable makes it work: OCIS_RUN_SERVICES: "nats,web,webdav"

Note that the above applies to the "gateway" service too. I'm closing the issue since it's solved, but I think a dependency chain is required among services.

This error message is a false positive. During the startup of the service mesh, the nats service is not ready from the beginning while a number of services are trying to connect. That requests are retried, no issue.


Expand filtered nodes

Can you expand the filtered nodes if the filtered nodes are children somewhere in the tree, and the parent nodes current state is collapsed?
As it is now only the parent nodes are shown (even if the desired filtered nodes are some level below the shown parent).

There's nothing build in to do this but you can use ivhTreeviewMgr to expand to a given node or even expand the entire tree when there's a filter if you like.
I agree it would be a nice feature and we'd be open to discussing how it might be implemented if you have any ideas or you're interested in implementing it.
Duplicate of #23.

@jtrussell question on how to achieve this behavior:  would the right way be to add an ng-change handler to the search text input field and then query the tree in one's controller doing this manually?  apologies for posting to a closed issue.

Hey @icfantv would you mind posting your question on stack overflow? You can tag me on the question or shoot me a message to answer there. We're just trying to move usage and "how do I" type questions there. Thanks!

@jtrussell, absolutely.  I'm on the UIBS OSS project and completely understand.  Cheers.

I'm unable to tag you - or at least SO is indicating that it is not tagging you.  Here is the link.


#361 FormCheckbox review functionality and fix styles and functionality

@mohammadsalehkarimi @mhmdtaherian Stories are not in this repository.


Examples of regression?

I was wondering if anyone had used snnTorch for regression, and perhaps how you set your networks up. Just looking for simple, general examples! MSELoss would likely be the type of loss used as I see it.

I would second this question, as it is not obvious for me how to decode spikes to the real line. If you decode rate based, then the prediction for large numbers would take more spikes (=more expensive?). A simple example would be great!

Have you got any example of Linear Regression using snnTorch? I'm trying to test simple f(x) = x linear regression problem using ssnTorch, following the tutorial 5, but could not improve the training loss. I'm using MSELoss as a loss function and Stochastic gradient descent SGD as an optimizer.

I actually used MSELoss and Adam in one of my projects albeit I'm sure it could still use some work (see my repo on event-based velocity prediction). I've also been trying to do some literature search of what appropriate loss and optimization functions are for SNNs in general.

I also used Adam but could not get model improved. I think my code has some problem. I have posted about in on discussion section if you or someone could help me to find where I made mistake.
https://github.com/jeshraghian/snntorch/discussions/122

The way I see it, there are a few ways to perform regression.

Set the target of the membrane potential of a spiking neuron to reach the desired value
Set the target of the total spike count at the end of the simulation to reach the desired value
Set the target of the spike time to reach the desired value

The most effective way to implement '1' would be to ensure the output layer has the reset mechanism disabled (e.g., snn.Leaky(beta=beta, reset="none") or by setting the threshold to an arbitrarily large value. Then you would need to decide at what point in time you will be measuring the output. E.g., are you only concerned with the membrane potential at the final time step, or for all time steps? This will be quite task-dependent.
The approach for '2' is quite straightforward, as you would simply sum together all of the spikes and try and set it to be the desired target value. The limiting factors here are i) the quantised nature of spike counts (i.e., it can only take on discrete values / natural numbers, and ii) the maximum permissible spike count is the total number of time steps. The second issue can be lessened by using multiple neurone to emit multiple spikes.
The approach for '3' is lesser explored in the context of a PyTorch backend, and in general, I find it is far less stable than rate-based loss functions. But I've put together a loss function with a few examples: mse_temporal_loss. I found that I often had to drop the threshold significantly to start seeing any action, especially as my networks became deeper.
https://snntorch.readthedocs.io/en/latest/snntorch.functional.html#snntorch.functional.loss.mse_temporal_loss
Let me know what types of problems you're trying to tackle, and I can account for it in future tutorials.

Thank you very much for the extensive answer! To begin with, a simple affine transformation of a temporal (linear) sequence would be helpful. That is,
x = [0, 1, 2, ..., t] / t (that is, linspace normalized between 0 and 1)
y = ax +b for some real numbers a,b (possibly chosen randomly for every sample)
As I already wrote earlier, I experimented with unrolled spiking lstm in combination with classical dense layers as a wrapper (also using different libraries), where I was able to overfit on small values of t, but at no extend to a precision like classical lstms.
So basically, I am stuck at trying to overfit on a single sample with, say, 25 - 100 timesteps for fixed values of a and b.

This should definitely be possible! Are you able to apply MSELoss() to the output membrane potential?
I have a notebook in another repo where I train the membrane potential to linearly increase over time given random inputs. I expect learning a linear mapping is quite easier. Check out the ipynb file in this repo for inspo:
https://github.com/jeshraghian/snn-tha

Thank you for the feedback! I just used the normal MSE in the optimizer when I had a standard 'Linear' layer as output. I'll check your file and report back!

It seems to work in principal (your approach 1 using membrane potentials), but I have some (explainable) artifact in my results. I try to overfit on a single vector [TIME=10, BATCH=1, FEATURE=1], which is a simple linspace (True):
True            Prediction
[('4.9626e-01', '0.0000e+00'),
 ('5.2647e-01', '0.0000e+00'),
 ('5.5669e-01', '0.0000e+00'),
 ('5.8691e-01', '0.0000e+00'),
 ('6.1713e-01', '6.1499e-01'),
 ('6.4735e-01', '6.4795e-01'),
 ('6.7757e-01', '6.7761e-01'),
 ('7.0779e-01', '7.0775e-01'),
 ('7.3800e-01', '7.3803e-01'),
 ('7.6822e-01', '7.6820e-01')]

I use the following network:
real input -> leaky integrator with linear weights -> leaky integrate and fire with linear weights -> leaky integrator with linear weights -> real output.
Now, how to get the "correct" results for the prediction also for the first iterations? I think in the beginning the membrane is not saturated enough ....

When you say iterations, do you mean time steps? I think it might be
challenging to achieve that as the neuron needs a bit of time to get
excited / hit steady state.
Could you share the code for your network architecture?
On Mon, 8 Aug 2022, 7:49 pm Alexander Henkes, @.***>
wrote:

It seems to work in principal (your approach 1 using membrane potentials),
but I have some (explainable) artifact in my results. I try to overfit on a
single vector [TIME=10, BATCH=1, FEATURE=1], which is a simple linspace
(True):
True            Prediction
[('4.9626e-01', '0.0000e+00'),
('5.2647e-01', '0.0000e+00'),
('5.5669e-01', '0.0000e+00'),
('5.8691e-01', '0.0000e+00'),
('6.1713e-01', '6.1499e-01'),
('6.4735e-01', '6.4795e-01'),
('6.7757e-01', '6.7761e-01'),
('7.0779e-01', '7.0775e-01'),
('7.3800e-01', '7.3803e-01'),
('7.6822e-01', '7.6820e-01')]
I use the following network:
real input -> leaky integrator with linear weights -> leaky integrate and
fire with linear weights -> leaky integrator with linear weights -> real
output.
Now, how to get the "correct" results for the prediction also for the
first iterations? I think in the beginning the membrane is not saturated
enough ....
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1208024552,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4VD2TZL7TFJJXOS24DVYDX25ANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

I used a different lib for the experiment, I will make a small script in snntorch for the whole code for everyone to check out and post it here. And yes, I mean "time steps". Maybe one could use dummy entries for the first few time steps?

Thanks for pointing out the paper! I already used 1024 as width, which seems to be quite an overkill for such a simple task, so I thought there would be more of a fundamental error :D

Well, I implemented my architecture from the other lib into snntorch and .... it worked?!? Here is my code:
"""A simple regression task using snntorch."""
import snntorch
import torch
import torch.utils.data

class Regression_dataset(torch.utils.data.Dataset):
    """Simple regression dataset."""

    def __init__(self, timesteps):
        """Linear relation between input and output"""
        lin_vec = torch.linspace(start=0.0, end=0.1, steps=timesteps)
        self.feature = lin_vec.view(timesteps, 1, 1)
        self.label = self.feature * 10

    def __len__(self):
        """Only one sample."""
        return 1

    def __getitem__(self, idx):
        """General implementation, but we only have one sample."""
        return self.feature[:, idx, :], self.label[:, idx, :]

class SNN(torch.nn.Module):
    """Simple spiking neural network in snntorch."""

    def __init__(self, timesteps, hidden):
        super().__init__()
        self.timesteps = timesteps
        self.hidden = hidden

        self.fc1 = torch.nn.Linear(in_features=1, out_features=self.hidden)
        self.lif = snntorch.Leaky(beta=0.5)
        self.fc2 = torch.nn.Linear(in_features=self.hidden, out_features=1)
        self.li = snntorch.Leaky(beta=0.5, reset_mechanism="none")

    def forward(self, x):
        """Forward pass for 10 time steps."""
        mem1 = self.lif.init_leaky()
        mem2 = self.li.init_leaky()

        cur3_rec = []
        mem2_rec = []

        for step in range(self.timesteps):
            cur1 = self.fc1(x[step, :, :])
            spk1, mem1 = self.lif(cur1, mem1)
            cur2 = self.fc2(spk1)
            cur3, mem2 = self.li(cur2, mem2)
            cur3_rec.append(cur3)
            mem2_rec.append(mem2)

        return torch.stack(cur3_rec, dim=0), torch.stack(mem2_rec, dim=0)

def main():
    """Training loop and prediction."""
    DEVICE = "cuda"
    TIMESTEPS = 11
    ITER = 2000
    HIDDEN = 1024

    dataloader = torch.utils.data.DataLoader(
        dataset=Regression_dataset(timesteps=TIMESTEPS)
    )

    model = SNN(timesteps=TIMESTEPS, hidden=HIDDEN).to(DEVICE)
    model.train()

    optimizer = torch.optim.Adam(params=model.parameters(), lr=3e-4)
    loss_function = torch.nn.MSELoss()

    feature = None
    label = None
    loss_val = None
    for i in range(ITER):
        train_batch = iter(dataloader)

        for feature, label in train_batch:
            feature = torch.swapaxes(input=feature, axis0=0, axis1=1)
            label = torch.swapaxes(input=label, axis0=0, axis1=1)
            feature = feature.to(DEVICE)
            label = label.to(DEVICE)

            cur, mem = model(feature)

            loss_val = loss_function(mem, label)
            optimizer.zero_grad()
            loss_val.backward()
            optimizer.step()

        print(f"Iter: {i}, Loss: {loss_val.detach().cpu().numpy()}")

    with torch.no_grad():
        model.eval()
        _, prediction = model(feature)

    label = torch.squeeze(label).cpu().numpy().tolist()
    prediction = torch.squeeze(prediction).cpu().numpy().tolist()
    result = list(zip(label, prediction))
    for i in result:
        print(i)

    return None

if __name__ == "__main__":
    main()

This are of course great news, but now I have to figure out, what exactly happened and why the other code is not working. By the way, for large TIMESTEPS I am not able to overfit, I think due to vanishing gradients. I will try your LSTM implementation on that.
Best

Nice! Keep us updated on if you find out why. I've run into some brickwalls
trying to explain the difference in performance between packages in the
past. Hope you have more luck.
On Mon, 8 Aug 2022, 11:50 pm Alexander Henkes, @.***>
wrote:

Well, I implemented my architecture from the other lib into snntorch and
.... it worked?!? Here is my code:
"""A simple regression task using snntorch."""
import snntorch
import torch
import torch.utils.data
class Regression_dataset(torch.utils.data.Dataset):
"""Simple regression dataset."""
def __init__(self, timesteps):
    """Linear relation between input and output"""
    lin_vec = torch.linspace(start=0.0, end=0.1, steps=timesteps)
    self.feature = lin_vec.view(timesteps, 1, 1)
    self.label = self.feature * 10

def __len__(self):
    """Only one sample."""
    return 1

def __getitem__(self, idx):
    """General implementation, but we only have one sample."""
    return self.feature[:, idx, :], self.label[:, idx, :]

class SNN(torch.nn.Module):
"""Simple spiking neural network in snntorch."""
def __init__(self, timesteps, hidden):
    super().__init__()
    self.timesteps = timesteps
    self.hidden = hidden

    self.fc1 = torch.nn.Linear(in_features=1, out_features=self.hidden)
    self.lif = snntorch.Leaky(beta=0.5)
    self.fc2 = torch.nn.Linear(in_features=self.hidden, out_features=1)
    self.li = snntorch.Leaky(beta=0.5, reset_mechanism="none")

def forward(self, x):
    """Forward pass for 10 time steps."""
    mem1 = self.lif.init_leaky()
    mem2 = self.li.init_leaky()

    cur3_rec = []
    mem2_rec = []

    for step in range(self.timesteps):
        cur1 = self.fc1(x[step, :, :])
        spk1, mem1 = self.lif(cur1, mem1)
        cur2 = self.fc2(spk1)
        cur3, mem2 = self.li(cur2, mem2)
        cur3_rec.append(cur3)
        mem2_rec.append(mem2)

    return torch.stack(cur3_rec, dim=0), torch.stack(mem2_rec, dim=0)

def main():
"""Training loop and prediction."""
DEVICE = "cuda"
TIMESTEPS = 11
ITER = 2000
HIDDEN = 1024
dataloader = torch.utils.data.DataLoader(
    dataset=Regression_dataset(timesteps=TIMESTEPS)
)

model = SNN(timesteps=TIMESTEPS, hidden=HIDDEN).to(DEVICE)
model.train()

optimizer = torch.optim.Adam(params=model.parameters(), lr=3e-4)
loss_function = torch.nn.MSELoss()

feature = None
label = None
loss_val = None
for i in range(ITER):
    train_batch = iter(dataloader)

    for feature, label in train_batch:
        feature = torch.swapaxes(input=feature, axis0=0, axis1=1)
        label = torch.swapaxes(input=label, axis0=0, axis1=1)
        feature = feature.to(DEVICE)
        label = label.to(DEVICE)

        cur, mem = model(feature)

        loss_val = loss_function(mem, label)
        optimizer.zero_grad()
        loss_val.backward()
        optimizer.step()

    print(f"Iter: {i}, Loss: {loss_val.detach().cpu().numpy()}")

with torch.no_grad():
    model.eval()
    _, prediction = model(feature)

label = torch.squeeze(label).cpu().numpy().tolist()
prediction = torch.squeeze(prediction).cpu().numpy().tolist()
result = list(zip(label, prediction))
for i in result:
    print(i)

return None

if name == "main":
main()
This are of course great news, but now I have to figure out, what exactly
happened and why the other code is not working. By the way, for large
TIMESTEPS I am not able to overfit, I think due to vanishing gradients. I
will try your LSTM implementation on that.
Best
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1208157865,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4S6EHKQ74GDQH4C3ATVYEGCPANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

I did some experiments with regard to the LSTM. I observed the following points:

If the threshold for the LSTM is not set to trainable, the loss will converge to a bad level.
If the threshold is trainable, I can reach losses in the order of O(1e-5), which is much worse than the architecture consisting only of LIF.
reset_mechanism does nothing with respect to the achievable loss.
This is all for 10 time steps. My expectation was, like in standard LSTM which work just fine, that the spiking LSTM will be able to overfit (a) the 10 time steps and (b) even larger time steps.

Do you have an idea why this happens or how to improve the LSTM?
P.S.: Concerning the other libs, I will dig into the implementation, write down the formulae and try to come up with a reasonable interpretation.
Here is the code to reproduce, you can switch between LSTM and LIF in the first layer by setting LSTM=True/False:
"""A simple regression task using snntorch."""
import numpy
import random
import snntorch
import snntorch.surrogate
import torch
import torch.utils.data

# Seed
torch.manual_seed(0)
random.seed(0)
numpy.random.seed(0)

class Regression_dataset(torch.utils.data.Dataset):
    """Simple regression dataset."""

    def __init__(self, timesteps):
        """Linear relation between input and output"""
        lin_vec = torch.linspace(start=0.0, end=1.0, steps=timesteps)
        self.feature = lin_vec.view(timesteps, 1, 1)
        self.label = self.feature * 1

    def __len__(self):
        """Only one sample."""
        return 1

    def __getitem__(self, idx):
        """General implementation, but we only have one sample."""
        return self.feature[:, idx, :], self.label[:, idx, :]

class SNN(torch.nn.Module):
    """Simple spiking neural network in snntorch."""

    def __init__(self, timesteps, hidden, lstm=False):
        super().__init__()
        self.timesteps = timesteps
        self.hidden = hidden
        self.lstm = lstm

        spike_grad = snntorch.surrogate.atan()

        if not self.lstm:
            beta_in = torch.rand(self.hidden)
            thr_in = torch.rand(self.hidden)

            self.fc1 = torch.nn.Linear(in_features=1, out_features=self.hidden)
            self.lif = snntorch.Leaky(
                beta=beta_in,
                threshold=thr_in,
                learn_beta=True,
                learn_threshold=True,
                spike_grad=spike_grad,
                reset_mechanism="subtract",
            )

        elif self.lstm:
            thr_lstm = torch.rand(self.hidden)

            self.slstm = snntorch.SLSTM(
                input_size=1,
                hidden_size=self.hidden,
                spike_grad=spike_grad,
                learn_threshold=True,
                threshold=thr_lstm,
                reset_mechanism="none",
            )

        else:
            raise SystemExit()

        beta_out = torch.rand(1)
        thr_out = torch.rand(1)

        self.fc2 = torch.nn.Linear(in_features=self.hidden, out_features=1)
        self.li = snntorch.Leaky(
            beta=beta_out,
            threshold=thr_out,
            learn_beta=True,
            learn_threshold=True,
            spike_grad=spike_grad,
            reset_mechanism="none",
        )

    def forward(self, x):
        """Forward pass for several time steps."""
        syn_in = None

        if not self.lstm:
            mem_in = self.lif.init_leaky()

        elif self.lstm:
            syn_in, mem_in = self.slstm.init_slstm()

        else:
            raise SystemExit()

        mem_out = self.li.init_leaky()

        cur_out_rec = []
        mem_out_rec = []

        for step in range(self.timesteps):
            x_timestep = x[step, :, :]

            if not self.lstm:
                cur_in = self.fc1(x_timestep)
                spk_in, mem_in = self.lif(cur_in, mem_in)

            elif self.lstm:
                spk_in, syn_in, mem_in = self.slstm(x_timestep, syn_in, mem_in)

            else:
                raise SystemExit()

            cur_out = self.fc2(spk_in)
            cur_out, mem_out = self.li(cur_out, mem_out)
            cur_out_rec.append(cur_out)
            mem_out_rec.append(mem_out)

        return torch.stack(cur_out_rec, dim=0), torch.stack(mem_out_rec, dim=0)

def main():
    """Training loop and prediction."""
    DEVICE = (
        torch.device("cuda")
        if torch.cuda.is_available()
        else torch.device("cpu")
    )
    TIMESTEPS = 10
    ITER = 2000
    HIDDEN = 1024
    LSTM = True

    dataloader = torch.utils.data.DataLoader(
        dataset=Regression_dataset(timesteps=TIMESTEPS)
    )

    model = SNN(timesteps=TIMESTEPS, hidden=HIDDEN, lstm=LSTM).to(DEVICE)
    model.train()

    optimizer = torch.optim.Adam(params=model.parameters(), lr=1e-3)
    loss_function = torch.nn.MSELoss()

    feature = None
    label = None
    loss_val = None
    for i in range(ITER):
        train_batch = iter(dataloader)

        for feature, label in train_batch:
            feature = torch.swapaxes(input=feature, axis0=0, axis1=1)
            label = torch.swapaxes(input=label, axis0=0, axis1=1)
            feature = feature.to(DEVICE)
            label = label.to(DEVICE)

            cur, mem = model(feature)

            loss_val = loss_function(mem, label)
            optimizer.zero_grad()
            loss_val.backward()
            optimizer.step()

        print(f"Iter: {i}, Loss: {loss_val.detach().cpu().numpy()}")

    with torch.no_grad():
        model.eval()
        _, prediction = model(feature)

    label = torch.squeeze(label).cpu().numpy().tolist()
    prediction = torch.squeeze(prediction).cpu().numpy().tolist()
    result = list(zip(label, prediction))
    for i in result:
        print(i)

    return None

if __name__ == "__main__":
    main()

Interesting. I feel LSTMs should be better than LIFs too.
My only guess at the moment is that perhaps the firing threshold of the
LSTM is too high.
Perhaps there is insufficient spiking activity propagating through the
network.
Lowering the threshold, rather than setting it randomly, might help with
this.
In my experience, the LSTM modules performed better than LIF on more
complex, time-varying tasks (e.g., seizure detection/prediction on EEG
data).
But I had to drop the threshold quite low to get there (e.g., 0.1, 0.01...)
On Tue, Aug 9, 2022 at 3:58 PM Alexander Henkes @.***>
wrote:

I did some experiments with regard to the LSTM. I observed the following
points:

If the threshold for the LSTM is not set to trainable, the loss
will converge to a bad level.
If the threshold is trainable, I can reach losses in the order of
O(1e-5), which is much worse than the architecture consisting only of LIF.
reset_mechanism does nothing with respect to the achievable loss.
This is all for 10 time steps. My expectation was, like in standard
LSTM which work just fine, that the spiking LSTM will be able to overfit
(a) the 10 time steps and (b) even larger time steps.

Do you have an idea why this happens or how to improve the LSTM?
P.S.: Concerning the other libs, I will dig into the implementation, write
down the formulae and try to come up with a reasonable interpretation.
Here is the code to reproduce, you can switch between LSTM and LIF in the
first layer by setting LSTM=True/False:
"""A simple regression task using snntorch."""
import numpy
import random
import snntorch
import snntorch.surrogate
import torch
import torch.utils.data
Seed
torch.manual_seed(0)
random.seed(0)
numpy.random.seed(0)
class Regression_dataset(torch.utils.data.Dataset):
"""Simple regression dataset."""
def __init__(self, timesteps):
    """Linear relation between input and output"""
    lin_vec = torch.linspace(start=0.0, end=1.0, steps=timesteps)
    self.feature = lin_vec.view(timesteps, 1, 1)
    self.label = self.feature * 1

def __len__(self):
    """Only one sample."""
    return 1

def __getitem__(self, idx):
    """General implementation, but we only have one sample."""
    return self.feature[:, idx, :], self.label[:, idx, :]

class SNN(torch.nn.Module):
"""Simple spiking neural network in snntorch."""
def __init__(self, timesteps, hidden, lstm=False):
    super().__init__()
    self.timesteps = timesteps
    self.hidden = hidden
    self.lstm = lstm

    spike_grad = snntorch.surrogate.atan()

    if not self.lstm:
        beta_in = torch.rand(self.hidden)
        thr_in = torch.rand(self.hidden)

        self.fc1 = torch.nn.Linear(in_features=1, out_features=self.hidden)
        self.lif = snntorch.Leaky(
            beta=beta_in,
            threshold=thr_in,
            learn_beta=True,
            learn_threshold=True,
            spike_grad=spike_grad,
            reset_mechanism="subtract",
        )

    elif self.lstm:
        thr_lstm = torch.rand(self.hidden)

        self.slstm = snntorch.SLSTM(
            input_size=1,
            hidden_size=self.hidden,
            spike_grad=spike_grad,
            learn_threshold=True,
            threshold=thr_lstm,
            reset_mechanism="none",
        )

    else:
        raise SystemExit()

    beta_out = torch.rand(1)
    thr_out = torch.rand(1)

    self.fc2 = torch.nn.Linear(in_features=self.hidden, out_features=1)
    self.li = snntorch.Leaky(
        beta=beta_out,
        threshold=thr_out,
        learn_beta=True,
        learn_threshold=True,
        spike_grad=spike_grad,
        reset_mechanism="none",
    )

def forward(self, x):
    """Forward pass for several time steps."""
    syn_in = None

    if not self.lstm:
        mem_in = self.lif.init_leaky()

    elif self.lstm:
        syn_in, mem_in = self.slstm.init_slstm()

    else:
        raise SystemExit()

    mem_out = self.li.init_leaky()

    cur_out_rec = []
    mem_out_rec = []

    for step in range(self.timesteps):
        x_timestep = x[step, :, :]

        if not self.lstm:
            cur_in = self.fc1(x_timestep)
            spk_in, mem_in = self.lif(cur_in, mem_in)

        elif self.lstm:
            spk_in, syn_in, mem_in = self.slstm(x_timestep, syn_in, mem_in)

        else:
            raise SystemExit()

        cur_out = self.fc2(spk_in)
        cur_out, mem_out = self.li(cur_out, mem_out)
        cur_out_rec.append(cur_out)
        mem_out_rec.append(mem_out)

    return torch.stack(cur_out_rec, dim=0), torch.stack(mem_out_rec, dim=0)

def main():
"""Training loop and prediction."""
DEVICE = (
torch.device("cuda")
if torch.cuda.is_available()
else torch.device("cpu")
)
TIMESTEPS = 10
ITER = 2000
HIDDEN = 1024
LSTM = True
dataloader = torch.utils.data.DataLoader(
    dataset=Regression_dataset(timesteps=TIMESTEPS)
)

model = SNN(timesteps=TIMESTEPS, hidden=HIDDEN, lstm=LSTM).to(DEVICE)
model.train()

optimizer = torch.optim.Adam(params=model.parameters(), lr=1e-3)
loss_function = torch.nn.MSELoss()

feature = None
label = None
loss_val = None
for i in range(ITER):
    train_batch = iter(dataloader)

    for feature, label in train_batch:
        feature = torch.swapaxes(input=feature, axis0=0, axis1=1)
        label = torch.swapaxes(input=label, axis0=0, axis1=1)
        feature = feature.to(DEVICE)
        label = label.to(DEVICE)

        cur, mem = model(feature)

        loss_val = loss_function(mem, label)
        optimizer.zero_grad()
        loss_val.backward()
        optimizer.step()

    print(f"Iter: {i}, Loss: {loss_val.detach().cpu().numpy()}")

with torch.no_grad():
    model.eval()
    _, prediction = model(feature)

label = torch.squeeze(label).cpu().numpy().tolist()
prediction = torch.squeeze(prediction).cpu().numpy().tolist()
result = list(zip(label, prediction))
for i in result:
    print(i)

return None

if name == "main":
main()
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1209043442,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4WMJPCBEJV4YVQN5WTVYIFRZANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

I tried different widths, thresholds, learnable/not learnable thresholds, reset mechanism and surrogate gradients. Nothing seems to work for timesteps > 10 ....

However, I am able to overfit, if I pass the membrane potential of the slstm to the LI layer instead of the spikes, which makes me wonder if the problem really lies in the LSTM or maybe in the LI layer? Does this count as cheating and going around the idea of spikes / spiking neural networks? :D

I tried different widths, thresholds, learnable/not learnable thresholds,
reset mechanism and surrogate gradients. Nothing seems to work for
timesteps > 10 ....

Have you tried plotting the output of the SLSTM layer?
This is usually a good way to help isolate the problem.
E.g., see if there is too much spiking? Not enough spiking?

Does this count as cheating and going around the idea of spikes / spiking
neural networks?

Haha in my view, there's no precise definition of what an SNN is tbh.
My personal, engineering-centric definition is one that uses 1/0
activations.
1 * W = W, where if W is stored in memory, the weight-multiply step is just
a read-out of W from memory.
0 * W = 0, where if your hardware sees a '0' activation, a smart
accelerator will just skip the memory access step.
My goal is to use SNNs to improve efficiency.
Some people have tighter definitions of SNNs; e.g., not only 1/0
activations, but also 1/0s during the backward pass.
If you go by this rule, then snntorch isn't actually using SNNs at training
time. Only inference.
But there are plenty of analog phenomena in the brain that are not
constrained to 1s and 0s...
So cheating only depends on the rules you impose on yourself / your goals!
On Tue, Aug 9, 2022 at 9:16 PM Alexander Henkes @.***>
wrote:

However, I am able to overfit, if I pass the membrane potential of the
slstm to the LI layer instead of the spikes, which makes me wonder if the
problem really lies in the LSTM or maybe in the LI layer? Does this count
as cheating and going around the idea of spikes / spiking neural networks?
:D
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1209368674,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4UNQNHUXJCU7SOUCQ3VYJK2LANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

Thank you for the explanation, the engineering point of view suites me well ;)
What do you think, do you want my simple example to be added in your tutorial section? We could also try something more difficult, like f(x) = x^2, I dont know ....We could close this "issue" and open a thread in the discussion area, maybe to prepare a PR?

Oh yeah absolutely! More than happy for you to take the lead on this
initiative, or we can co-write a tutorial together.
Go ahead and convert this to a discussion and we can brainstorm an outline.
On Wed, 10 Aug 2022, 4:41 pm Alexander Henkes, @.***>
wrote:

Thank you for the explanation, the engineering point of view suites me
well ;)
What do you think, do you want my simple example to be added in your
tutorial section? We could also try something more difficult, like f(x) =
x^2, I dont know ....We could close this "issue" and open a thread in the
discussion area, maybe to prepare a PR?
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1210352234,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4SJYSSJLE4HL5ZA6W3VYNTKZANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

Just read through the discussion. I'd love to contribute to the tutorial or join in on the discussion as it's written - I've been using snnTorch quite frequently for my work and this would be a great opportunity for me too I think!

I think another interesting tutorial that could be done is using event data (timestamp, x, y, polarity) with regression in order to predict something about what is being captured by an event camera. Perhaps this is a bit more advanced regression problem, but I did my Masters thesis recently on this and have open-sourced my dataset, or there are a ton of existing ones in Tonic that could maybe be used albeit I'm not sure they have regression applications. Interested in hearing your thoughts or other potential ideas for a more real-world example of regression with SNNs.

I am currently working on a baseline code. Unfortunately, I have a memory leak which needs to be fixed, for which I have tank experience in pytorch. If everything is working, I'll report back!

Thats awesome, lets do it. Do you have discord? We're currently
brainstorming in the tutorial-dev channel there.
I haven't used input spike times directly (without expanding them out to
tensors), but I'm definitely intrigued!
On Thu, 11 Aug 2022, 11:52 pm Shilpa Kancharla, @.***>
wrote:

I think another interesting tutorial that could be done is using event
data (timestamp, x, y, polarity) with regression in order to predict
something about what is being captured by an event camera. Perhaps this is
a bit more advanced regression problem, but I did my Masters thesis
recently on this and have open-sourced my dataset, or there are a ton of
existing ones in Tonic that could maybe be used albeit I'm not sure they
have regression applications. Interested in hearing your thoughts or other
potential ideas for a more real-world example of regression with SNNs.
—
Reply to this email directly, view it on GitHub
https://github.com/jeshraghian/snntorch/issues/98#issuecomment-1212176030,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJTFT4QMXSP57XDCMJUU7IDVYUOUPANCNFSM5SGMR4BA
.
You are receiving this because you commented.Message ID:
@.***>

Thats awesome, lets do it. Do you have discord? We're currently brainstorming in the tutorial-dev channel there. I haven't used input spike times directly (without expanding them out to tensors), but I'm definitely intrigued!
…
On Thu, 11 Aug 2022, 11:52 pm Shilpa Kancharla, @.> wrote: I think another interesting tutorial that could be done is using event data (timestamp, x, y, polarity) with regression in order to predict something about what is being captured by an event camera. Perhaps this is a bit more advanced regression problem, but I did my Masters thesis recently on this and have open-sourced my dataset, or there are a ton of existing ones in Tonic that could maybe be used albeit I'm not sure they have regression applications. Interested in hearing your thoughts or other potential ideas for a more real-world example of regression with SNNs. — Reply to this email directly, view it on GitHub <#98 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTFT4QMXSP57XDCMJUU7IDVYUOUPANCNFSM5SGMR4BA . You are receiving this because you commented.Message ID: @.>

I do have Discord ~ I'm currently away on work but I will contact you as soon as I get back, I'd love to be part of that channel.

What's the name of the discord?

Oops, teaching quarter just finished and I'm catching up on life now.
The channel name is just snnTorch & the link is in the readme "Chat" badge.
Converting this to a discussion in the meantime!


Bug fix/8541 issue fixes

Bug fix/8541  issue fixes

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 3

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 2

See the complete overview on Codacy


Add option to use externally provided {fmt} library and fix a few warnings

Added a CMake option named LUNATIC_USE_EXTERNAL_FMT (OFF by default) to allow users of the library to provide their own {fmt} library.
These changes also include setting -fbracket-depth=4096 for Clang in the test project as the new default used in the compiler included in VS2019 16.11.0 is too low.
Also cleaned up a few compilation warnings and fixed the include paths for proper CMake installation.

Thanks! The PC initialisation was a fallout changing the CPU interface heh.


updated readme typos

Readme typos
I have corrected some typos in the readme file

Should be a straight-forward merge 😄  @jamesblasco


Event listeners for javascript

Can't for the life of me get the event listeners working for Javascript. Anyone have a working code example they can share? Thanks

same issue in nativescript-vue

As well. NS Core, JS

i think the ticket can be closed now after 2 years ;)
  function pageLoaded(args) {
    //needed so that view is scrollable, even if keyboard is visible
    let page = args.object;
    try {
      const video = page.getViewById("nativeVideoPlayer");
      video.on(Video.currentTimeUpdatedEvent, (ev) => {
        if (ev.position >= 4500 && highlightText == 1) {
          highlightText = 2;
        } else if (ev.position < 4500 && highlightText != 1) {
          highlightText = 1;
        }
      });
    } catch (err) {
      console.log(err);
    }
  }


fix misuse of hidelogwindow

fix black screen if hidelogwindow=yes
https://github.com/neutrinolabs/xrdp/issues/414

Can you squash commits into one commit and include "Fixes #414." like 06ce0d7? Then #414 will be automatically closed when this PR merged.


Fixed unable to select None for character spawn priority

About the PR
Minor fix for #24586
'None' option was was added but isn't the default and cannot be selected. The UI reverts to Arrivals upon clicking save.
This PR fixes it.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog
:cl: Krunk

fix: Characters can properly select 'None' as a preference for their spawn priority.

Oop


Can I use / inside a hash definition?

Let's test this case
Shortcut

  ~

Can I navigate to "#pageA/pageB" regardless of the presence of "/" inside hash attribute?

This behavior is obvious... I mean, if the separator is / then de-facto I can't define __sub__hashesh. So, the next step to take is to throw an error if a hash's router contains the reserved letter /.
By the way... Let's customize the separator, and let's open the possibility to recognize \ as separator too.

Abandoned


Add Publication resource type

This will be split into smaller tasks in due course

Type, Title, Authors, Name of journal/conference, link (just the one), year, date range for conferences all added.
Still to do - location for conferences if we want. Vol, issues and pages aren't really necessary to identify the publication.


B1 COMPLETED

Name: minikfare
Id: bbc350348054be0f
30 DAYS COMPLETED

concordium-testnet-node-1b7b0619f86edef0a0481d2a7e2a82c80108a77c767e61fad852cc30eab77f2a.log
concordium-testnet-system-report.log

confirmed:B1
node:bbc350348054be0f


Minor speed/scrolling optimization (for Develop branch)

Basically the same pull request: https://github.com/namespace-ee/react-calendar-timeline/pull/283
but for develop branch.

hey @pencilcheck im going to close this PR. Im not sure the issues you were facing with performance but I was able to get perf gains without doing the optimizations you're proposing.  Once I look at performance again in the near future I'll look at this PR to see if I can use some of it but for now, I'm going to hold off.
Thanks anyways for your effort :)


My local RAG return garbled text

Hi,
I followed steps in README but failed at the latest step. The return from the prompt is garbled text.

Wish anyone can guide me to the right path. 🫠
But the server's search seems to be correct? 🤔

Have Done

[x] Built WASM binary
[x] Started qdrant
[x] Fed paris.txt

Can you paste the command you used to start the server?

I copy-paste the one in README
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-2-7b-chat-hf-Q5_K_M.gguf \
    --nn-preload embedding:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf \
    rag-api-server.wasm \
    --model-name Llama-2-7b-chat-hf-Q5_K_M,all-MiniLM-L6-v2-ggml-model-f16 \
    --ctx-size 4096,384 \
    --prompt-template llama-2-chat \
    --rag-prompt "Use the following pieces of context to answer the user's question.\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n" \
    --log-prompts \
    --log-stat

Can you add --model-alias default,embedding to the command and try again? Thanks!

 Please install the plugin and restart WasmEdge."
[2024-06-15 15:07:40.370] [error] wasi-logging plugin not installed. Please install the plugin and restart WasmEdge.
[2024-06-15 15:07:40.371] [error] execution failed: host function failed, Code: 0x40e
[2024-06-15 15:07:40.371] [error]     When executing function name: "_start"

but if I install wasi-logging, wasi_nn will be removed

Version
rag: 0.6.6
wasmedge: 0.14.0

You can install both plugins. Just re-run the installer. It will auto install both.
curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s -- -v 0.13.5

@chengr4 Please update --prompt-template llama-2-chat to --prompt-template llama-2-chat,embedding.

i ran:
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-2-7b-chat-hf-Q5_K_M.gguf \
    --nn-preload embedding:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf \
    rag-api-server.wasm \
    --model-name Llama-2-7b-chat-hf-Q5_K_M,all-MiniLM-L6-v2-ggml-model-f16 \
    --ctx-size 4096,384 \
    --prompt-template llama-2-chat,embedding \
    --rag-prompt "Use the following pieces of context to answer the user's question.\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n" \
    --log-prompts \
    --log-stat

However, I still got garbled

Version
rag: 0.7.1
wasmedge: 0.13.5


added highlighting on entry from direct links

This adds highlighting to the respective entry whenever you direct link to it. This is for the #75 enhancement. It adds a better visual clarity. Gold is a personal choice (was also thinking purple).
This is how it looks

Thanks so much for doing this! I made a few tweaks (changed the outline color to purple and extended it to cover the arrow border as well, removed the icon highlighting, and added a small glow effect for emphasis).


fail to run "ember server"

git clone https://github.com/MyPureCloud/ember-webrtc-devices
cd ember-webrtc-devices
npm install
bower install
npember server
-> reports errors.
Livereload server on https://localhost:49152
Serving on https://localhost:4200/
Build Canceled: Broccoli Builder ran into an error with Babel plugin. 💥
EPERM: operation not permitted, symlink 'C:\MyDvlp\Ember-Learn\WebRTC\ember-webrtc-devices\tmp\broccoli_merge_trees-output_path-e1Bcssmf.tmp' -> 'C:\MyDvlp\Ember-Learn\WebRTC\ember-webrtc-devices\tmp\broccoli_persistent_filterbabel__babel_ember_webrtc_devices-input_base_path-LX0sfbS4.tmp'
Error: EPERM: operation not permitted, symlink 'C:\MyDvlp\Ember-Learn\WebRTC\ember-webrtc-devices\tmp\broccoli_merge_trees-output_path-e1Bcssmf.tmp' -> 'C:\MyDvlp\Ember-Learn\WebRTC\ember-webrtc-devices\tmp\broccoli_persistent_filterbabel__babel_ember_webrtc_devices-input_base_path-LX0sfbS4.tmp'

infomations:
ember-cli: 3.0.0
node: 8.9.4
os: win32 x64 (windows10)

You need to set up symlinks for ember proojects to build and serve


Edit census.ipynb example: Use dask.dataframe.read_hdf vice pd.read_hdf

I referenced this issue in the dask issue tracker, created a pull request for documentation; and am wondering if we should do the same to update the datashader example.
Cross reference issue #664 for the write up showing how this will help folks trying to learn datashader and dask.  Moreover, it makes running the census example easier on computers with little RAM and enables out of core operations since the .castra file is no longer available (discussed in detail with screen shots here)
I apologize if this is the wrong forum; trying to contribute to something I found useful.
Change from:
%%time
try:
    import dask.dataframe as dd
    df = dd.from_castra('data/census.castra')
    df = df.cache(cache=dict)
except:
    df = pd.read_hdf('data/census.h5', 'census')
    df.race = df.race.astype('category')

to
%%time
try:
    import dask.dataframe as dd
    df = dd.from_castra('data/census.castra')
    df = df.cache(cache=dict)
except:
    df = dd.read_hdf('./data/census.h5', '/census',chunksize=1000000)
    dfRace = df.categorize(columns=['race'])
    dfRace.compute()

Thanks for the suggestion, and sorry that it took me a while to get around to looking at it! I think this is definitely a datashader issue rather than one for the dask site.  The castra file is probably still available in the same location (I don't think we deleted it), but we stopped having people download it by default because the file only works on Python2.
Unfortunately, the various file formats and loading options vary a lot in performance.  On my 16GB MacBook Pro, I get the following speeds for the three different suggested options:
castra using dask: 10 seconds to load, 5 seconds to aggregate
hdf5 using pandas: 67 seconds to load, 16 seconds to aggregate
hdf5 using dask: 114 seconds to load, 31 seconds to aggregate
Castra+dask is in a class all its own, so it's a big shame that it won't work cross platform.  But as you can see, I get a pretty substantial slowdown from using dask instead of pandas with HDF5, at least on this machine and with the code listed below.  So I'd be reluctant to make that the default way the data is loaded -- people will then think that's the performance of datashader, when really it's about the data itself and how it is stored and accessed.
@jcrist, maybe there is some flag I could pass to dask so that the hdf5/dask version is faster somehow?  I think I heard that the hdf5 conda package was not compiled with multithreading enabled, which presumably limits its performance.  Any other suggestions?  See also issue #129.
%%time
try:
    import dask.dataframe as dd
    df = dd.from_castra('data/census.castra')
    df = df.cache(cache=dict)
    print("Read using dask from castra")
except:
    try:
        df = dd.read_hdf('./data/census.h5', '/census',chunksize=10000000)
        df.categorize(columns=['race']).compute()
        print("Read using dask from HDF5")
    except:
        df = pd.read_hdf('data/census.h5', 'census')
        df.race = df.race.astype('category')
        print("Read using pandas from HDF5")

Makes sense to me; I was almost maxed out on RAM for the pandas example.  But, performance explanation makes sense.
By the way, your YouTube videos where you say downsampling can lead to incorrect assumptions on the data is correct.  I plotted a full data set using datashader and saw some biases (holes) I never would have seen if I were only downsampling.
This is a fantastic tool.


Adds clock_res_get to the finished list and adds SwiftWasm

Before we had no known usage of clock_res_get. It seems that SwiftWasm
at least references it, though I'm not sure which code will require it.
Anyway, this checks clock_res_get as done, and mentions why we did it
retroactively (our policy wasn't to implement things unused, so this is
a potential use case).
See https://github.com/swiftwasm/swiftwasm.github.io/blob/9d5da94dca5066ae8bc5e409db51772b2e7da6f0/polyfill/polyfill.js

oops I think I got this wrong. it looks like a copy of Emscripten's WASI polyfill...

I'll roll this back to just the formatting change and saying we're done. I don't know of a non-browser thing and we shouldn't hunt arbitrarily.


bump to 24 6

This updates the codebase to work with Mojo 24.6.0
Benchmark results with current changes:
Benchmark results
--------------------------------------------------------------------------------
name          , met (ms)    , iters   , min (ms)    , mean (ms)   , max (ms)    , duration (ms)
HeaderEncode  ,     0.002075,   571763,     0.002075,     0.002075,     0.002075,  1186.646000
HeaderParse   ,     0.013825,    87064,     0.013825,     0.013825,     0.013825,  1203.660000
RequestEncode ,     0.014535,    88400,     0.014535,     0.014535,     0.014535,  1284.910000
RequestParse  ,     0.059278,    34772,     0.059278,     0.059278,     0.059278,  2061.219000
ResponseEncode,     0.071990,    59699,     0.071990,     0.071990,     0.071990,  4297.749000
ResponseParse ,     0.241491,    10000,     0.241491,     0.241491,     0.241491,  2414.913000

What seems to work:

Running server with basic handlers, like lightbug.🔥, bench_server.mojo
Running benchmark with bench.mojo
No compilation errors

Some problems with this PR so far:

Tests with magic run test are failing due to some invalid pointer free errors:

/lightbug_http/tests/lightbug_http/test_client.mojo::test_mojo_client_redirect_external_req_302()
src/tcmalloc.cc:302] Attempt to free invalid pointer 0x14360a8d0

******************** Failure: '/lightbug_http/tests/lightbug_http/test_client.mojo::test_mojo_client_redirect_external_req_google()' ********************
Unhandled exception caught during execution
/lightbug_http/tests/lightbug_http/test_client.mojo::test_mojo_client_redirect_external_req_google()
src/tcmalloc.cc:302] Attempt to free invalid pointer 0x1357082a0
2. When running `mojo run client.mojo` getting an exception, probably related to #1
3. When running `mojo run lightbug.🔥` seeing a lot of "Failed to close connection" logs, looks like from SysConnection.close() method, looks like an issue with `external_call`s?

Also , small_time dependency does not have a 24.6 version yet, so had to comment out code related to Cookie expiration date/other datetime.now() usage. Need to make sure this is back before merging this PR

Could the file descriptor issue be related to the symbol clash described by alain on Discord?
https://discord.com/channels/1087530497313357884/1238682851717812245/1320453635251114095

I see you guys are hard at work on 24.6 updates! If there are still some issues when I'm back from vacation I'll try to contribute too!

I see you guys are hard at work on 24.6 updates! If there are still some issues when I'm back from vacation I'll try to contribute too!

thanks man! I think we'll merge this now, and for remaining fixes/improvements there are a few which we decided to extract into separate issues/PRs, will let you know once those are created


relocate location of bootstrap.min.css loading

Due to the issue that there is no option to provide several css files in theme.conf, we need to load bootstrap.min.css in theme specific css file (in this case otcdocstheme.css) to load general layout statements first to enable us to overwrite statements without using !important or having to option to overwrite !important statements of bootstrap itself.

Did a click-through test on UI level. LGTM.


Waiting on Meteor Method data, with Flow-Router SSR and Fast-Render

This is a complex issue but I'll try to describe it. With collection data, we have the following flow when we make a new request:

Simulate the subscription in React Komposer on the server.
Flow-Router SSR gets the data on the server and generates markup.
Show SSR-generated markup.
Fast-Render gets the data on the server and injects it in the document.
Subscribe instantly in React Komposer on the client.
Show app.

This means that when the user loads the app, A) the page is immediately populated with the server-generated markup and B) Minimongo is immediately populated with the fast-render generated data. So (if I'm not mistaken) Meteor.subscribe('posts').ready() in React Komposer will always return true on first load, and that's what we're not seeing any loading screens when using Flow-Router SSR.
Now if the data is instead coming from a method, this doesn't quite work as well. Let's say that you have a single value stored on the server you want to make available to your whole app before it's loaded (in my case, the server timezone).
First of all Meteor.call is not reactive, so we have to use reactive method instead.
But even so, we have the issue that there is no Fast-Render equivalent for methods or arbitrary variables. So if we want to wait for the result of this method, we can't bypass our "loading…" screen, which defeats the point of having SSR in the first place.
In other words, with a method the flow becomes:

Simulate the method call in React Komposer on the server.
Flow-Router SSR gets the method data on the server and generates markup.
Show SSR-generated markup.
Subscribe in React Komposer on the client.
Show loading. (bad!)
Subscription is ready
Show app.

I'm not sure how I can get around this, apart from not using methods at all… Any ideas?

OK, so I think the solution is to use the inject-initial package instead of methods, in order to make the data available right from the start.
The only problem is that I don't see a way to send authenticated data using this method (i.e. data that is not part of a collection, but is only available depending on the current user) but it shouldn't matter for my use case.

Can you take a look at https://github.com/kadirahq/flow-router/issues/655 ?
Looks like it is a similar problem.

So @SachaG did you find a solution please ?

I did find a workaround, see above.

So with React FlowRouter SSR initial data is already injected to the page. so I dont know why I should use something like this again ?
Did you had the same issue as described here ? https://github.com/kadirahq/flow-router/issues/655

<script type="text/inject-data">%7B%22fast-render-

FlowRouter SSR will inject two things:

the HTML markup
the initial collections data (via fast-render)

My use case was that I needed to access some other arbitrary piece of dynamic data that was not stored in a collection. So it's different from the issue you refer to, which concerns collection data.

Right. Ok thanks.. I knew 30% my issue would be different.


Added code coverage reporting

What was a problem?
Missing specific reporting on code coverage for the repo.
How this PR fixes the problem?
Adds a basic CodeCov report for the repo. Also added a few basic badges for the readme.
Check lists

[X] Test passed
[ZX] Coding style (indentation, etc)

Additional Comments (if any)
N/A

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@211d245). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #3   +/-   ##
=========================================
  Coverage          ?   97.56%
=========================================
  Files             ?        5
  Lines             ?       82
  Branches          ?        0
=========================================
  Hits              ?       80
  Misses            ?        2
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 211d245...1017f02. Read the comment docs.


OnDelete/OnUpdate constraints with many2many relationships?

Your Question
Is it possible to specify ON DELETE and ON UPDATE for the foreign keys of a many2many relationship?
I tried the following, but it didn't add the ON DELETE part when creating the table:
type User struct {
   // ...
   Albums []Album `gorm:"many2many:user_albums;constraint:OnDelete:CASCADE;"`
}

I would like a structure where whenever a User or an Album is deleted, any related user_albums entries should automatically be deleted as well.
The document you expected this should be explained
The documentation touches on this Many to many - foreign key constraints, but the example seems uncomplete or invalid.
It says "You can setup OnUpdate, OnDelete constraints with tag constraint, [...] for example:".
And is then followed by a code example that doesn't use the constraint tag at all. Am I missing something?
Expected answer
I expected the example in the documentation to clearly show how the constraint tag is used for many2many relations.

https://gorm.io/docs/many_to_many.html#Customize-JoinTable

@jinzhu The possibility to get the same result another way doesn't make the documentation error disappear. This should be fixed rather than closing the issue.
I was also confused by this documentation. In my case, I did get it working through trial and error, using the constraint:OnDelete:CASCADE tag like this:
type Accessory struct {
	AccessoryProperties
	Id                   string         `gorm:"primaryKey;size:256"`
	CreatedAt      time.Time
	UpdatedAt     time.Time
	Branches       []Branch    `gorm:"many2many:accessory_branches"`
}

type Branch struct {
	Id                  string          `gorm:"primaryKey;size:256;constraint:OnDelete:CASCADE"`
	Accessories []Accessory `gorm:"many2many:accessory_branches;constraint:OnDelete:CASCADE"`
}

I open an issue on Stackoverflow. I can't achieve a cascade delete on One-to-Many & Belongs-To association
https://stackoverflow.com/questions/67749708/cascade-delete-in-gorm-does-not-remove-associated-tables

type User struct {    // ...    Albums []Album  }
HasMany Association Mode :
dbGormOrigin.Model(&item).Select("Albums ").Delete(item)
it works.

@jinzhu The possibility to get the same result another way doesn't make the documentation error disappear. This should be fixed rather than closing the issue.
I was also confused by this documentation. In my case, I did get it working through trial and error, using the constraint:OnDelete:CASCADE tag like this:
type Accessory struct {
    AccessoryProperties
    Id                  string   `gorm:"primaryKey;size:256"`
    Branches            []Branch `gorm:"many2many:accessory_branches"`
}

type Branch struct {
    Id          string      `gorm:"primaryKey;size:256;constraint:OnDelete:CASCADE"`
    Accessories []Accessory `gorm:"many2many:accessory_branches;constraint:OnDelete:CASCADE"`
}

This works. However, AutoMigrate doesn't add the constraint when it's added after the model has been defined and looks like the only solution is to drop the table and let AutoMigrate recreate it

@jinzhu The possibility to get the same result another way doesn't make the documentation error disappear. This should be fixed rather than closing the issue.
I was also confused by this documentation. In my case, I did get it working through trial and error, using the constraint:OnDelete:CASCADE tag like this:
type Accessory struct {
    AccessoryProperties
    Id                  string   `gorm:"primaryKey;size:256"`
    Branches            []Branch `gorm:"many2many:accessory_branches"`
}

type Branch struct {
    Id          string      `gorm:"primaryKey;size:256;constraint:OnDelete:CASCADE"`
    Accessories []Accessory `gorm:"many2many:accessory_branches;constraint:OnDelete:CASCADE"`
}

This works. However, AutoMigrate doesn't add the constraint when it's added after the model has been defined and looks like the only solution is to drop the table and let AutoMigrate recreate it

https://github.com/go-gorm/gorm/issues/4110

As of v1.24.2, adding the constraint directly into the parent struct will work properly. Here is an example:
type Accessory struct {
    AccessoryProperties
    Id                  string   `gorm:"primaryKey;size:256"`
    Branches            []Branch `gorm:"many2many:accessory_branches;constraint:OnDelete:CASCADE;"`
}

I reach the result by changing sequence of struct tags gorm:"many2many:categories_courses;constraints:OnDelete:CASCADE" ❌, gorm:"constraints:OnDelete:CASCADE;many2many:categories_courses" ✅
type Category struct {
	Model
	Name string
	Image string
	Courses []Course `gorm:"constraint:OnUpdate:CASCADE,OnDelete:CASCADE;many2many:categories_courses"`
}

type Course struct {
	Model
	Name        string
	Image       string
	Language    string
	LessonCount int
	Description string
	PriceUzs    int
	Topics      []Topic    `gorm:"constraint:OnUpdate:CASCADE,OnDelete:CASCADE"`
	Categories []Category `gorm:"constraint:OnUpdate:CASCADE,OnDelete:CASCADE;many2many:categories_courses"`
}


Update default reviewers

Part I of our retirement gift to @rpetrusha 🎁
This only sets the default reviewers on the codeowners file.

Thank you, @mairaw!

What are you doing here? 😄 Go relax!

I just happened to be checking email.


MountVolume.SetUp failed for volume "docker-sock" on AKS

Summary
Just as described in this previously filed issue I've just setup an AKS cluster (1.19.7) and tried to run the hello-world.yaml example.
Yet, the pods cannot be started:
62s         Warning   FailedMount         pod/hello-world-t4zhm                       MountVolume.SetUp failed for volume "docker-sock" : hostPath type check failed: /var/run/docker.sock is not a socket file
42s         Warning   FailedMount         pod/hello-world-t4zhm                       Unable to attach or mount volumes: unmounted volumes=[docker-sock], unattached volumes=[podmetadata docker-sock default-token-62lxs]: timed out waiting for the condition
2m56s       Warning   FailedMount         pod/hello-world-t4zhm                       Unable to attach or mount volumes: unmounted volumes=[docker-sock], unattached volumes=[default-token-62lxs podmetadata docker-sock]: timed out waiting for the condition

As mentioned above, in the original issue several other folks already confirmed (after the issue was closed) that behavior in their setup as well
Diagnostics
What Kubernetes provider are you using?

Azure Kubernetes Service AKS v1.19.7 (also tested with v1.19.6), RBAC enabled, kubenet, calico
AKS v1.19.x uses containerd (no /var/run/docker.sock available on nodes)kubectl get nodes -o wide
NAME                                  STATUS   ROLES   AGE   VERSION   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-appnodepool-51613324-vmss000000   Ready    agent   28m   v1.19.7   <none>        Ubuntu 18.04.5 LTS   5.4.0-1039-azure   containerd://1.4.3+azure
...

Notice: Azure Kubernetes Service AKS v1.18.10 works since it still uses docker as container-runtimeNAME                                  STATUS   ROLES   AGE     VERSION    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-appnodepool-51613324-vmss000000   Ready    agent   51s     v1.18.10   <none>        Ubuntu 18.04.5 LTS   5.4.0-1039-azure   docker://19.3.14
...

What version of Argo Workflows are you running?

 Give it a 👍. We prioritise the issues with the most 👍.

I think I'm also affected by this. Were you able to get argo working on AKS 1.19.7?

Well yes, I got it up and running - and finally it was my fault. Basically, with AKS 1.19.x Azure abandoned docker as container-runtime ... hence the docker.sock cannot be mounted anymore.
But when reading through the docs I found this: https://argoproj.github.io/argo-workflows/workflow-executors/
I changed from docker (as default) to k8sapi and got it working - you can do this via config-map:
apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
data:
  # https://argoproj.github.io/argo-workflows/workflow-controller-configmap.yaml

  # Specifies the container runtime interface to use (default: docker)
  # must be one of: docker, kubelet, k8sapi, pns
  # It has lower precedence than either `--container-runtime-executor` and `containerRuntimeExecutors`.
  #
  # This change becomes necessary with AKW >= 1.19.x (since Azure abandoned docker in favor of containerd)
  containerRuntimeExecutor: k8sapi

Thanks for the quick response! This also seems to have solved it for me.
So far I've only used the k8sapi executor locally with kind, but since Kubernetes deprecated dockershim maybe we can make this the new default @alexec ?

where can I find the configmap?

where can I find the configmap?

https://github.com/argoproj/argo-workflows#quickstart installs argo. The yaml contains the workflow-controller-configmap ConfigMap. I just extended this with the data provided by @schoeffm.

error: configmaps "workflow-controller-configmap" is invalid
A copy of your changes has been stored to "/tmp/kubectl-edit-2372724702.yaml"

Hello all facing this error followed steps by @schoeffm but getting this problem can anyone help?