Updates to the latest Glutin version

Alacritty is now using the latest Glutin/winit. We are temporarily on a
fork while a couple of patches get upstreamed. This branch won't be
merged until the fork is not needed.

Oh, and tests don't pass yet, but the code should work at least on Linux/X11.

Looks like the patches to winit have been upstreamed. Possibly aside from the tests, is this ready to merge?

@auscompgeek not yet. There were a few regressions with winit when it was split out from Glutin and after some recent refactoring. I expect this to get wrapped up some time next week.

Modifier keys are not being recognized in ubuntu 16.04
glutin event: DeviceEvent { device_id: DeviceId(X(DeviceId(15))), event: Key(KeyboardInput { scancode: 50, state: Pressed, virtual_keycode: Some(LShift), modifiers: ModifiersState { shift: false, ctrl: false, alt: false, logo: false } }) }

This is on an older, broken version of Glutin/winit at this point. Will follow up soon with a new attempt.


adding gmp to docker images

i tried to use the crystallang/crystal:0.18.4 image but i couldn't compile my code since gmp is missing.
this adds gmp to remove complication restrictions
thx

Thank you for this. I added the change manually because this needed a rebase.


feat: add gen helm doc in makefile

What type of PR is this?
/kind feature

What this PR does / why we need it:
use make helmdoc to gennerate chart valus in readme.md
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

/lgtm
/approve

/approve


Reduce boiler plate code for Messaging

At the current situation we must duplicate always the information about sending/receiving a message in annoatation and the real doing inside code with same message id.
This is very cumbersome:
In next example we got REQUEST_SCHEDULER_STATUS_UPDATE always mentioned twice
@IsSendingAsyncMessage(MessageID.REQUEST_SCHEDULER_STATUS_UPDATE)
	private void sendUpdateSchedulerStatusEvent() {
		DomainMessage request = DomainMessageFactory.createEmptyRequest(MessageID.REQUEST_SCHEDULER_STATUS_UPDATE);
		eventBusService.sendAsynchron(request);
	}

Variant 1
If would be nice to have something like
DomainMessageFactory.createEmptyRequest(DomainMessageUtil.getMethodMessageId());
or another solution where we do not have to duplicate the message ids.
Additional thought: One benefit of the duplication is the possiblity to mark another (upper) method for the documentation but handle on another - this would be no longer possible than.

Variant 2
Or we do a spring approach and provide magic methods for domain request handling.
e.g.
@IsSendingAsyncMessage(MessageID.REQUEST_SCHEDULER_STATUS_UPDATE)
	private DomainMessage sendUpdateSchedulerStatusEvent() {
		return DomainMessageFactory.createEmptyRequest();
	}

So there would be a need for a method interceptor which would automatically do the
eventBusService.sendAsynchron(request); for all methods annotated in such way. The result of the method would be used for sending...
(sounds nice)

New (and better way):
@IsSendingAsyncMessage(MessageID.REQUEST_SCHEDULER_STATUS_UPDATE)
private DomainMessage sendUpdateSchedulerStatusEvent() {
      return DomainMessageFactory.createRequest(MessageIdFetcher.fetchFromThisMethod())
}

MessageIdFetcher.fetchFromThisMethod()
will create stacktrace array by Thread, inspect the caller side (as done in #203)
then simply create class by name, fech method, fetch annotation, done...


Mark PTO as dead in the projects page; add matrix-ircd

Done: https://github.com/matrix-org/matrix-doc/commit/32cc1ded861e7edeb2f15098713e09baad3541d0


Firebase

Firebase migration of users/events/login

I cant actually force merge this for some reason, so you'll need to review it @ROODAY

weird, ima go fix perms for the eboard


Johnson algorithm davisp

Description of new feature, or changes

Checklist

[x] Your branch is up-to-date with the base branch
[x] You've included at least one test if this is a new feature
[x] All tests are passing

Related Issues and Discussions

People to notify

I must acknowledge that I copied code from the following pull requests:  https://github.com/Fair2Dare/algorithms-js/tree/dijkstra and https://github.com/wbira/algorithms-js/tree/bellman-ford. Since Johnson's algorithm uses both the Bellman-Ford algorithm and Dijkstra's algorithm, I decided to use other contributors' code.


enhancement - embedded gui tests

Gui Test is integrated in a special screen.
See if Cypress test can be replaced by this solution.

additional script will be downloaded if requested in the debug scren

embedded gui test will be added to another future project

analyse if possible


Error: Cannot find module 'canvas'

Cannot run "node index.js"
and it shows "Error: Cannot find module 'canvas'"
How to solve it?

Cannot run "node index.js"
and it shows "Error: Cannot find module 'canvas'"
How to solve it?

Im getting same, did you get how to solve?

Assuming you already ran npm install you could try this .
https://stackoverflow.com/questions/14771781/nodejs-cannot-find-module-build-release-canvas/22222797

I tried these but still isnt helping im getting
"Failed at the<EMAIL_ADDRESS>install script."


fix: support plain typing.Tuple type

Change Summary
We currently support plain typing.Dict, typing.List but not typing.Tuple. Now we do.
Related issue number
closes #2132
Checklist

[x] Unit tests for the changes exist
[x] Tests pass on CI and coverage remains at 100%
[ ] Documentation reflects the changes where applicable
[x] changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)

Tried it and it fixes my problem indeed. Thanks !


MSVC_2019: ScopedEvent: attempting to reference a deleted function

This is a regression when updating from v30.0 to v31.0.
Previously compiled code is breaking compilation with:
error C2280: '<someclass>::Read::ScopedEvent57::EventFinalizer::EventFinalizer(<someclass>::Read::ScopedEvent57::EventFinalizer &&)': attempting to reference a deleted function
<somepath>(57): note: see declaration of '<someclass>::Read::ScopedEvent57::EventFinalizer::EventFinalizer'
<somepath>(57): note: '<someclass>::Read::ScopedEvent57::EventFinalizer::EventFinalizer(<someclass>::Read::ScopedEvent57::EventFinalizer &&)': function was explicitly deleted

Failing code looks very simple:

Thanks for reporting. Fixed by https://android-review.git.corp.google.com/c/platform/external/perfetto/+/2355222

Nice! Unfortunately I can't see the patch from outside... Is it already synced to github?

Sorry updated the URL (we changed how URLs work so I cannot just copy paste links from the address bar :( )
The fix has not landed but should do so soon.

Thanks a lot! This was quick :-) Do you plan to release a new version with the fix?

We're overdue on the December release at this point and given people are going to be out for Christmas, we may just skip this release and do one for v32 in January.

Sounds good! I'll add an official patch to conan recipe until then.

Closed as fixed.


Feature suggestion: run Django unittests

From @jvaesteves on January 16, 2017 22:28
Django unittest extends from unittests, but you can't run as the former on this extension. For this, you need to "manage.py test", but it would be awesome if there were those same shortcuts.
Copied from original issue: DonJayamanne/pythonVSCode#648

From @guhuajun on January 19, 2017 6:51
I did something like following. I think it's supported now.

From @jvaesteves on January 19, 2017 12:13
I tried your suggestion, copying the same parameters on the settings.json file, but running the "Run Unit Test Method..." command still doesn't work. The Output Console gives me the following message:
ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
And then, when I configure this environment variable to my global_settings.py path, the message change to: ImportError: Import by filename is not supported.

@jvaesteves Will be closing this issue as @guhuajun has a solution.

@jvaesteves please confirm @guhuajun suggestion works, so I can close this issue.

From @guhuajun on January 20, 2017 0:22
https://github.com/DonJayamanne/pythonVSCode/wiki/unitest-(Standard-Unit-testing)-Framework#test-discovery
Please try to place a init.py.
Yes, missing DJANGO_SETTINGS_MODULE is annoying. I encountered this issue when writing tests for a django-cms project.

From @jvaesteves on February 2, 2017 10:46
No, it didn't.

From @james-perretta on March 5, 2017 4:35
It looks to me like the suggestion from @guhuajun gets us part of the way there for running Django test cases. Unless I'm mistaken, you can't really use unittest to run tests for any serious Django project.
When you run python manage.py test, it first does general setup (figuring out which settings module to use and calling django.setup()). After that, it hands the rest off to a test-runner class that does even more important things like creating and destroying the databases that the test cases need to operate on.
I did a bit of digging, and although I'm not particularly familiar with the implementation details of your plugin, I think this is a starter summary of what it would require to add this:

Add client modules similar to those for nose and pytest (I have almost zero knowledge of how this part works)
Add handling for testFx == 'django' in testlauncher.py. This would involve code similar to what you see here in the runtests.py snippet.
Possibly add a workspace settings option that lets the user specify the value for DJANGO_SETTINGS_MODULE. That environment variable needs to be set before calling django.setup()
Certain manage.py test options should always be passed, such as --no-input
Add support for other django-specific unit test options, such as --tag, --exclude-tag, and maybe --parallel

Thoughts?

@PlugaruT  Environment variables can be set for the workspace. Just create a file named .env in your workspace root and add the necessary variables in there.

From @PlugaruT on October 3, 2017 11:15
Because I have different settings files for each environment (production, local, test), I can't run tests because with this plugin because DJANGO_SETTINGS_MODULE is not set when trying to run the tests. I would be nice if I could set this env variable in in the workspace.json or somewhere else.

Any update here?, I have a bunch of tests file, I'm not sure where to put django.setup() if that is a real solution.

You don't need to configure DJANGO_SETTINGS_MODULE if you use the option: "--settings >filesetting>" Just saying because maybe we can make thah unittest plugin doesn't need env variable, just tell him the setting and if not, use default.

Any decision on this? This is somewhat small ask, but it will make editing better for those who use django and not pytest/nose/etc.

@yardensachs not at the moment (we're tied up with other things that impact all Python developers). If anyone wants to submit a PR to introduce a fix for this we would happily review it.

I came across this issue, not because I actually need Django support, but rather the ability to run some arbitrary test command (in my case using Flask with a test database setup/teardown, similar to Django application).
I've recently moved over from Sublime, where I use the very useful Anaconda IDE plugin to run my tests. Might be a useful pointer for anyone looking to solve this. Their config (though not documented particularly well) allows for complete extensibility, e.g:
{
  "test_command": "./test.sh",
  "test_delimeter": "::",
  "test_filepath_patterns": true,
  "test_method_delimeter": "::",
  "test_virtualenv": "~/.virtualenvs/my-venv"
}

As a workaround for anybody still looking for django support for tests, I've had some pretty good luck with using pytest as the test runner and pytest-django in place of native django test. I don't know if there are places where they won't be entiriely compatible, but so far seems to be working for the several projects I've tried.

When you guys get started on working on this issue, please add the automatic setup of the test environment and test databases like the python manage.py test would.

I thought I'd just follow up to my previous comment re: having a pre-test setup. By doing things the "pytest way" I was actually able to achieve exactly what I wanted. Just as a pointer:

By leveraging pytest fixtures with a session scoping, we were able to set-up and teardown a database as necessary
In addition, we used the following plugins to get us to a good state

pytest-env we were able to automatically set the necessary testing environment variables
pytest-flask-sqlalchemy allowed us to run tests within transactions
pytest-xdist allowed us to run tests multi-threaded

Its a shame that for 3 years no one of the Microsoft team decided to put some effort in making the VSCode way more popular than PyCharm by only making it possible to run Django tests. Which i am sure is no more than 1 week job ( at most ). Long time ago ive adapted a plugin to simply call django.setup() on the test start and tear down on test end. It took me 1 day. Cant imagine how long it would takes to you guys.

@slavpetroff Please be careful about making assumptions about how long it would take to make such a change. There's UX, testing, etc. that would have to go into this which does not make it a simple one-liner fix. But if you would like to try and come up with a design for this and a subsequent PR then you are welcome and we will review it.

I took a glance at this over the weekend, and can definitely confirm it's not a one-line change. Besides the new test runner objects, and the hundred or so places it looks like I'd have to connect the new objects into the existing code, I think the django test might have to work differently from any of the existing test runners in terms of launching (I don't really understand all the code yet, but I think the existing ones don't need to run an external python script), but also django tests can use different runners (django's unittest based one, nose, or pytest) which will definitely need some extra design and/or UX.
This looks much bigger than I would have assumed before looking.

@Cetlan unfortunately there's no specific docs on the design, but you can feel free to ask here or on Discord at https://aka.ms/python-discord-invite.

I managed to find a little more time last weekend to look into this a bit further. I see that the unittest version of the testing code runs via visualstudio_py_testlauncher.py, and routes the test information back via sockets. Does this imply that output from python -m unittest -v [...] isn't enough to display proper test results and details?
If that's the case, then I think to make this work with basic django tests we'd need a custom django test runner. It should be possible to run manage.py from the vcode-python testing framework without too many changes, and django_nose.NoseTestSuiteRunner might be able to work mostly out-of-the-box because it can produce xunit output, which the vscode-python nose runner already deals with. But for the standard django tests, we'd need either a replacement for manage.py test similar to how visualstudio_py_testlauncher.py replaces python -m unittest, or a custom runner that provides the hooks like visualstudio_py_testlauncher.py, or xunit output, etc, but can be injected into the tests with mange.py test --testrunner <..>

@Cetlan this might be useful: https://github.com/xmlrunner/unittest-xml-reporting#django-support. I had to tweak the settings to output a single xml file for compatibility with our ci/cd pipeline, but it pretty much works right out of the box.

I have tried to add
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project_name.settings')

along with

I'm also eager for a real integration to run Django tests! In the meanwhile, if anyone just wants to run Django tests without the need for parsing the results, I use this VS Code extension with great success: https://github.com/Pachwenko/VSCode-Django-Test-Runner
Just remember to set up the prefix command setting to cd into the correct directory.
These are my settings. With the --keepdb, this extension is a great enabler for my workflow as it makes it really fast to execute a single or a few tests. I never run the whole test suite locally anyway because it takes too much time.

Hope this can help someone.

I managed to get around this issue by adding import django and django.setup() at the top of all my test cases.

Any news about officially supporting Django unittest?

Maybe the new update to the Python Extension testing revamp helped?
Couldn't test that yet myself...

@djbrown I am afraid those changes haven't fixed the Django testing experience.

No news I'm afraid. We are still working through more low-level/fundamental issues in our testing code before we can tackle something like this.

@brettcannon I hope you will place this request in your backlog, as you already provide explicit support to Django in the launch.json.

Note: The StackOverflow discussion here has workarounds for Bash (macOS/Linux) and Git-Bash (Windows): https://stackoverflow.com/questions/56858808/how-do-i-debug-individual-django-tests-in-vscode .
Quoting that discussion
Niels writes:

If you're on a Mac or Linux, the following launch config should work for a single unit test executed by Django:
    {
        "name": "Python: Django Debug Single Test",
        "type": "python",
        "request": "launch",
        "program": "${workspaceFolder}/manage.py",
        "args": [
            "test",
            "`echo -n ${relativeFileDirname} | tr \/ .`.${fileBasenameNoExtension}"
        ],
        "django": true
    },

Arenwino writes:

If you're using Windows and the git bash, the launch config with tr will also work, but you'll need to quote the variable substitution and quote the double backslashes.
    {
        "name": "Python: Django Debug Single Test",
        "type": "python",
        "request": "launch",
        "program": "${workspaceFolder}/manage.py",
        "args": [
            "test",
            "`echo -n \"${relativeFileDirname}\" | tr \\\\ .`.${fileBasenameNoExtension}"
        ],
        "django": true
    },

Here is generic way to get Django tests to run with full vscode support

Configure python tests

Choose unittest
Root Directory
test*.py

Then each test case will need to look like the following:

from django.test import TestCase

class views(TestCase):

    @classmethod
    def setUpClass(cls):
        import django
        django.setup()

	def test_something(self,):
		from user.model import something
		...

Any functions you want to import have to be imported inside the test case (like shown). The setUpClass runs before the test class is setup and will setup your django project. Once it's setup you can import functions inside the test methods. If you try to import models/views at the top of your script, it will raise an exception since django isn't setup. If you have any other preinitialization that needs to run for your django project to work, run it inside setUpClass

Note: The StackOverflow discussion here has workarounds for Bash (macOS/Linux) and Git-Bash (Windows): https://stackoverflow.com/questions/56858808/how-do-i-debug-individual-django-tests-in-vscode .

This workaround has stopped working for me on macOS (just noticed today). It seems like VSCode now escapes the pipe character, but I don't remember exactly what the launcher command output looked like when it was working.
/usr/bin/env /path/to/venv/python /path/to/.vscode/extensions/ms-python.python-2022.6.2/pythonFiles/lib/python/debugpy/launcher 53076 -- /path/to/manage.py test --settings example "`echo -n path/to/example \| tr /.`.test_example"
System check identified no issues (0 silenced).
E
======================================================================
ERROR: path/to/example | tr /  (unittest.loader._FailedTest)
----------------------------------------------------------------------
<TRACEBACK OMITTED>
ImportError: Failed to import test module: path/to/example | tr /

@midouest running into the same issue as well.

This no longer works since VSCode for some reason doesn't perform the bash inline string manipulation anymore!

Here is generic way to get Django tests to run with full vscode support

Configure python tests

Choose unittest
Root Directory
test*.py

Then each test case will need to look like the following:

from django.test import TestCase

class views(TestCase):

    @classmethod
    def setUpClass(cls):
        import django
        django.setup()

    def test_something(self,):
        from user.model import something
        ...

Any functions you want to import have to be imported inside the test case (like shown). The setUpClass runs before the test class is setup and will setup your django project. Once it's setup you can import functions inside the test methods. If you try to import models/views at the top of your script, it will raise an exception since django isn't setup. If you have any other preinitialization that needs to run for your django project to work, run it inside setUpClass

This no longer works since VSCode for some reason doesn't perform the bash inline string manipulation anymore!

Note: The StackOverflow discussion here has workarounds for Bash (macOS/Linux) and Git-Bash (Windows): https://stackoverflow.com/questions/56858808/how-do-i-debug-individual-django-tests-in-vscode .

This workaround has stopped working for me on macOS (just noticed today). It seems like VSCode now escapes the pipe character, but I don't remember exactly what the launcher command output looked like when it was working.
/usr/bin/env /path/to/venv/python /path/to/.vscode/extensions/ms-python.python-2022.6.2/pythonFiles/lib/python/debugpy/launcher 53076 -- /path/to/manage.py test --settings example "`echo -n path/to/example \| tr /.`.test_example"
System check identified no issues (0 silenced).
E
======================================================================
ERROR: path/to/example | tr /  (unittest.loader._FailedTest)
----------------------------------------------------------------------
<TRACEBACK OMITTED>
ImportError: Failed to import test module: path/to/example | tr /

I documented another workaround in the comments of that SO page (https://stackoverflow.com/questions/56858808/how-do-i-debug-individual-django-tests-in-vscode/72666662#72666662).
BTW, I don't think it's the pipe that is escaped, even using something simple as `echo -n "${relativeFileDirname}"` will result in an empty value, so I think  it's the tick that is being handled differently now.

No update. We will post here when there's something to share.

Thank you.
On 18-10-2022 12:17 , Brett Cannon wrote:

No update. We will post here when there's something to share.
--
Reply to this email directly or view it on GitHub:
https://github.com/microsoft/vscode-python/issues/73#issuecomment-1282888993
You are receiving this because you commented.
Message ID: @.***>

--
Matthew

@PlugaruT Environment variables can be set for the workspace. Just create a file named .env in your workspace root and add the necessary variables in there.

Hi there, I've created the .env file and added this DJANGO_SETTINGS_MODULE="my_project.settings" it's work but not correctly. as it will find part of the test files not all of them. I've found that for the apps that their tests aren't found there is an error:
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
so is there anything else that I had to put in .env or anywhere else?

@mh-firouzjah, you also have to make sure that django.setup() is called before your tests run. That's what loads the apps into the django app registry, which django requires before using any of the models. I don't think anyone really has a good solution for calling that, which is why some people have suggested adding it to the test files, I've suggested a workaround with pytest-django, and this feature request exists in the first place.
The only other idea I've had meanwhile is to maybe hack a django.setup in through sitecustomize - but that'll force it to run all the time, which django itself might have a problem with, since it's not meant to be called more than once.
If you did already call django.setup, then I'm as stumped as you are.

I was able to get my tests partially working by calling django.setup() at the top of my setUpClass method. However, I attempt to initialize a couple models in setUpTestData() and that is pulling tables from the live database instead of the test database when run through VS Code. When I call it through manage.py it works properly. I call super().setUpClass() after django.setup(), which is what triggers the setUpTestData()` call. I wonder if there is another method I need to call to get the database properly cloned.

I spent quite a bit of time on this in the last couple of days and I think I've got an alright solution. I wanted the same Django behaviour when running tests from the VSCode test explorer (setup the testing environment / create and destroy the database). Couple of other things - I didn't want to run setup / teardown methods on a class basis(via setUpClass() or other) and I also didn't want to alter the behaviour of Django manage.py test. unittest 3.1 provides hooks for injecting code on when any test is about to run or after all tests are executed. Additionally, Django exposes utilities for setting up / tearing down the test environment and database.
In my project, I have multiple apps with separate test dirs. At the root of every test dir I included an init.py containing the same code.
from django.conf import settings

if not settings.configured:
    import unittest
    import os
    from django import setup

    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "example.settings")
    setup()

    from django.test.utils import (
        setup_test_environment,
        teardown_test_environment,
        teardown_databases,
        setup_databases,
    )

    VERBOSITY = 1
    INTERACTIVE = False

    def djangoSetUpTestRun(self):
        setup_test_environment()
        self._django_db_config = setup_databases(VERBOSITY, INTERACTIVE)

    def djangoTearDownTestRun(self):
        teardown_databases(self._django_db_config, VERBOSITY)
        teardown_test_environment()

    setattr(unittest.TestResult, "startTestRun", djangoSetUpTestRun)
    setattr(unittest.TestResult, "stopTestRun", djangoTearDownTestRun)

Unfortunately - this will still run the setup and teardown for every sub module. Wondering if leveraging load_tests protocol would allow discovery / packaging of all tests into a single unittest.TestSuite so that we could run setup and teardown only once per test package in VSCode test runner?

What's the drawback of running the setup/teardown methods on a class basis? If that is the only drawback and otherwise you have the system functional inside vscode, that sounds like a worthwhile tradeoff to me.

I have finally figured out a way to make unittests work with VSCode and Django with only one python file and without any additional configuration or setup.
VSCode test runner does test discovery every time, regardless of the amount of tests that are about be run.
We can leverage this discovery by running Django test setup during this phase. File or directory name for the setup file must be set so that the discovery finds it first. I use a directory in my project root called _unittest.
This is only tested on Ubuntu, also you must update filepaths to reflect your environment.
_unittest/__init__.py
"""
Hacks to make unittest work desirably.

VSCode + Python + Django + unittest integration is currently broken.
Django tests can be run using `python3 manage.py test` but this is not enough as VSCode unittest
runner only uses `python3 -m unittest`.

The purpose of this file is to simulate what the manage script is doing before any tests are run.
The trick is to make this load first when unittest does test discovery, hence the path `_unittest`.

This clearly cannot work when directly pointing to a test since test discovery is not used,
for example `python3 -m unittest server.component.test`.
However VSCode points to target tests differently and discovery still gets run making this work.

This all hopefully becomes obsolete if Django support is added to VSCode Python.
The issue: https://github.com/microsoft/vscode-python/issues/73
"""

import inspect
import os

import django
from django.conf import settings
from django.test.utils import get_runner

def _check_stack() -> bool:
    """
    Dig stack for the filepath of the script that imports this file.

    Return True if improting and invoking scripts are as expected.
    """
    # Skip first two calls which are this module and this function
    for frame in inspect.stack()[2:]:
        # Skip internal libraries which start with <
        if frame.filename[0] != "<":
            importing_script = frame.filename
            break
    else:
        raise RuntimeError("Importing script not found")

    # Unittest loader should be used
    if importing_script != "/usr/lib/python3.11/unittest/loader.py":
        return False

    # Django manage script uses the same loader, so check for that as well and skip if used
    first_invoking_script = inspect.stack()[-1].filename
    if first_invoking_script == "/workspace/backend/manage.py":
        return False

    return True

# This script should only be imported, so ignore direct execute
if __name__ == "__main__":
    pass

# Only allow this hack to work with unittest loader
elif _check_stack():
    # Setup Django for testing
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.testing")
    django.setup()
    settings.ALLOWED_HOSTS += ["testserver"]

    # Setup test database for testing
    get_runner(settings)(interactive=False).setup_databases()

Hi. I raised this on mastodon, and @brettcannon suggested to post here.
I'm trying work out what we're missing from Django in order for this to work. The django-admin test command wraps a DiscoverRunner (src) that's based on unittest, so the thought is it should work, and I wasn't able to see exactly what's missing.
I asked:

Is this because we need a --discover flag to the management command or something...?

…to which Brett pointed out that you're calling unittest API via Python, and not the CLI, so it becomes, roughly, what API are we missing from DiscoverRunner that would enable test discovery to work here? Presumably we need to be able to call into build_suite…🤔 — I guess my question is, what would this take from VS Code's end? What does it need to look like? If it's just exposing the right API, then surely we can get this working — or is there a structural issue we'd need to resolve first?
Any insight welcomed. Thanks.

maybe it would be possible to patch the extension so it could check if a manage.py file is present in the root folder, then use it to test else do the old way(current way)

on a new, minimal django proect (polls app from tutorial),
the call for discovering tests looks like this for me: C:\Program Files\Python39\python.exe" ~\.vscode\extensions\ms-python.python-2022.18.2\pythonFiles\testing_tools\unittest_discovery.py . test*.py
so the vscode-python plugin calls its own disovery script: unittest_discovery.py
settings.json looks like this:
{
    "python.testing.unittestArgs": [
        "-v",
        "-p",
        "test*.py"
    ],
    "python.testing.pytestEnabled": false,
    "python.testing.unittestEnabled": true
}

full output (vscode > output > python)
> "C:\Program Files\Python39\python.exe" ~\.vscode\extensions\ms-python.python-2022.18.2\pythonFiles\testing_tools\unittest_discovery.py . test*.py
сwd: .
[ERROR 2023-1-9 18:29:1.465]: Error discovering unittest tests:
 Failed to import test module: polls.tests
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\unittest\loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "C:\Program Files\Python39\lib\unittest\loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "c:\Users\myusername\projects\django-vscode\polls\tests.py", line 6, in <module>
    from .models import Question
  File "c:\Users\myusername\projects\django-vscode\polls\models.py", line 4, in <module>
    class Question(models.Model):
  File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\db\models\base.py", line 127, in __new__
    app_config = apps.get_containing_app_config(module)
  File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\apps\registry.py", line 260, in get_containing_app_config
    self.check_apps_ready()
  File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\apps\registry.py", line 137, in check_apps_ready
    settings.INSTALLED_APPS
  File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\conf\__init__.py", line 92, in __getattr__
    self._setup(name)
  File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\conf\__init__.py", line 72, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

similarly as @jvaesteves posted on January 19, 2017 12:13

ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Looping in @eleanorjboyd and @karthiknadig to look into the questions that @carltongibson is asking.

I'm not sure if vscode will use .venv/bin/python or not, but if it does, adding these lines of code at the top of ~/~/.vscode/extensions/ms-python.python-<version>/pythonFiles/testing_tools/unittest_discovery.py will set the DJANGO_SETTINGS_MODULE and then problem is solved.
DON'T TRY AT HOME
from pathlib import Path
if Path(start_dir + "/manage.py").is_file():
    with open(start_dir + "/manage.py", "r") as management_file:
        contents = management_file.readlines()
        if '"""Django\'s command-line utility for administrative tasks."""\n' in contents:
            print("django management file found!")
            for line in contents:
                if line.strip().startswith("os.environ.setdefault"):
                    eval(line.strip()) # this is the not recommended part!!!

worth to mention that adding the code above to ~/.vscode/extensions/ms-python.python-<version>/pythonFiles/unittestadapter/discovery.py would help vscode to find django tests but it is not enough as vscode still unable to run the tests and it seems vscode just uses that file to discover the tests but not to run them. also I'm not sure but I think vscode will use the unittest of the selected interpereter to run these tests so we have find a way to fix the DJANGO_SETTINGS_MODULE and django.setup() for that part.

Aside on the recent comments:
import django
django.setup()

I didn't look at the details of the workarounds here but, just from a Django perspective, calling django.setup() is going to be needed. It requires an import path to a settings module, either via DJANGO_SETTINGS_MODULE env var or a --settings CLI flag to django-admin. (The default manage.py file sets a default for DJANGO_SETTINGS_MODULE to the settings file created with the startproject command, so that's not a bad fallback value.)

@carltongibson I'm doing my best to make the changes more reliable and stable.(a new commit has been published)

Hi, we have solved this problem in our project.
It needs a bit more than running django.setup().
Our code runs code below and full vscode testing pane works okay.
import django
from django.apps import apps
from django.test.utils import setup_databases, setup_test_environment
if not apps.ready:
    django.setup() # Normal Django setup
    setup_test_environment() # This does a lot of stuff inside Django tests
    # The next one is very important: it creates test databases and changes settings.DATABASES to point to them
    # otherwise tests will run against live database.
    setup_databases(verbosity=1, interactive=False, keepdb=True)
    # keepdb probably should be a setting inside vscode.
    # Our project takes an hour to run migrations from scratch, so we need keepdb,
    # but normally no one wants to keep test databases around.

As a note if anyone is planning to do it before it is implemented in the extension.
This all is running in every <app>/tests/__init__.py.
Because if you are using something like Django Factory, which touches Django models on import, you'll get an exception just because discovery imported <app>/test/factories.py directly.
But if extension sets everything up - this won't be needed.
Also we run it only if there is "testlauncher.py" or "unittest_discovery" in sys.argv
So this workaround does not kick in under normal ./manage.py test

Hello @carltongibson, great to meet you! Thank you for your in-depth reply! I am currently thinking on the next steps and talking with the team so I will get back to you soon. Your answers are extremely helpful in narrowing down what is needed and how to create compatibility. I am excited about the shared types with unittest and how this can make the integration more seamless.
Thanks!

So.. any update on this? I tried setting up our company project with VSCode and the extension is still unable to find django test modules.

@khamaileon Nice. Good spot. I didn't even know about that other issue.

Apparently the next step is here and we need votes to get things going. If the 300+ people who liked this post could make their voices heard.

As long as the proposed procedure is taking approval, I have tested a new idea, and it worked well for me. I am awaiting feedback and information on whether it works properly for you and others, or if there are any issues.

I expected that after running the tests, there would be a change in the main project database Which it did not and the main database is clear; because in this approach, I did not use a CustomTestRunner.

NOTE: This method requires fewer changes in the vscode extension and is also easier to modify the django_settings_module for each project, especially when we have multiple modules as settings for different scenarios such as development, production, and so on.

1- Vscode Python Extension Requirement
edit the __init__.py module in ~/.vscode/extensions/ms-python.python-VERSION/pythonFiles/unittestadapter/__init__.py and add the code below:
try:
    import django
    django.setup()
except:
    pass

2- Django Project Requirement
create a .env file and inside of it declare DJANGO_SETTINGS_MODULE. an example is:
DJANGO_SETTINGS_MODULE=my_django_project_name.settings

@mh-firouzjah I tried but it didn't work out.
Here's my vscode settings:
{
  "editor.codeActionsOnSave": {
    "source.organizeImports": "explicit"
  },
  "editor.formatOnSave": true,
  "python.defaultInterpreterPath": "~/.virtualenvs/acme-api/bin/python",
  "python.envFile": "${workspaceFolder}/.env.local",
  "editor.rulers": [100],
  "python.testing.unittestArgs": ["-v", "-s", "./src", "-p", "test*.py"],
  "python.testing.pytestEnabled": false,
  "python.testing.unittestEnabled": true
}

And here's the kind of output I got:
2024-02-06 10:36:32.276 [info] Telemetry level is off
2024-02-06 10:36:32.276 [info] Experiments are disabled, only manually opted experiments are active.
2024-02-06 10:36:32.276 [info] Default formatter is set to ms-python.black-formatter for workspace /home/me/workspace/acme/mars-api
2024-02-06 10:36:32.276 [info] Test server listening.
2024-02-06 10:36:32.276 [info] VS Code was launched from an activated environment: 'mars-api', selecting it as the interpreter for workspace.
2024-02-06 10:36:32.276 [info] Python interpreter path: ~/.virtualenvs/mars-api/bin/python
2024-02-06 10:36:34.315 [info] Starting Pylance language server.
2024-02-06 10:36:43.381 [info] Discover tests for workspace name: mars-api - uri: /home/me/workspace/acme/mars-api
2024-02-06 10:36:43.405 [info] > . ~/.virtualenvs/mars-api/bin/activate && echo '<some_uuid>' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 10:36:43.405 [info] shell: bash
2024-02-06 10:36:43.457 [info] > ~/.virtualenvs/mars-api/bin/python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/testing_tools/unittest_discovery.py ./src test*.py
2024-02-06 10:36:43.457 [info] cwd: .
2024-02-06 10:36:44.068 [error] Error discovering unittest tests:
 Failed to import test module: account.tests
Traceback (most recent call last):
  File "/usr/lib/python3.12/unittest/loader.py", line 394, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/unittest/loader.py", line 337, in _get_module_from_name
    __import__(name)
  File "/home/me/workspace/acme/mars-api/src/account/tests.py", line 15, in <module>
    from account.models import Session, User
  File "/home/me/workspace/acme/mars-api/src/account/models.py", line 3, in <module>
    from django.contrib.auth.models import AbstractUser
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/contrib/auth/base_user.py", line 58, in <module>
    class AbstractBaseUser(models.Model):
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/db/models/base.py", line 129, in __new__
    app_config = apps.get_containing_app_config(module)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
    self.check_apps_ready()
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/apps/registry.py", line 138, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

DJANGO_SETTINGS_MODULE=project.settings is read correctly, otherwise I'm told it's necessary.
The problem may be that my code is in a src folder and not in the root. 🤷♂

hi @khamaileon,
yes, DJANGO_SETTINGS_MODULE is read. but for the error you got it seems that django.setup() had not been called at the beginning of the process.
being in a subdirectory could be the case but if you also have error when using manage.py check/runserver/...
otherwise I don't think thats the problem here.

Yet I'm able to launch ir with the manage command inside vscode and outside (from the src folder).
Here is my launch.json config file.
{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: Django",
      "type": "python",
      "request": "launch",
      "program": "${workspaceFolder}/src/manage.py",
      "args": ["runserver"],
      "django": true,
      "envFile": "${workspaceFolder}/.env.local",
      "justMyCode": true
    }
  ]
}

Thank you @mh-firouzjah! Your solution works great :)

Hello Everyone! The long requested ask is live!! Django tests are now supported in the VS Code python extension! Currently this is pre-release and would greatly appreciate Django users try this out and give feedback / submit bugs! (there will likely be bugs so I appeciate your patience as we work through them)
Steps:

Change your python extension so you are on the pre-release of the extension

specifically version v2024.13.2024081501

Add an environment variable called MANAGE_PY_PATH='path-string-to-manange.py-path' - below are steps specifically if you aren’t sure how to do this:

create a .env file
add MANAGE_PY_PATH='path-string-to-manange.py-path' to the file
add the following to your settings.json "python.envFile": "${workspaceFolder}/.env”  to pick up this file as part of the environment (edit path as necessary if .env not at root)

Now click on the “beaker” icon and pull up the test explorer panel
If not already selected, select “unittest” as your testing framework (you can also do the following in settings.json "python.testing.unittestEnabled": true,)
From here your tests should be discovered and populated in the tree and then run and debug should work! if this is not working try:

adding "python.testing.unittestArgs": [], maybe the tests just aren’t being found
look at the python output panel- this will show error messages
try from command line to make sure it works, then transfer that exact command to how it needs to be represented in VS Code. For example you have python manage.py test --arg to get the same in VS Code make MANAGE_PY_PATH='./manage.py' "python.testing.unittestArgs": [--arg]  and check the output as it should print the final command that is actually run
attempt using absolute path for manage.py if you are using relative to start

Let me know how it went! If it works or doesn’t please let me know so we can get an idea of how the feature is doing
To submit a bug:

try it again but with your logs on verbose, turn this one via “developer: set log level” command in the command palette
copy these logs and include them in your issue below
use the command palette and select python: report issue.. command, follow steps there and add your logs
either describe your repo structure / tests or include your repo (or it can be a minimal repro if your repo is private) so I can try it myself.

Thank you!

@eleanorjboyd just some quick spelling mistakes to save anyone copy pasting:

In 2 and 2ii, manange.py should be fixed to manage.py in MANAGE_PY_PATH=..., i.e. it should be MANAGE_PY_PATH='path-string-to-manage.py-path'
In 2iii, the setting JSON snippet should end in a straight double quote (") rather than a closing (curly) double quote (”)

And some very initial feedback: manage.py seems to be called with a verbosity -v flag without providing a verbosity level, causing the test discovery to error out:
2024-08-15 14:02:27.371 [info] Discovering unittest tests for workspace /home/eric/django-test with arguments: /home/eric/.vscode-server/extensions/ms-python.python-2024.13.2024081501-linux-x64/python_files/unittestadapter/discovery.py,--udiscovery,-v,-s,./mysite,-p,test_*.py

2024-08-15 14:02:27.371 [info] > ./.direnv/python-3.11.0rc1/bin/python ~/.vscode-server/extensions/ms-python.python-2024.13.2024081501-linux-x64/python_files/unittestadapter/discovery.py --udiscovery -v -s ./mysite -p test_*.py
2024-08-15 14:02:27.371 [info] cwd: .
2024-08-15 14:02:27.557 [info] MANAGE_PY_PATH is set, running Django discovery with path to manage.py as: $./manage.py
Running Django tests with command: ['/home/eric/django-test/.direnv/python-3.11.0rc1/bin/python', './manage.py', 'test', '--testrunner=django_test_runner.CustomDiscoveryTestRunner', '-v', '-s', './mysite', '-p', 'test_*.py']

2024-08-15 14:02:28.233 [error] usage: manage.py test [-h] [--noinput] [--testrunner TESTRUNNER] [--failfast]
                      [-t TOP_LEVEL] [-p PATTERN] [--keepdb]
                      [--shuffle [SEED]] [-r] [--debug-mode] [-d]
                      [--parallel [N]] [--tag TAGS]
                      [--exclude-tag EXCLUDE_TAGS] [--pdb] [-b]
                      [--no-faulthandler] [--timing] [-k TEST_NAME_PATTERNS]
                      [--version] [-v {0,1,2,3}] [--settings SETTINGS]
                      [--pythonpath PYTHONPATH] [--traceback] [--no-color]
                      [--force-color]
                      [test_label ...]
manage.py test: error: argument -v/--verbosity: expected one argument

Django test discovery process exited with non-zero error code See stderr above for more details.

This is reproducible against a generated Django v5.1 project created with django-admin startproject mysite as per the Django docs tutorial. Also note the -s ./mysite argument, which also does not seem to be supported.
LMK if you want me to flesh this out a proper GH issue.

Hi @swebra thank you for trying it! Ill make those fixes to the instructions. So for the args, you should be able to go into python.testings.unittestArgs in your workspace settings.json and remove all those args and just add the args you need for Django (not required just if you want a verbosity level or anything). The -v is a carry-over from unittest args but since they share the same setting for both we will have to update the default there. Thanks!

Ah of course. Test execution now works as expected in the generated project!
The only thing else of note there is a rogue error without any message during test discovery, just before the Found X test(s) info log. Seem to happen both on manual discovery (shown) and automatic discovery, and I similarly see it in "real" (non-generated) projects:
024-08-16 14:52:42.652 [debug] Testing: Manually triggered test refresh
2024-08-16 14:52:42.652 [debug] Testing: Clearing all discovered tests
2024-08-16 14:52:42.652 [debug] Testing: Forcing test data refresh
2024-08-16 14:52:42.652 [debug] Testing: Refreshing all test data
2024-08-16 14:52:42.674 [debug] Python API env change detected /home/eric/django-test/.direnv/python-3.11.0rc1/bin/python update
2024-08-16 14:52:42.675 [info] Discover tests for workspace name: django-test - uri: /home/eric/django-test
2024-08-16 14:52:42.675 [info] Running discovery for unittest using the new test adapter.
2024-08-16 14:52:42.675 [debug] Starting Test Discovery named pipe
2024-08-16 14:52:42.675 [debug] Creating named pipe server on /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock
2024-08-16 14:52:42.676 [warning] could not find a pixi interpreter for the interpreter at /home/eric/django-test/.direnv/python-3.11.0rc1/bin/python
2024-08-16 14:52:42.683 [info] Discovering unittest tests for workspace /home/eric/django-test with arguments: /home/eric/.vscode-server/extensions/ms-python.python-2024.13.2024081501-linux-x64/python_files/unittestadapter/discovery.py,--udiscovery,-p,test_*.py

2024-08-16 14:52:42.684 [info] > ./.direnv/python-3.11.0rc1/bin/python ~/.vscode-server/extensions/ms-python.python-2024.13.2024081501-linux-x64/python_files/unittestadapter/discovery.py --udiscovery -p test_*.py
2024-08-16 14:52:42.684 [info] cwd: .
2024-08-16 14:52:42.740 [info] MANAGE_PY_PATH is set, running Django discovery with path to manage.py as: $./manage.py

2024-08-16 14:52:42.740 [info] Running Django tests with command: ['/home/eric/django-test/.direnv/python-3.11.0rc1/bin/python', './manage.py', 'test', '--testrunner=django_test_runner.CustomDiscoveryTestRunner', '-p', 'test_*.py']

2024-08-16 14:52:42.986 [debug] new client is connected to the socket, connectionCount:  1 /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock
2024-08-16 14:52:42.986 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock connected
2024-08-16 14:52:42.988 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock received data
2024-08-16 14:52:42.988 [debug] Testing: Resolving item /home/eric/django-test
2024-08-16 14:52:42.989 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock received data
2024-08-16 14:52:42.989 [debug] client emitted close event, connectionCount:  0
2024-08-16 14:52:42.989 [debug] connection count is <= 0, closing the server:  /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock
2024-08-16 14:52:42.989 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock closed
2024-08-16 14:52:42.989 [debug] Test Discovery named pipe /run/user/1000/python-test-discovery-1334eef7af79d65a9496.sock disposed
2024-08-16 14:52:43.074 [error]

2024-08-16 14:52:43.074 [info] Found 1 test(s).

2024-08-16 14:52:43.086 [debug] deferredTill EOT resolved
2024-08-16 14:52:43.194 [debug] Testing: Resolving item /home/eric/django-test/mysite

hm yes I see that error- ill see if I can track it down. Does it change the outcome of the discovery or just appears in the logs?

Just appears in the logs as far as I can tell.


Fragmentation/Reassembly in ReactiveSocket

(Discussed with @benjchristensen, filing for possible discussion/consideration).
The fragmentation and reassembly concern seems better relegated to a layer below ReactiveSocket: this is an application-layer protocol, and fragmentation/reassembly seems more like a transport-level function. Practically, this would mean that this functionality would become the responsibility of the various DuplexConnection implementations, and disappear from the protocol.

Pushing Fragmentation/Reassembly down into the DuplexConnection opens the door for head of line blocking on long messages. Fragmentation is an inherent part of allowing the protocol to prevent (or at least account for the possibility of) head of line blocking at the protocol level.

The protocol docs already have the follow flag e.g. for Responses.  If we remove this we generally remove the ability for a specific implementation to support streaming responses (java.io.OutputStream etc) without making it an application level concern.

@vjn are you suggesting that the fragmentation/reassembly would be defined in a lower layer protocol (http/2, websocket, etc.) so it isn't required by ReactiveSocket?
In absence of the fragmentation flag in the frame, it wouldn't be possible to reassemble, isn't?

@NiteshKant, yes, that is what I had in mind: that this is a lower-level concern. However, I do note that AMQP (for instance) also elevates fragmentation/reassembly to the message transport level of the protocol. @tmontgomery's point is regarding head-of-line blocking. I understand that; in that case, do we expect users to break up all messages into manageable-sized fragments, or use up to the 2G maximum size? If they can use such large frames, won't even a single frame cause HOL blocking? And multiple such fragments of a giant message swamp the endpoints?

@vjn fragmentation/reassembly only applies when using ReactiveSocket over non-framed protocols like TCP, so yes you are correct that with HTTP/2, WebSocket, it isn't required but can not be completely removed.
IMO, HOL blocking in the scenario you have mentioned can only be removed by having stream priorities (assuming only always request-1 per stream is not a viable solution) but I am interested in hearing what @tmontgomery thinks about it.

I might end up leaving this discussion with a new mental model, how would you both consider a streamed response over a framed protocol.  e.g. I'm about to generate 20mb JSON document and I can easily stream it to the other process, using the fragmentation flag to define that these individual frames should be reconstituted into a single response in requestResponse or one fo the responses in a stream.
Specifically given support in ReactiveSocket for streaming of reassembled responses, the receiving end can start processing the response before the sender finishes.

@yschimke Making it possible for a receiver to start processing a response in chunks is contradictory to fragmentation. Typically, if you have fragmented frames, ReactiveSocket implementation would reassemble the frames before it emits them. So, fragmentation and reassembly is really an implementation detail and not something that gets exposed to the user of the API.
I think what you are trying to achieve is better suited with request-stream containing a single response. If it contains multiple responses, it will become an application level concern to delimit one response (over multiple frames) from another.

Typically, if you have fragmented frames, ReactiveSocket implementation would reassemble the frames before it emits them.

I think I'm biased by prior experience with HTTP frameworks instead of ReactiveStreams, so I'm happy to assume if this was a common problem that it would be part of the APIs.

Agreed with @NiteshKant regarding the exposure to application level. If that is wanted then the app has to do the chunking itself which would work well on top of the request-stream ReactiveSocket model.

Closing this, based on recent discussion. Thanks.


How to determine the required number of qubits for a VQC?

I am trying to understand the factors for determining the number of qubits.

I thought the number of qubits had to equal the number of classes because a class is interpreted from a bitstring. However, this tutorial for multiclass classification uses only two qubits for a 3 class classification. Does it mean we can classify N classes with log(N) qubits?
The number of features determines the number of qubits. However, I see two ways of increasing the number of parameters in the circuit to account for the features. We can increase the repetitions in the circuit, or we can increase the number of qubits. Is there a tradeoff?

Thank you very much!

Hello,

The number of qubits does not relate directly to the number of classes. In case of VQC you the underlying QNN is SamplerQNN that maps a measured bit string to a class. For $N$ qubits we get a bit string of length $2^N$, thus, in theory, we can classify up to $2^N$ classes from a $N$-qubit circuit. So, answering your question, yes, that's right.
The number of features usually determine the number of qubits, but not necessarily. In case of rotation encoding, the number of qubits equals to the number of features. In case of amplitude encoding the number of qubits is $log(N)$, where $N$ is the number of features. But nothing prevents me to encode, say, 10 features into 5 or 6 qubits. You can increase the number of trainable parameters by increasing complexity of your ansatz, e.g. by increasing the number of repetitions. Increasing the number of qubits to increase the number of parameters is still possible, but may be not the best idea considering the limited number of qubits in real hardware. Tradeoff is very simple - design a model in the way that it requires a limited number of qubits and parameters and in the same time it produces good results. As usual, good metrics are the number of qubits and the depth of your circuit. Both of them is a limited resource.

Let me know if you have questions.

Thank you, that helped!
I understand that first, I should try increasing the complexity of the circuit (more repetition) before increasing qubits to account for encoding more features. In the documentation for Circuit library, there is some distinction between Data encoding circuits and N-local circuits. However, can use any of them for data encoding?
Also, unable to find RawFeatureVector, has it been deprecated? Is there an equivalent?

My mistake, strange that the search result for RawFeatureVector was only the Reference page. Thank you!

Yes, Thank you!


Area title HUD

Added a little animated title that temporarily pops up when you go to a new area

Devs can use the HudAreaLabel resource to activate a title whenever they like:
fn show_area_title(mut hud: ResMut<HudAreaLabel>) {
    hud.show_area_title("Overworld");
}

I used individual images because UI nodes do not support spritesheets. However, the full spritesheet is included in case this is added to Bevy, as well as the source file (made in Affinity Designer) and an SVG vector file for compatibility with other software.

I see this was left unmerged without any more issues to address. I'm merging it.


[Component Request] CSS Class for Button Action Floating

Description
I would like to request additional CSS class to button to create the floating action button. It is seen on most new Google apps.
Material Design Spec link
http://www.google.com/design/spec/components/buttons-floating-action-button.html#buttons-floating-action-button-floating-action-button

For your convenience, I have provided an example for desktop
.mdl-button--floating-action {
position: fixed;
right: 24px;
bottom: 24px;
padding-top: 24px;
margin-bottom: 0;
z-index: 998;
}

Is this request for the button itself (which we currently have) or a way to position it somewhere precisely on the page?

@Garbee Correct material  designed buttons exist, but I didn't see the action floating properties anywhere. I was thinking adding CSS class named something like .mdl-button--floating-action that would place the button in lower left corner (24dp margin on desktop/tablet and 16dp margin on phone).

@Garbee I included the Material Design specification in my request.

That's the problem though, positioning. We'd need multiple positions since the button can be placed anywhere really. Plus, depending on design and placement of items, people will need them at different places even where we do support.
Looking over our classnames... ugh. I'm not sure BEM allows double-modifiers. We'd need to either make Fab's a new element level item so modifiers can be applied or leave this to custom code until 2.0.
Also --floating-action isn't a sound modifier imo, --fab exists which is the semantic acronym for the same thing, could get confusing.
So, any suggestions for how to implement this in a BC way?
P.S. Yea, I saw the spec sheet, it only raises more doubts in my mind on implementing it well. But I could also just be insane right now.

Thanks for adding it to the wish list. I have a few others that I will create tickets for as I implement. I will create a ticket for each with reference to MD documentation and suggested implementation. Thanks again.

Just wanted to add link to [materializecss] (http://materializecss.com/buttons.html) who has implemented this button behaviour.

Pulling from their example though...

      mode_edit

insert_chart

format_quote

publish

attach_file

Note the very first line, how they are applying absolute positioning manually. That is what you can do right now with our setup. Your request is for classes to handle this automatically, which can be done but is a bit more complicated to handle well (partly due to our naming convention being slightly off.)

@Garbee Thank you for pointing this out. Perhaps this is just an update to documentation.
I need to add this css to get a floating action button with MDL.
.mdl-button--floating-action {
bottom:45px;
margin-bottom:100px;
padding-top:24px;
position:fixed;
right:24px;
z-index:998;
}

That is general CSS. Not something we should directly document without a
very compelling reason.
On Jul 9, 2015 11:53 AM, "Dan"<EMAIL_ADDRESS>wrote:

@Garbee https://github.com/Garbee Thank you for pointing this out.
Perhaps this is just an update to documentation.
I need to add this css to get a floating action button with MDL.
.mdl-button--floating-action {
bottom:45px;
margin-bottom:100px;
padding-top:24px;
position:fixed;
right:24px;
z-index:998;
}
—
Reply to this email directly or view it on GitHub
https://github.com/google/material-design-lite/issues/901#issuecomment-120046436
.

The "floating action button" is well-documented in Material Design guidelines (own section) and used across all new Google apps e.g. Inbox.
Isn't this reason enough to have an example on how to implement a "floating action button" with MDL?
Would you like me to do a pull request on update to MDL documentation?

Closing this on out.
We can't directly handle classes to position since:

It doesn't directly have defined locations.
Dynamically figuring it out is intensive when abstracted to the degree we would need to.

Perhaps in V2 docs we can have some example usage sections to show custom things like this. However, it is low-priority since positioning is a basic concept of CSS that developer should know when implementing a design like this.

This should be reopened: Inline style rules is a NO-NO. The designer can over-ride just like all the other modules here.

@CelticParser Where do inline styles come in? That is one choice a developer may make when implementing their own styling to move things around. However it is still completely valid (and what I assumed) that developers make their own classes to do these things.

this really should be implemented as part of the "library" and not something the developer is required to implement

I just brought this up yesterday for V2 and we will make it explicit in V2 that developers are to implement this on their own. The placement is not defined and it depends upon the application's UX where exactly it should go. Therefore, it is best left to developers.
This is fairly trivial absolute positioning, no real reason for us to include some form by default.

Agree with the benefits of this, using material after my time with bootstrap, and love the css class "pull-right". Missing all the small assistant styles... The more options the a developer has (even though its a few lines of CSS), the happier they are using a full pledged system with all the bells and whistles. Of course, this adds the complexities, documentation and weight to a (almost) weightless system.

We are not a full framework. We are specifically providing Material Design components to use within your application. Doing everything for you to just have classes to build things is not the intent of the project.
Placing a FAB absolutely is non-trivial. It's proper placement depends upon your sites design. There isn't a feasible way we can include this type of placement modifier in the core.

You can probably copy the 'view-source' button style as shown in https://getmdl.io/templates/index.html examples

I just moved on to a more mature framework.


Proposal: extract core checkbox and radio behavior into separate @Directives

Do you want to request a feature or report a bug? feature
What is the current behavior?
A User can use md-checkbox / md-radio-X components to create chebox's and radio's.
What is the expected behavior?
User can apply checkbox / radio behaviour to any element as well as use the currently available
md-checkbox / md-radio-X components.
What is the motivation / use case for changing the behavior?
By implement checkbox & radio group/button as directive's users can apply them on other elements.
The current implementation of md-checkbox & md-radio-X will use the directives in their implementation so we are not duplicating code.
Having this, user's can easily create a Toggle button or a group of toggle buttons as well as radio group made of buttons (as in bootstrap).

This will also enable #112 today and reduce the code for it.

Other information
This is generally how the directive should look like

import {
    Directive,
    Provider,
    forwardRef,
    Input,
    Output,
    EventEmitter
} from 'angular2/core';

import {
    NG_VALUE_ACCESSOR,
    ControlValueAccessor
} from 'angular2/src/common/forms/directives/control_value_accessor';

let nextId = 0;

const CHECKBOX_CONTROL_VALUE_ACCESSOR = new Provider(
    NG_VALUE_ACCESSOR, {
        useExisting: forwardRef(() => AsCheckbox),
        multi: true
    });

@Directive({
    selector: '[asCheckbox]',
    host: {
        'role': 'checkbox',
        '[id]': 'id',
        '[class.md-checkbox-checked]': 'checked',
        '[class.md-checkbox-disabled]': 'disabled',
        '[attr.tabindex]': 'disabled ? null : tabindex',
        '[attr.aria-checked]': 'ariaChecked',
        '[attr.aria-disabled]': 'disabled',
        '(click)': 'onInteractionEvent($event)',
        '(keydown.space)': 'onKeyDown($event)',
        '(keyup.space)': 'onInteractionEvent($event)',
        '(blur)': 'onTouched()'
    },
    providers: [CHECKBOX_CONTROL_VALUE_ACCESSOR]
})
export class AsCheckbox implements ControlValueAccessor {
    @Input() id: string = `md-checkbox-${++nextId}`;
    @Input() disabled: boolean = false;
    @Input() tabindex: number = 0;

    @Input() get checked(): boolean {
         return this._checked;
     }

     set checked(checked: boolean) {
         if (checked === this._checked) return;
         this._checked = checked;
         this.change.emit(this._checked);
     }

    get ariaChecked(): 'true' | 'false' {
        return this.checked ? 'true' : 'false';
    }

    @Output() change: EventEmitter<boolean> = new EventEmitter();

    private _checked: boolean = false;
    private _changeSubscription: {unsubscribe: () => any} = null;

    toggle() {
        this.checked = !this.checked;
    }

    onInteractionEvent(event: Event) {
        if (this.disabled) {
            event.preventDefault();
            event.stopPropagation();
            return;
        }
        this.toggle();
    }

    onKeyDown(evt: Event) {
        evt.preventDefault();
    }

    writeValue(value: any) {
        this.checked = !!value;
    }

    registerOnChange(fn: any) {
        if (this._changeSubscription) {
            this._changeSubscription.unsubscribe();
        }
        this._changeSubscription = <{unsubscribe: () => any}>this.change.subscribe(fn);
    }

    registerOnTouched(fn: any) {
        this.onTouched = fn;
    }

    onTouched(): any {}

}

From here, creating a toggle button is as simple as
Toggle me

The code above was writting on the fly so it's just for the idea.
I'v implemented it in some components I work with not related to material, I can post a PR if you like the idea.

Notes:

In my project I also use inheritance since a lot of the code for radio group and checkbox is similar.
This is also true for implementing ControlValueAccessor, same code. maybe mixins can help here or not if it makes the code less readable.
Indeterminate state left out, should be added on md-checkbox

This is an interesting thought on composable behavior. It's really more of a md-checkable than "checkbox" (you can also break down the behavior into, say, md-disableable and md-touchable).
I generally like this idea of extracting individual behaviors as building blocks for components. We've definitely talked about it before, and had been leaning on mixins as being the approach. Mixins make sense because, while all of these "checkable" components share some behavior, there is special behavior for each one, meaning that you'd end up building a custom component for each anyway. The result is easier to test and reuse.
Aside from toggle-buttons and slide-toggle, do you have any other use cases in mind for such a md-checkable directive? We already plan on both of these in the near future (#112, #90).

@jelbourn In my project I do use mixins, this is because I have a Checkable class that is good for both checkbox and radio-group and ControlValueAccessor mixin as well..
The reason I did not emphasise it was the goal's of this project, it is intended to be an example of how to do Angular 2 and mixins, while wonderful and powerful are a bit more difficult to understand as opposed to classical inheritance... One have to connect the dots, tough its not that bad.
As for more ideas, I have mentioned that checkbox is also used for radio, which in turn enables Exclusive selection in mdToggleButtonGroup with the help of RadioDispatcher
md-disableable and md-touchable are doable but they need to be both mixin classes and directives so in the code the md-checkbox will not use it in the template but mix it into itself. (avoid boilerplate)
This is quite a robust composition system :)
With your permission, I would like to offer my contribution with a small PR that will introduce a refactor for md-chackable directive and Checkable mixin class.
Once review iteration is done I will start introducing more composable units until we have full composition for radio and checkbox.
Is this an option?
Thanks, Shlomi

@shlomiassaf I'd rather not introduce a new checkable directive at this point; it's not clear to me yet that it would be very valuable to users. As far as refactoring the existing code into a mixin, @DevVersion may have already started on refactoring checkbox code out to be used in the new slide-toggle. I'm not sure what his status on that is at the moment.

Sure thing, let me know if you need my help

Now that TypeScript 2.2 supports mixins we will be exploring breaking out functionality into reusable pieces for the cdk. See #3944 for the first explorations into this space.


Fix OTLP timestamps

Description
OTLP timestamp are multiplied by 10^9 before being stored, even though they should already be in nano format.
How was this PR tested?
Describe how you tested this PR.

ok, the problem was in the test, the dummy value provided was not detected as a nano timestamp


Test stream

Adding some tests for the base Stream class

@Hopding Back to you. Thanks for being so prompt!
On an unrelated note, I'm working on an Acroforms API. Is there any work begun on this that isn't yet in a branch?

@cshenks I'm glad to hear you're interested in working on this! Creating an Acroforms API has been near the top of my unofficial pdf-lib roadmap for some time now. I just haven't had the time to get around to it myself yet. I'd be happy to collaborate on this with you. I have some high-level ideas about how to best implement such an API in pdf-lib.
While there is not currently any work being done on an Acroforms API, you might mind the following issues to be helpful. They demonstrate how to create/edit Acroforms with the low-level APIs:

https://github.com/Hopding/pdf-lib/issues/39
https://github.com/Hopding/pdf-lib/issues/48
https://github.com/Hopding/pdf-lib/issues/109
https://github.com/Hopding/pdf-lib/issues/111
https://github.com/Hopding/pdf-lib/issues/122

@Hopding I'd be happy to collaborate! Is there any chance you'd be willing to give me permissions to add a new project for this?


How to make it obfuscate string all class without use anotation?

Hi,
I want to obfuscate string on every class in my project. How can i do that without adding @Obfuscate on all class?
If this feature is not available, where can i fast modify your code to make it?
Thank you!

Hi,
This is a duplicate of #23. If you want to make you fork without @Obfuscate I believe you can assume that @Obfuscate is applied to every class here. But I'm going to make a change in the library quite soon that will allow to configure obfuscation from build.gradle.


Can we talk we Email?

Is it possible to share your email?

Sure!
<EMAIL_ADDRESS>

Thank you so much.


game crashed when i killed adjudicator

https://crashy.net/3c896a30-6bec-4198-bfe7-b04979d67cbb

Fixed for an upcoming release


negative loss and failed to load model for retraining

I encountered 2 issues when following the work.
The first one is when I trained the model from the scratch. I got some negative loss. I read your paper and think this case is normal, right?
However,  after 7 epochs about 22000 iterations, I synthesized the audio,  got some white noise. Are there any mistakes?  How many epochs and iterations do you train?
The second one is I retrained the model from your "waveglow_old.pt ",  when loading the model but lost "iteration", "optimizer" and "learning_rate". When training, I got "tensor(nan., device='cuda:0')" and loss is Nan.
Can you help me or give some advices?

I moved the waveglow into my training framework and trained it with DataParallel from scratch. I also got negative loss and 'nan' during training.

Besides, the infered samples even reach 1e17.

@chaiyujin so, what your  training framework?  retrain from the provided model, did it work? I am lost your words

@benlaitang Sorry about my english. I have my own training framework.  I trained glow from scratch. Never test fine-tuning from provided model.

I got exactly the same problem.

@wenyong-h did you solve the problem?

No, I'm training from scratch now.

Yes, the loss should be negative. We trained the model for 580 iterations with batch size 24. 22k iters with batch size 3 is probably not enough to produce intelligible speech.
The second model is shared for inference only, not resuming training.

@rafaelvalle thanks a lot. I will try from scratch again.

Closing. Please re-open if necessary.

@chaiyujin Did you solve the issue of nan loss? Because I encounter similar issue. My training curve is something like these:

@yoyololicon

When training with default model given by this repo, I encounter NAN.
[Solve]: I initialize the upsample layer weight to be 1.0, bias to be 0.0.
When training with multi-gpus, I encounter NAN again.
[Solve]: Use one gpu, batch_size = 4

I'm not sure how to avoid nan.
It works to train with inited upsample layer and on one gpu.

@chaiyujin thanks for your solution. I also met the NAN problem. When I set the one GPU with batch size 1, the training loss is fine. But when I set 8 GPU by using torch.nn.parallel.data_parallel with batch size 8, I got NAN loss after a few thousand steps. I haven't found any solution to solve it.


fix(dropdown): Added support for class directives in the dropdown component

Added support for class directives in the dropdown component. The component currently supports attribute directives only.

class directive support has been intentionally removed after this discussion :
https://github.com/angular-ui/bootstrap/issues/2156

Thanks for the find @Delapouite - closing as addressed by #2156.


[BUG] overrideConfig for parameter in 2 nodes not working as documented

Describe the bug
In calling a prediction endpoint of a chatbot via API, I  tried using the overrideConfig method for passing an overrideConfig to two parameters named the same way in two different nodes (in my case openAIEmbeddings and chatOpenAI, both having a modelName parameter), However, the call seems to allow override of all chatflow parameters except the modelName for either of the two nodes. The value of modelName remains fixed to the default, regardless of the values given in overrideConfig. This happens despite the fact that I am specifying the two model names as described in the documentation (in the Embed window of the chatflow web interface):
body_data = {
        "question": <some_question>,
        "overrideConfig": {
            "modelName[openAIEmbeddings_0]": <embedding_model_name>,
            "modelName[chatOpenAI_0]": <chat_model_name>,
            ...
        }
}

(I know that the structure of the body_data argument is different in the image snippet, with no "overrideConfig", but I followed the structure of the main documentation. Also, trying to use the format without "overrideConfig" results in none of the parameters being updated)
(I double checked, that seems to be the name of the two nodes, at least according to the export of the chatflow schema)
If I don't specify the node name (i.e. just using one "modelName": ...), the value seems to be read correctly, but then of course one of the two nodes complains about a non-existing model.
The issue seem to be caused by using the "<param_name>[<node_id>]" syntax, since using it for a uniquely named parameter in the chatflow also results in the override value not being used.
To Reproduce
I am calling the API from a python script, following the format suggested in the API documentation:
# ... API url and key were read from .env, overrideConfigs stored in settings dictionary
headers = {"Authorization": f"Bearer {API_KEY}"}

def query(body_data) -> Response:
    response = requests.post(API_URL, json=body_data, headers=headers)
    return response

def main() -> None:
    body_data = {
        "question": settings["question"],
        "overrideConfig": {
            "modelName[openAIEmbeddings_0]": settings["modelName_embeddings"],
            "modelName[chatOpenAI_0]": settings["modelName_chat"],
            # other params
        },
    }
    output = query(body_data)

The chatflow schema is a simple qna from a text-file document with in-memory store:

Expected behavior
When using an overrideConfig parameter containing the fields "modelName[<node_id>] is expect the value of the modelName parameter to be updated.
Flow
test_overrideConfig_issue Chatflow.json
Setup

Installed via docker
Flowise Version: 1.4.12
OS: Amazon Linux 2023
Browser: n.a.

can you try this instead?
# ... API url and key were read from .env, overrideConfigs stored in settings dictionary
headers = {"Authorization": f"Bearer {API_KEY}"}

def query(body_data) -> Response:
    response = requests.post(API_URL, json=body_data, headers=headers)
    return response

def main() -> None:
    body_data = {
        "question": settings["question"],
        "overrideConfig": {
            "modelName": {
                 "openAIEmbeddings_0": model1,
                 "chatOpenAI_0": model2
            }
        },
    }
    output = query(body_data)

Hi Henry, thanks for the suggestion. It still seems like something is off, since I now get this error:
DEBUG:root:b"400 We could not parse the JSON body of your request. (HINT: This likely means you aren't using your HTTP library correctly. The OpenAI API expects a JSON payload, but what was sent was not valid JSON. If you have trouble figuring out how to fix this, please contact us through our help center at help.openai.com.)"
(to me it seems like the value of the "modelName" field is still sent all to both of the nodes, leading to OpenAI API complaining that they are getting the wrong format for modelName in the call)

I've tested it works:

check if you are passing in correct model name (string)

You are right, somehow I must have made a mistake in calling the model name. Sorry about that, trying again with your suggested format worked well. Thanks a lot!


Unexpected trigger behavior when using a function on old and new

Curious whether this behavior is expected and therefore a gotcha that should be documented, or unexpected and should be changed.
The schema below inserts a OneLog object every time One is updated by concatenating a string, but if the One objects themselves are passed into a function doing the same it will only treat the object as the __old__ one.
    type One {
        required is_true: bool;

        trigger log_update after update for each
      do (
        insert OneLog {
          info := <str>__old__.is_true ++ ' -> ' ++ <str>__new__.is_true,
          function_info := make_info(__old__, __new__)
        }
      );
    }

    type OneLog {
        required info: str;
        required function_info: str;
    }

    function make_info(first_one: One, second_one: One) -> optional str using (
        <str>first_one.is_true ++ ' -> ' ++ <str>second_one.is_true
    );

Queries:
myinstance:edgedb> insert One { is_true := true };
{default::One {id: e716bc8c-b022-11ee-b4d7-2351659bae9c}}
myinstance:edgedb> update One set { is_true := false };
{default::One {id: e716bc8c-b022-11ee-b4d7-2351659bae9c}}
myinstance:edgedb> select OneLog {*};
{default::OneLog {id: eaefad0a-b022-11ee-b4d7-d3c7245f7ecc, function_info: 'true -> true', info: 'true -> false'}}

Thanks for writing this up as a bug report. I suspect this is unexpected, but eager to hear what others think, thanks again.


Create cluster role definition for nonrootbuilder

A cluster role is needed to link a service account to a scc when deployed via ArgoCD (command line creates this role automatically).

Fixed via: https://github.com/ploigos/openshift-actions-runner-chart/pull/18


feat: migrate history to id_indexed_v1 store

What was wrong?
History storage is still using the legacy store.
How was it fixed?
This PR migrates it to the new storage scheme.
Remaining work afterwards will be to remove legacy store and cleanup code where applicable.
To-Do

[x] Clean up commit history and use conventional commits.

I ran the database migration of 5 of my nodes. Nothing seems to be wrong, I did have to rebase the PR for the fix for blobs. Other then that I didn't see any issues. It took 50 minutes for the migration to run though.

It took 50 minutes for the migration to run though.

That's definitely way more than I expected. I'm wondering why it takes so long. How big are the databases, both in GB and how many entries?
I checked the migration code, and this is what it does (sadly, SQLite doesn't support renaming indices, otherwise this would be much simpler):

Rename table:ALTER TABLE history RENAME TO ii1_history

Deleting old indices:DROP INDEX history_distance_short_idx;
DROP INDEX history_content_size_idx;

Updating usage stats:SELECT COUNT(*) as count, TOTAL(content_size) as used_capacity FROM ii1_history

Creating new indices:CREATE INDEX IF NOT EXISTS ii1_history_distance_short_idx ON ii1_history (distance_short);
CREATE INDEX IF NOT EXISTS ii1_history_content_size_idx ON ii1_history (content_size);

I think renaming the table and dropping indices should be fast (maybe few seconds).
Creating new indices might be slow, but I believe that 50 minutes is too much. You were playing with creating and deleting indices in the past in order to improve performance. How long would it usually take?
So the issue might be calculating usage stats. Now that I wrote everything down, that one is definitely slower then in needs to be because it runs without any content_size index being present. @KolbyML , would you be able to test migration again if I modify the migration code? I can also add some debug statements and we can see how long each operation takes.

I rebased this PR.
I also created a commit (not part of this pr, yet) that hopefully improves migration: https://github.com/morph-dev/trin/commit/176f64a13cbe51bfce6a26c77527c5116c28befd
I will try to test it on my machine, but it will be a small db. I hope at least to be able to figure out expected outcome.

The nodes are only configured to hold like 100mb or something of logs so they got deleted before I could see them

It looks like the time is now 7-8 minutes to migrate instead of 50 minutes on a 10gb database with ~1 million keys.

It looks like the time is now 7-8 minutes to migrate instead of 50 minutes on a 10gb database with ~1 million keys.

I would say that's reasonable. If everyone agrees, I would proceed with merging this PR.
@KolbyML Do you have the logs so we can see how long each part takes? It would be useful to know for potential future migration

@KolbyML Do you have the logs so we can see how long each part takes? It would be useful to know for potential future migration

The nodes are only configured to hold like 100mb or something of logs so they got deleted before I could see them

I would say that's reasonable. If everyone agrees, I would proceed with merging this PR.

I think it is fine, I just think we need status info logs which are like
info!("Trin Database Migration from X to Y in progress: Z% done") or something along those lines just so people know something is happening

I think it is fine, I just think we need status info logs which are like
info!("Trin Database Migration from X to Y in progress: Z% done") or something along those lines just so people know something is happening

We can't have "%done part", because we only run simple SQL commands. I changed "Migration started" and "Migration finished" logs to info level. I would say that's enough, considering that this will happen only once.

I would just make it up if we have 6 steps 1/6= how much percent is done


Security Vulnerabilities (NSP) - dependency upgrades

$ nsp check(+)

Denial-of-Service Extended Event Loop Blocking
Dependency: qs
Installed: 0.6.6
Vulnerable: <1.0.0
Patched: >= 1.x
Path<EMAIL_ADDRESS>><EMAIL_ADDRESS>><EMAIL_ADDRESS>More Info: https://nodesecurity.io/advisories/28

Denial-of-Service Memory Exhaustion
Dependency: qs
Installed: 0.6.6
Vulnerable: <1.0.0
Patched: >= 1.x
Path<EMAIL_ADDRESS>><EMAIL_ADDRESS>><EMAIL_ADDRESS>More Info: https://nodesecurity.io/advisories/29

Regular Expression Denial of Service
Dependency: hawk
Installed: 1.0.0
Vulnerable: < 3.1.3  || >= 4.0.0 <4.1.1
Patched: >=3.1.3 < 4.0.0 || >=4.1.1
Path<EMAIL_ADDRESS>><EMAIL_ADDRESS>><EMAIL_ADDRESS>More Info: https://nodesecurity.io/advisories/77

ReDoS via long string of semicolons
Dependency: tough-cookie
Installed: 0.9.15
Vulnerable: >=0.9.7 <=2.2.2
Patched: >=2.3.0
Path<EMAIL_ADDRESS>><EMAIL_ADDRESS>><EMAIL_ADDRESS>More Info: https://nodesecurity.io/advisories/130

It looks like request needs to be updated to the latest version to fix these.

Can you do a package version for npm to get the repo code out?  https://www.npmjs.com/package/node-linkedin is still pointing to the 8 mo old code.

done :)


pre-allocated namespace size considerations in LVM2 mode

Currently implemented method is: we pre-allocate fixed-size namespaces which will become a Phys.Volumes for LVM2. The fixed size for pre-allocation can be given as argument and defaults to 32GB.
This idea of fixed-size pre-alloc was implemented based on example on openstack side, I think.
What is advantage to pre-allocate multiple namespaces per Region overall?
In theory, the advantage is increased flexibility, for example one can reduce number of Phys.Volumes without need to resize existing. But our driver will not do/need that.
If we leave manual re-config options off the table, driver would work equally if it had just one namespace per region configured (or two if we support fsdax and sector modes with separate Namespaces).
Single-namespace mode has advantage that we don't waste smaller-than-block space that would remain as leftover, also we consume minimum amount of labels and namespace-size overhead.
But we dont want to hard-code single-namespace mode either, right.
To keep maximum flexibility, I propose we add special value to pre-allocated size,   expressed with some impossible option value  like 0 or -1, meaning "single space" instead of fixed-size blocks.

shortcomings of current approach are addressed in #108

merge of #108 addressed most of concerns raised here.
The two remaining ones are:

do we need special mode creating single namespace instead of fixed blocks (asked by blocksize=-1)
any gain from using striped mode of LVM2 in this driver

As this issue contains multiple solved aspects and longish discussion, what about closing this and creating new more focused issues (if the two listed ideas are worth it) ?

I propose, no additional modes/options, let's keep what we have, and close this issue


[BUG] Reloading the page breaks the avatar path

What version of Node.js are you using?
does not matter
What version of npm are you using?
No response
What operating system are you using?
does not matter
Describe the Bug
/citizen
Before reload

After reload

Before reload

After reload

Console: Failed to load resource: net::ERR_NAME_NOT_RESOLVED

Expected Behavior
--
To Reproduce
No response

This should be fixed now. Let me know if it's not!

No, it still doesn't work

No, it still doesn't work

Are you able to run yarn in the root folder of the CAD. Then re-build the client, then check if it works? There is a new package that has to be installed

I always do this after updating the repository

Everything is working now. Cool!


fix #80 add missing augmentations to Test and Suite

reactivate PackageTest suite
bump dependency towards mocha to ^5.2.0
bump dependency towards @types/mocha to ^5.2.0

I've deleted the Mocha namespace, now that @types/mocha is a dependency we no longer need them.

@pana-cc I am merging this now an will rebase #82 to master

Ups, I fucked up. Need to investigate.


ArrayLit: use AssignableConv

Codecov Report

Merging #163 (b6701af) into main (0a7940c) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #163   +/-   ##
=======================================
  Coverage   94.82%   94.82%
=======================================
  Files          15       15
  Lines        5935     5935
=======================================
  Hits         5628     5628
  Misses        236      236
  Partials       71       71

Impacted Files
Coverage Δ

codebuild.go
95.82% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a7940c...b6701af. Read the comment docs.


Home directory mixup

Windows Version
10.0.17134 (Build 17134)
Summary
WSL gets home directory mixed up when more than one distro and user is created
How to replicate

Install Ubuntu from the Marketplace
On completion/launch create user: user01
Uninstalled Ubuntu
Install Ubuntu 18.04 from Marketplace
On completion/launch create user: user02
echo $HOME    # /home/user02
Go to your local git repo; run git fetch

Could not create directory '/home/user01/.ssh'.
The authenticity of host 'host.demo (x.x.x.x)' can't be established.
RSA key fingerprint is SHA256:zz...1A.
Are you sure you want to continue connecting (yes/no)? yes
Failed to add the host to the list of known hosts (/home/user01/.ssh/known_hosts).
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

On further investigation I found /etc/passwd was different when logged in as user02 and root
Logged in as user02:
user02@MY-MACHINE: cat /etc/passwd
user02:x:1000:1000:,,,:/home/user02:/bin

Logged in as root:
root@MY-MACHINE: cat /etc/passwd
user01:x:1000:1000:,,,:/home/user01:/bin

Further, and what is more perplexing, when logged in as user02, using tail or less will return a row for user01 (none for user02)
user02@MY-MACHINE: tail -n 1 /etc/passwd
user01:x:1000:1000:,,,:/home/user01:/bin

What I expect

/etc/passwd file to be static

Each distribution has its own /etc/passwd file. Think of each distribution you download has a completely separate environment with its own users, packages, and filesystem.

@benhillis

Does that explain why cat /etc/passwd and tail /etc/password is not the same?
How do I know what distribution I'm currently in?

How do I know what distribution I'm currently in?

Remains a good question. But as a practical matter, with the Distros currently shipping in the Store, you can get an idea with lsb_release -a.

How do I completely remove the first distribution (Ubuntu)
Uninstall the app. Alternitively you can use wslconfig.exe:

wslconfig.exe /list
wslconfig.exe /unregister <distro_you_wish_to_remove>

It looks like Ubuntu has been uninstalled. So it seems some artifacts such as /etc/passwd are still lingering around
C:\Users\w_user01>wslconfig.exe /list
Windows Subsystem for Linux Distributions:
Ubuntu-18.04 (Default)

so it seems some artifacts such as /etc/passwd are still lingering around

Precisely why I avoided your third question. wslconfig.exe lacks /cleanupturds. Someone will tell you to violate this blog and delete:
C:\Users\<you\>\AppData\Local\Microsoft\WindowsApps\CanonicalGroupLimited*
YRMV

wslconfig.exe lacks /cleanupturds
Let me rename this issue to that :D

Will try deleting C:\Users<you>\AppData\Local\Packages\CanonicalGroupLimited* later.

I've done the following and it still doesn't fix the issue:

Uninstalled Ubuntu 18.04
Deleted the dir: C:\Users\AppData\Local\Packages\CanonicalGroupLimited*
Restarted computer
Installed 18.04
Created user: user02

cat /etc/passwd displays user02
tail -n 1 /etc/passwd still displays user01
ie. Same issue as reported
This data must either be in /tmp or in registry

Then uninstall everything and delete HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Lxss

cat /etc/passwd displays user02
tail -n 1 /etc/passwd still displays user01

I don't understand what is happening here.  /etc/passwd is a unique file in each distribution's filesystem which is located in their distinct app data directory.  I can't reproduce this locally, so there must be some additional step that I'm missing.

@onomatopellan I did the following:

Uninstalled Ubuntu 18.04
Removed primary key and all sub keys (if present): Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Lxss\{12345678-1234-5678-0123-456789abcdef}
Restarted computer
Also ran wsreset.exe (on another iteration)
Reinstalled 18.04

No change.

@benhillis On the first distro install I may have hit escape while in the shell command. I can't remember exactly which step. It was waiting for user input (I think) rather than executing something. I may have accidentally hit escape. That is the only thing I can think of that is out of the ordinary.

What is your process for logging in as a different user? Are you starting the same every time (clicking on an icon, running wsl.exe) and then running su? Does nano /etc/passwd look like cat or tail? Can you do a quick check of 'alias tail; alias cat; which tail; which cat'  to make sure nothing looks out of the ordinary?

Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Lxss\{12345678-1234-5678-0123-456789abcdef}

Something strange.
{12345678-1234-5678-0123-456789abcdef} is GUID for Legacy instance (Ubuntu 16.04).
From the output of wslconfig.exe /list, you should not have installed it.

@Brian-Perkins

What is your process for logging in as a different user
After install, I create a user, in this example, user02. To start WSL, i usually start cmd/powershell/conemu then type bash. I automatically get logged in as the created user. I don't try to change user manually. user01 was created when I installed the first distro, which I later uninstalled.
To run as root, I run sudo su and enter my password for my created user.

Does nano /etc/passwd look like cat or tail?
nano looks like cat. tail seems to be the only command I can see that is out of the ordinary

Can you do a quick check of 'alias tail; alias cat; which tail; which cat' to make sure nothing looks out of the ordinary?

$ alias tail
bash: alias: tail: not found
$ alias cat
bash: alias: cat: not found
$ which tail
/usr/bin/tail
$ which cat
/bin/cat

@0xbadfca11 Actually I copied that key from another machine just to show the string pattern. The actual key is different and as far as I could tell changed every install.

@benhillis

I can't reproduce this locally, so there must be some additional step that I'm missing.

I don't think you will be able to reproduce simply by uninstalling then installing.
I think there is some significance that Ubuntu was installed first then Ubuntu 18.04.
If you installed 18.04 first, then uninstalled 18.04, then followed my steps, it is possible you won't be able to reproduce.
When I have time, I may be able to do a system refresh and try install Ubuntu 18.04 first then Ubuntu and see if there is difference.


Use community users sf

This PR, incorporates both the iOS changes, and the SF changes.

The new setup doesn't need the sample data from the previous version.
Please remove the isMaster__c field from Account, as well as the sample data itself.

@muenzpraeger I believe I've addressed all your concerns.