Always use LF instead of CRLF for *.ipynb files regardless of platform and when sending code to the kernel

Environment data

VS Code version: 1.53.0-insider (2021-01-29T05:13:48.533Z)
Jupyter Extension version (available under the Extensions sidebar): v2020.12.414227025
Python Extension version (available under the Extensions sidebar): 2021.2.518958983-dev
OS (Windows | Mac | Linux distro) and version: Windows 10
Python and/or Anaconda version: Python 3.8.5 using Anaconda
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
Jupyter server running: Local

Expected behaviour
When editing a notebook, lines ending in "\n" shouldn't be modified to "\r\n"
Actual behaviour
It appears as soon as I modify the content of a cell, the editor is replacing all "\n" by "\r\n". I don't understand why, and it's causing downstream tools (e.g. nbdev/nbparse) to no longer be able to parse the .ipynb properly.
Steps to reproduce:
This seems to happen for any .ipynb file on my PC. I did a git clone of nbdev (https://github.com/fastai/nbdev/blob/master/nbs/00_export.ipynb) and ran the notebook, but chances are you only need to open a file:

download https://github.com/fastai/nbdev/blob/master/nbs/00_export.ipynb
open the file in vscode
modify a cell
do a diff on the cell.
notice that the newlines are messed up.

Logs

Output for Jupyter in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Jupyter)

User belongs to experiment group 'pythonJoinMailingListVar1'
User belongs to experiment group 'jupyterTest'
User belongs to experiment group 'NativeNotebookEditor'
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py -c "import jupyter"
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py -c "import notebook"
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py jupyter kernelspec --version
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.jupyter_daemon -v
> D:\Users\claforte\Anaconda3\envs\tensorflow\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py -c "import ipykernel"
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py -c "import ipykernel"
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_launcher_daemon -v
Started kernel Python 3
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_launcher_daemon -v
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_launcher_daemon -v
> D:\Users\claforte\Anaconda3\envs\fastai\python.exe c:\Users\claforte\.vscode-insiders\extensions\ms-toolsai.jupyter-2020.12.414227025\pythonFiles\pyvsc-run-isolated.py vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_launcher_daemon -v

XXX

Hi @claforte, does toggling the following VS Code setting stop the line ending conversions? You can get to this UI with Ctrl+Shift+P to open the Command Palette, then type 'Open Settings (UI)'.

Hi @claforte, does toggling the following VS Code setting stop the line ending conversions? You can get to this UI with Ctrl+Shift+P to open the Command Palette, then type 'Open Settings (UI)'.

Hi Joyce,
It was set to auto already. I'm not sure how you want me to toggle it, but I just tried setting it to \n and modified a few code cells. It seems some code cells get stuck to \r\n no matter what files.eol value I set, while other cells are stuck at \n.

Hi Joyce,
It was set to auto already. I'm not sure how you want me to toggle it, but I just tried setting it to \n and modified a few code cells. It seems some code cells get stuck to \r\n no matter what files.eol value I set, while other cells are stuck at \n.

If you open a freshly cloned ipynb after setting the files.eol setting to your preferred setting, do the endings still get converted?
The default line ending on Windows is '\r\n' (this is most likely what you are getting with the 'auto' setting), so I believe that is what VSCode is applying to your files. I don't believe changing that setting results in existing '\r\n' being converted to the new '\n', so unfortunately I think you'd need to do a find and replace on the files that have already been converted.

If you open a freshly cloned ipynb after setting the files.eol setting to your preferred setting, do the endings still get converted?
The default line ending on Windows is '\r\n' (this is most likely what you are getting with the 'auto' setting), so I believe that is what VSCode is applying to your files. I don't believe changing that setting results in existing '\r\n' being converted to the new '\n', so unfortunately I think you'd need to do a find and replace on the files that have already been converted.

Hi Joyce, I tried what you suggested (set vscode-insiders eol to \n, manually replaced all \r\n by \n in notepad++, restarted vscode-insiders, modified a cell, saved) and it looks like no new \r\n appeared.
Still it would be great if vscode-jupyter made sure it never added \r\n at the end of code line, since Jupyter notebook, AFAIK, never uses \r\n, and a lot of tools probably assume that.
Thanks,
Christian

Hi Joyce, I tried what you suggested (set vscode-insiders eol to \n, manually replaced all \r\n by \n in notepad++, restarted vscode-insiders, modified a cell, saved) and it looks like no new \r\n appeared.
Still it would be great if vscode-jupyter made sure it never added \r\n at the end of code line, since Jupyter notebook, AFAIK, never uses \r\n, and a lot of tools probably assume that.
Thanks,
Christian

Great to hear the setting change stops the conversions.
Thank you for highlighting this. From related issues like https://github.com/jupyter/nbconvert/issues/1062 it does seem that Jupyter notebooks tend to default to LF even on Windows. The files.eol setting is contributed by VS Code,  but perhaps one solution here would be for the Jupyter extension to override the default VS Code setting specifically for ipynb files. However, I don't believe there is currently a way to control files.eol based on file type, so this may require upstream changes in VS Code. Our team will discuss this at our weekly triage meeting.

Thanks a lot for your responsiveness Joyce, I appreciate it!

However, I don't believe there is currently a way to control files.eol based on file type, so this may require upstream changes in VS Code. Our team will discuss this at our weekly triage meeting.

The Jupyter extension is responsible for writing out the contents of the file, not the user. Hence we are in full control over whether we use \n or \r\n. I don't see any need for a VS Code update here.
@joyceerhl /cc

This is honestly very annoying behavior (I mean conversion of LF to CRLF). After such a notebook uploaded to Jupyter Lab server, I start to have incorrect character typed. This is another bug of Jupyter Lab, which unfortunately destroy your DEV experience. See here:
https://github.com/jupyterlab/jupyterlab/issues/2951
(bug reproduced in latest Jupyter Lab)
So basically after editing of a notebook in VSCode you have destroyed notebook in any external server!

So basically after editing of a notebook in VSCode you have destroyed notebook in any external server!
exactly true !!!

I have used several different jupyter notebook editors. When running "jupyter notebook" on both Windows and linux based systems, the notebook output consistently uses "\n" in the double-quoted lines of each cell -- not "\r\n". The only editor that I've seen output "\r\n" is  The VS Code jupyter notebook extension.  This inconsistency causes headaches when diff'ing notebook file versions stored in git repositories.
The issue is not the same as the common problem of Windows based systems insisting upon ending text file lines with the control characters \r\n and other systems just using \n. (git has features to help cope with this, which you are probably aware.) So, in my opinion, the "File:EOL" setting in VS Code should not be used to control the line ending characters of the notebook cell double-quoted lines.  I think the simple answer is to just follow the convention of "\n" to represent line end in the double-quoted lines.

The only editor that I've seen output \r\n is the VS Code Jupyter notebook extension. This inconsistency causes headaches when diff'ing notebook file versions stored in git repositories.

Is there any way that we could prioritize resolving this bug? Destroying the diff makes it impossible to sanely understand changes to notebook files during code review -- which will be even more important as we encourage folks to use version control for their .ipynb files.

Users can configure files.eol as a temporary workaround, and we can consider fixing this in the builtin ipynb extension /cc @roblourens @DonJayamanne

Sound like somethign for debt week.
Basically we need to remove all CR from the text we get in the code cells.

This is causing problems for many Jupyter users: https://github.com/jupyterlab/jupyterlab/issues/2951
Can you please fix this?

/verified
Tried with multiple notebooks. Existing and new. All only have LF in them:

The verification doesn't appear to be testing the end-of-line representation for each line of notebook cell.  You would need to make a notebook cell with at least two lines to see this. Attached is a small notebook file with two cells each with multiple lines. (File name extension changed to txt to allow upload.) This was created using Visual Studio Code 1.60.2 with the Jupyter extension v2021.8.2041215044.  Note that the representation of all but the final cell line of each cell ends with "\r\n". This illustrates the undesired behavior.
{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "This is an example notebook with a markdown cell and python code cell. Each cell\r\n",
    "has multiple lines to illustrate how end-of-line is represented in the `ipynb`\r\n",
    "file for each line in a cell."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "print( \"Aardvarks are first line.\" )\r\n",
    "print( \"Badgers are next in line.\" )"
   ],
   "outputs": [],
   "metadata": {}
  }
 ],
 ...
}

endofline_test ipynb.txt

@mburghart-qualitrol my screenshot was just to show no /r in the file happened to have open. It works with multiple lines in a cell too:

I many cases the files itself looks like one line, is there a way to tell VS Code to parse it correctly?

@RoyiAvital sorry not sure what you're saying. That the json for your ipynb file has no linefeeds in it?

@rchiodo , Indeed. Any way to enforce VS Code to parse in int a multi line manner?

Any way to enforce VS Code to save it a multi line manner?

You mean your code cells all have a single line in them? What does the ipynb look like?
Cells usually have a format like so:
{
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import os\n",
    "\n",
    "a,b,c = 'hihi', os, tf.__version__\n",
    "print(a, b, c)"
   ]
  },

And I believe you're saying your notebook looks like so?:
{    "cell_type": "code",   "execution_count": null,   "metadata": {},   "outputs": [],   "source": [    "import tensorflow as tf",  "import os",    "",   "a,b,c = 'hihi', os, tf.__version__",  "print(a, b, c)"   ]  },

Or does it have the \n in the source lines?

It is one liner like this:

While on VS Code it looks as expected.

@rchiodo , Can open and display it correctly.
Just when looking at the RAW file (Using text editors) it is one liner.
I wonder abut 2 things:

How did it happen? Why did VS Code saved it like that?
Is there a way to make it save again with all lines separated?

@RoyiAvital what is your version of VS Code and the jupyter extension. We might have at one time saved the file as a single line, but the code that saves a file now should write it out with linefeeds.

My version is VS Code 1.65.2 with Jupyter Extension 2022.2.103. Both are the latest stable versions.

I'm creating a new issue. This shouldn't be happening:
https://github.com/microsoft/vscode-jupyter/issues/9491


TypeScript type declarations for sevm module

For v0.5.3: Right now when I try to import in my typescript code, I get this error:

Seems related to typescript types not being included.
Not sure why, because I can see the *.d.ts files in the package.
I can still use the imported classes, and I can hide the error with // @ts-ignore but I don't get any type resolving on anything in the package.

Hi @shazow, this should be fixed in sevm 0.6.7. Essentially the issue was that the (legacy) package entry point was not defined, i.e., the main field was missing in package.json. See 90da9e3d4e99c11fff77e0388b5453a9216fbe5d where it has been fixed.
More specifically, even if the exports fields is defined in package.json, it is not considered when moduleResolution is set to classic. That's the case for files that are outside the TypeScript project's tsconfig.json. What happened is that your PR https://github.com/shazow/whatsabi/pull/62 tsconfig.json excludes the examples folder, thus using classic node resolution there and failing to resolve sevm properly without the legacy main field.
Hope it helps, let me know how it goes.

Oh wow thanks for debugging this, that must have been a pain!!


CURLOPT_RESOLVE

Q
A

Bug?
no

New Feature?
yes

Version
6.3.0

I would like to pass a RESOLVE option to the curl getopt function (https://curl.haxx.se/libcurl/c/CURLOPT_RESOLVE.html).
Should this be added in the request constant list, or is this something I could add to a "generic" opt list when constructing the client?
If you think this would be a nice addition to the library, I could help out to add it.

I think I should have a look at: https://github.com/guzzle/guzzle/blob/master/src/Handler/CurlFactory.php#L48-L50.

You can just pass this option to a request, as any other cURL-specific option:
$client->request('GET', '/', [
    'curl' => [
        CURLOPT_RESOLVE => 'www.google.com:443:<IP_ADDRESS>'
    ]
]);

Does it solve the problem, or you are talking about something different?

That was indeed the solution. Thanks!

Thanks @alexeyshockov! I didn't know this was possible and it took me a while to find this... it's not super clear in the docs and it took me many searches to turn this up.
I'd just like to add one little addendum - CURLOPT_RESOLVE requires an array of strings, i.e.:
$client->request('GET', '/', [
    'curl' => [
        CURLOPT_RESOLVE => [
            'www.google.com:443:<IP_ADDRESS>',
        ],
    ],
]);

@simonhamp, thanks for your comment!
About the docs: there is an FAQ section about custom cURL options. I think it's enough, but many people still ask questions on the topic. If you have any thought on how to improve the docs, feel free to comment/PR :)


tox won't pass

Describe the bug
I was trying to contribute a fix but:
$ pip install tox
$ tox

fail on master with:
ModuleNotFoundError: No module named 'pure_eval'

on Python 3.9.4.

I will have to figure out how to use tox and test it. I'm guessing that it
needs something else than requirements.txt to figure out dependencies.
On Sun, May 23, 2021 at 12:39 PM Julien Palard @.***>
wrote:

Describe the bug
I was trying to contribute a fix but:
$ pip install tox
$ tox
fail on master with:
ModuleNotFoundError: No module named 'pure_eval'
on Python 3.9.4.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/aroberge/friendly/issues/219, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEZXQSV6OQIVZGFCEWEYNDTPEOS3ANCNFSM45L2JKFA
.

Found the issue, it's simple:
in your setup.cfg you have version = attr: friendly.__version__, in this case setuptools has two ways to find your actual version:

It tries to parse the init.py file in hope to find the version value without importing it, if this fails:
It tries to import the file.

In friendly case as version is imported, and not physically in __init__.py the first try fail, and setuptools has to import your file. But importing your file mean importing all the file imports too.
To there's three ways to fix this:

Add pure_eval and many dependencies as build dependencies in pyproject.toml (bad bad bad)
Duplicate the version in version.py AND in init.py (bad bad)
Tell setup.cfg the version is in friendly.version.__version__ (I don't know why it does not work)
Move the actual version in the init.py

I'm working outside right now, enjoying the warm weather. I'll take care of
this when I finish some yard work. :-)
On Sun, May 23, 2021 at 1:40 PM Julien Palard @.***>
wrote:

Found the issue, it's simple:
in your setup.cfg you have version = attr: friendly.version, in this
case setuptools has two ways to find your actual version:

It tries to parse the init.py file in hope to find the version
value without importing it, if this fails:
It tries to import the file.

In friendly case as version is imported, and not physically in
init.py the first try fail, and setuptools has to import your file.
But importing your file mean importing all the file imports too.
To there's three ways to fix this:

Add pure_eval and many dependencies as build dependencies in
pyproject.toml (bad bad bad)
Duplicate the version in version.py AND in init.py (bad bad)
Tell setup.cfg the version is in friendly.version.version (I
don't know why it does not work)
Move the actual version in the init.py

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/aroberge/friendly/issues/219#issuecomment-846591149,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEZXQVVUIKR2TNYSEIOYX3TPEVWZANCNFSM45L2JKFA
.

I did move version inside init and everything works correctly locally but the git workflow fails as it apparently does not install pure_eval (and likely other requirements).  Leaving this open as I do not know currently how to fix it. I'll be focusing on other issues for now.

See https://github.com/aroberge/friendly/actions/runs/869377266 as an example.

I only kept Ubuntu since Windows was failing (and Mac OS didn't run) due to a tox-related error (not an error with my code). As I work on Windows, it gets thoroughly tested.

I only kept Ubuntu since Windows was failing (and Mac OS didn't run) due to a tox-related error (not an error with my code). As I work on Windows, it gets thoroughly tested.

According to https://github.com/aroberge/friendly/actions/runs/870518597 Windows and MacOS tests were successfull.
I think it's still good to test PRs on Windows, just to validate other contributors like me don't break something.

On Mon, May 24, 2021 at 11:57 AM Julien Palard @.***>
wrote:

I only kept Ubuntu since Windows was failing (and Mac OS didn't run) due
to a tox-related error (not an error with my code). As I work on Windows,
it gets thoroughly tested.
According to https://github.com/aroberge/friendly/actions/runs/870518597
Windows and MacOS tests were successfull.
I think it's still good to test PRs on Windows, just to validate other
contributors like me don't break something.

Yes, probably ... except that now it is failing on Ubuntu. I tried to add
some modules that should be excluded from coverage and have not been able to
fix it.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/aroberge/friendly/issues/219#issuecomment-847099027,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEZXQT366WIX7YSCL553DDTPJSPNANCNFSM45L2JKFA
.

Let me check this :)

@aroberge found the issue, coverage was configured both in tox.ini and .coveragerc. .coveragerc was winning, but missing a line.
I deduplicated the config in https://github.com/aroberge/friendly/pull/227 and test passes for Windows, MacOS, and Linux.
I choose to put coverage config in tox.ini to have one less file, but don't hesitate to move the configuration the other way around if you prefer, just don't duplicate it.

For information: putting the "omit" information in tox.ini vs having that information in .coveragerc instead is yielding a different set of files being included in coverage: using tox.ini leaves out files having 100% coverage omitted from the report (and reducing the overall reported coverage).

Using tox.ini, the reported coverage is 84%:

Module | statements | missing | excluded | branches | partial | coverage
-- | -- | -- | -- | -- | -- | --
Total | 5083 | 676 | 416 | 1875 | 275 | 84%
friendly\__init__.py | 68 | 6 | 6 | 12 | 3 | 89%
friendly\__main__.py | 79 | 19 | 0 | 22 | 9 | 70%
friendly\config.py | 99 | 2 | 48 | 24 | 1 | 98%
friendly\core.py | 359 | 19 | 57 | 134 | 18 | 92%
friendly\editors_helpers.py | 68 | 8 | 0 | 16 | 0 | 88%
friendly\info_generic.py | 121 | 8 | 0 | 8 | 1 | 93%
friendly\info_specific.py | 88 | 9 | 0 | 8 | 2 | 89%
friendly\info_variables.py | 232 | 22 | 0 | 115 | 14 | 89%
friendly\my_gettext.py | 32 | 3 | 0 | 2 | 1 | 88%
friendly\path_info.py | 70 | 17 | 0 | 32 | 10 | 70%
...

Note how console_helpers.py,  debug_helper.pyandformatter.py` are not included above.

Using .coveragerc, the reported coverage is 87%.

Module | statements | missing | excluded | branches | partial | coverage
-- | -- | -- | -- | -- | -- | --
Total | 5085 | 677 | 413 | 1875 | 275 | 84%
friendly\__init__.py | 68 | 6 | 6 | 12 | 3 | 89%
friendly\__main__.py | 79 | 19 | 0 | 22 | 9 | 70%
friendly\config.py | 99 | 2 | 48 | 24 | 1 | 98%
friendly\console_helpers.py | 65 | 0 | 104 | 12 | 0 | 100%
friendly\core.py | 359 | 19 | 57 | 134 | 18 | 92%
friendly\debug_helper.py | 8 | 0 | 11 | 0 | 0 | 100%
friendly\editors_helpers.py | 68 | 8 | 0 | 16 | 0 | 88%
friendly\formatters.py | 43 | 0 | 111 | 16 | 0 | 100%
friendly\info_generic.py | 121 | 8 | 0 | 8 | 1 | 93%
friendly\info_specific.py | 88 | 9 | 0 | 8 | 2 | 89%
friendly\info_variables.py | 232 | 22 | 0 | 115 | 14 | 89%
friendly\my_gettext.py | 32 | 3 | 0 | 2 | 1 | 88%
friendly\path_info.py | 70 | 17 | 0 | 32 | 10 | 70%
...

using tox.ini leaves out files having 100% coverage omitted from the report.

TL;DR: Either put the whole coverage config in tox.ini, either in .coveragerc, but don't mix both.
Got it! If there's a .coveragerc, coverage use it, and won't try to search another config file, missing the [coverage:report] skip_covered = True from the tox.ini.
If there's no .coveragerc, coverage will search for the config elsewhere, and find it in tox.ini (no need to tell it).
I double checked by copying both sections to .coveragerc and then to tox.ini (beware of adding/removing the coverage: prefix, and beware of removing the empty .coveragerc during the tox.ini test.
I've noticed no difference between the two runs, tried using:
coverage run -m pytest; coverage combine; coverage report > a
# moved the conf
coverage run -m pytest; coverage combine; coverage report > b
dif a b

Tox just run the commands, there's no magic conf editing or nothing, but beware: there's a slight difference running coverage manually with a single Python version and running it via my tox config because in my tox config I use multiple Python versions and then merge the reports, see:
via tox:
TOTAL                                                6938    756   2149    415    85%

manually using python 3.9:
TOTAL                                                6938    822   2149    438    84%

This way if you have:
if sys.version > (3, 5):
    ...
else:
    ...

both the if and the else will be merged during the coverage combine step, which is nice.

Thanks, I'll try removing the [coverage:report] section.
Yesterday, I did remove the [coverage:run] section entirely from tox.ini and, for a while, all the tests WERE passing.  Then, last night, one windows-latest test failed again.... But I still have the [coverage:report] section in tox.ini as you point out.
Instead of using tox to combine reports, this is what I do in a .bat file:
call venv-friendly3.6\scripts\activate & python -m pytest --cov=friendly --cov-report html tests/
call venv-friendly3.7\scripts\activate & python -m pytest --cov=friendly --cov-append --cov-report html tests/
call venv-friendly3.8\scripts\activate & python -m pytest --cov=friendly --cov-append --cov-report html tests/
call venv-friendly3.9\scripts\activate & python -m pytest --cov=friendly --cov-append --cov-report html tests/
call venv-friendly3.10\scripts\activate & python -m pytest --cov=friendly --cov-append --cov-report html tests/

Running full tests with coverage can be too long. I have 6 virtual environments (one note tested automatically, to be used with IPython/Jupyter) and can activate any one with a very simple call (e.g. ae 3.10). Then, I can:

run pytest only; my default when I am working on adding a new case.
create the entire tracebacks for this specific Python version to be put in the docs, and see if anything has changed; in the past, I caught some subtle bugs that were not captured by the (incomplete) unit tests -- including changes/missing translations for the French version, not directly tested by unit tests
run pytest (no coverage) with all Python versions
run pytest with coverage (as above)
update the documentation for all Python versions

To reduce the test time I run tests in parallel, but pytest-cov have hard times with parallel runs, so I avoid using it.
On friendly, on my machine, tox -p all takes 22s, (as a reference, tox -e py39 takes 9s, pytest takes 6s).


[WFLY-20177] Use ModuleDependency.Builder instead of deprecated ModuleDependency ctor in UNDERTOW

https://issues.redhat.com/browse/WFLY-20177

Hi @ropalka Please reopen once nearly all the 35.x PRs are processed.


Raspberry Pi 3 - Device Name

Hello,
I have an issue where the device name is not being sent.  It shows up blank unless I actually connect to the device then it will show up.  I am not needing to set the environment BLENO_DEVICE_NAME as I am using the hostname.
I am also using this to startAdvertising.
bleno.startAdvertising(os.hostname(), [sd.app.config.ble.uuid], (error: any) => {
});

Hi @Uniphix
I have tested your code and its working fine on Ubuntu. Have you checked the output of os.hostname() ?

@venkatraopasupuleti This works and the output of os.hostname() has always returned for me.  This is not working for RASPBIAN STRETCH LITE with the most updated kernel.  It always happens to cause problems with the name until I connect to it via bluetooth, and once connected and I disconnect it shows up.  If I turn off our mobile application clear etc. all of it.   Reboot the Raspberry Pi and start it up again.  The name shows up just fine.  It's always the very first time for a specific mobile device.

Hi @Uniphix
I have tested it on RPI 3 B, RASPBIAN STRETCH LITE, kernel version 4.9.59. The bluetooth name appears as given in the code(os.hostname()). I have tested by turning off and on still no issue in broadcast of the name.

Closing for now due to lack of feedback, and @venkatraopasupuleti said things are working on their setup.


CI failing continuously with 404 error

The CI workflow has been failing continuously since https://github.com/nextstrain/status/actions/runs/8444313476 with a 404 error
[...]
Waiting for Steampipe plugin schemas to be ready…
Running psql
ERROR:  rpc error: code = Unknown desc = github: GET https://api.github.com/repos/nextstrain/hbv/contents/dynamic/pages/pages-build-deployment?ref=workflows: 404 No commit found for the ref workflows []
Stopping Steampipe Postgres service…
Steampipe database service stopped.
make: *** [Makefile:13: build/pathogen-workflows.json] Error 3
make: *** Deleting file 'build/pathogen-workflows.json'

I've manually disabled the CI workflow for now.

Something weird was going on with the hbv workflows
select *
    from
        github_workflow
    where
        repository_full_name = 'nextstrain/hbv';
ERROR:  rpc error: code = Unknown desc = github: table 'github_workflow' column 'pipeline' requires hydrate data from GitHubWorkflowFileContent, which failed with error GET https://api.github.com/repos/nextstrain/hbv/contents/dynamic/pages/pages-build-deployment?ref=workflows: 404 No commit found for the ref workflows [].

This made me dig into the individual workflows without accessing the pipeline column:
select
        repository_full_name,id,path,name,html_url
    from
        github_workflow
    where
        repository_full_name = 'nextstrain/hbv';

nextstrain/hbv|88698912|dynamic/pages/pages-build-deployment|pages-build-deployment|https://github.com/nextstrain/hbv/actions/workflows/pages/pages-build-deployment
nextstrain/hbv|67029840|.github/workflows/run-builds.yaml|Run the builds|https://github.com/nextstrain/hbv/blob/master/.github/workflows/run-builds.yaml

I realized that the original error came from the pages-build-deployment workflow with path dynamic/pages/pages-build-deployment, which is automatically generated by GitHub to deploy GH pages from a defined publishing source.
Not sure why this just started causing a problem since the hbv workflow was triggered 3 weeks ago. Maybe dynamic workflows just started getting returned in the API queries for GH workflows?
I will update the query to only include our own workflows with path like .github/workflows/*.


[Feature Request]: Implement lazy loading throughout the website

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Learn about lazy loading in NextJS and implement in the whole website to reduce the bundle size and create different bundles for different components for fast loading and reduce performance issues.
Use Case
Performance enhancement
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[ ] I'm a GSSOC'24 contributor
[X] I have starred the repository

Thank you for creating this issue! 🎉 We'll look into it as soon as possible. In the meantime, please make sure to provide all the necessary details and context. If you have any questions or additional information, feel free to add them here. Your contributions are highly appreciated! 😊
You can also check our CONTRIBUTING.md for guidelines on contributing to this project.

This issue has been automatically closed because it has been inactive for more than 30 days. If you believe this is still relevant, feel free to reopen it or create a new one. Thank you!

Hello @ashutosh-rath02! Your issue #362 has been closed. Thank you for your contribution!


ctapipe-extra on detached branch

when following  installation procedure, submodule ctapipe-extra branch is detached:
(cta_test)>git branch
* (HEAD detached at 536490d)
  master

I need to execute a git checkout it's master branch before running tests to obtain all datafiles.
If I did not make any mistake, we need to update documentation.

Or, rather than change documentation, we can change init target of main Makefile to add git checkout master command

I guess with ctapipe-extra being a module we can close this now right?

I agree


build(docs): set site_url from ReadTheDocs env var

Motivation

site_url is required for proper styling on 404s, canonicalization, and creating a sitemap.xml

Also for certain features like:

Instant loading
Versioning (we use RTD's versioning, but may want some features from here like the version drop-down)

Modifications

Add site_url config to mkdocs.yml

use RTD env var

Verification

mkdocs serve
Go to http://<IP_ADDRESS>:8000/en/latest/ now -- works
Go to http://<IP_ADDRESS>:8000/en/latest/sitemap.xml -- actually has content now
Go to http://<IP_ADDRESS>:8000/en/latest/wrong -- has a properly styled 404 page. Screenshot:

site_url is required for canonicalization and creating a sitemap.xml
Also for certain features like:

Ah I think this was also required for to make 404 links (like https://argo-workflows.readthedocs.io/en/release-3.4/wrong) render CSS properly, and that was the original purpose of this, but I've forgotten what links I followed. https://github.com/mkdocs/mkdocs/issues/2318 is related

Ah I think site_url was also required for to make 404 links (like https://argo-workflows.readthedocs.io/en/release-3.4/wrong) render CSS properly, and that was the original purpose of this PR, but I've forgotten what links I followed (went down a rabbit hole at the time).

Found it again: https://docs.readthedocs.io/en/stable/reference/404-not-found.html, under the MkDocs tab:

MkDocs automatically generates a 404.html which Read the Docs will use. However, assets will not be loaded correctly unless you define the site_url configuration value as your site’s canonical base URL.


Easier access to Release Notes and Github Repo

Is your feature request related to a problem? Please describe.
I already an App (My Portfolio) in Blazor with Core 3 Preview, It's few months old. So today I was looking to upgrade it and find the new version of Blazor and check the Release Notes also maybe Upgrade guide. But on main Blazor Page there is no mention of current version of link to Release Notes.
Video Recording on trying to find Release Notes:

Describe the solution you'd like
A clear Release Notes link.
Describe alternatives you've considered
A clear Current Version link that takes to current version release notes. In addition to it a main clear Github Repo button for every related product would also be really nice.
Additional context
An example of such links/buttons.

@rowanmiller @danroth27 can you please take a look at this feedback about finding the Blazor release notes?

Blazor is considered part of ASP.NET Core, so the Blazor release notes with preview to preview migration steps are included with the ASP.NET Core 3.0 preview announcement blog posts. We don't currently have separate release notes from these blog posts, which is admittedly problematic, as they are difficult to reference in a formal way.
@glennc @Pilchie

There is a link to the Blazor source code on https://blazor.net, although it is not part of the header:

@danroth27 I personally really like the fact Blazor is part of ASP.NET Core, But for a person who is coming back to dotnet after a while (like me) or someone who is coming from somewhere else, fresh wanting to try out some new goodies.
I think a clear reference to these things for individual products would be really helpful and is somewhat expected, being the norm everywhere else (somewhat 😃 ).
 Still thanks for let me know I'll check them out.


Fix Hulkenberg first and last name in fastf1._DRIVER_TEAM_MAPPING

Fix for #225

Looks good, I'll go ahead and merge this. Thank you for contributing to FastF1 and for correcting this embarrassing little mistake!


Also run coverage tests.

In preparation to send it to codecov
Signed-off-by: Henner Zeller<EMAIL_ADDRESS>

I think this should be ready, but codecov does not show anything yet https://app.codecov.io/gh/chipsalliance/verible
From the confusing error messages it sounds like that it doesn't have a 'base report', which would be plausible as this is the first push, but other than that the CI output does not show any error. So I suggest that we submit this and then see if there are corrections needed later.

It shouldn't need a token, but I'll try that now as well


error: Build input file cannot be found: Bulk_UnityEngine.UnityConnectModule_0.cpp

Hi,
I am trying to build this demo on my mac which has unity 2018.2.11f1 and xcode version 10.0. I am getting the following error:
error: Build input file cannot be found: '/Users/myUser/Desktop/react-native-unity-demo-master/ios/UnityExport/Classes/Native/Bulk_UnityEngine.UnityConnectModule_0.cpp'
Any ideas?

@omer-halici did you fixed it ?

@hungdt-ibl  did you fixed the issue?. I am facing the same issue. any help highly appreciated.


Requests Are Ignored

Hi,
Probably not the best place to ask this, but here it goes anyway...
I have a working installation of Docker, which I am trying to control with Docker.DotNet. The problem is, try as I might, I can't get any of the calls to return anything, but I also don't get any exceptions!
My basic code is:
var client = new DockerClientConfiguration(new Uri("tcp://<IP_ADDRESS>:2376")).CreateClient();
var containers = await client.Containers.ListContainersAsync(new ListContainersParameters());
I am monitoring the docker.log file, and I can see that no request gets there. Using the command line, everything works perfectly.
What can I be doing wrong?
Thanks!
RP

Tried with and without X509 authentication, still nothing... The certificate is loaded, and the password is correct.

Can you try using http instead of tcp?

But for that I need to change the VM settings... Any other way I can debug Docker.DotNet? I try to pass even an invalid URL, but it always fails silently.

Umm not really. tcp:// is what docker understands and it actually still uses http:// under the hoods. This library only uses http:// which should just work with your Docker engine configured to run on tcp://ip:port.

It's the same thing, with HTTP, HTTPS or TCP. The docker.log file does not show any requests being received, whereas it does when I use the docker command line tool.

I am using Visual Studio 2015, .NET 4.5.2, and I have the typical Docker environment variables set, which, for the command line, work perfectly.

DOCKER_CERT_PATH=C:\Users\rperes.docker\machine\machines\default
DOCKER_HOST=tcp://<IP_ADDRESS>:2376
DOCKER_MACHINE_NAME=default
DOCKER_TLS_VERIFY=1

Well, your environment variables are not being used at all, in this case. This is the first time I'm hearing such a problem. I suggest checking out the source code and building symbols of this, this may give you a chance to debug by stepping into library code.
Can you also paste the code that you used to initialize the client and made the API call?

Sure:
var credentials = new CertificateCredentials(new X509Certificate2(Path.Combine(Environment.CurrentDirectory, "key.pfx"), "tcuser"));
var client = new DockerClientConfiguration(new Uri("tcp://<IP_ADDRESS>:2376"), credentials).CreateClient();
As you can see, I added the key.pfx file to the project, with "copy if newer" action, and used the same password as for the Docker VM. Password is ok, if I use an invalid one, I get an exception.
Then I'm doing something like:
var containers = await client.Containers.ListContainersAsync(new ListContainersParameters());

BTW, I tried with both the DLL from NuGet and from a locally built one.

Got it! :-D

@rjperes ? Is it working now?

 :-)
Apparently, I was returning from the method without giving it time to actually perform the asynchronous operation! By using a ManualResetEvent I solved it, but I'm not sure it's the best way.
Do you understand my problem?

Yeah I'm not really a System.Threading expert nowadays.. :) Glad it worked.

Thanks, anyway!

Sorry, just one more question: is there an API for the Docker run command?

@rjperes docker run is not a single command. Please read this entire page if you're planning to use this library. http://docs.docker.com/reference/api/docker_remote_api_v1.20/#3-1-inside-docker-run


added a day only option in calendar method

allow moment().calendar( withoutTime ) method to accept a boolean parameter as an option to display date strings format without the time.
eg:
now = feb 7
moment().calendar(); // Today at 10:59pm
moment().calendar(true); // Today

Are you in need of both with the time strings and without the time strings? The only reason I ask is because this will need to be added to all the translations, and I'm wondering if it's worth the filesize increase on the lang files.
If you dont need both at the same time, you can just modify the strings directly or create a modified lang file for it.
moment.calendar = {
    sameDay : '[Today]',
    nextDay : '[Tomorrow]',
    nextWeek : 'dddd',
    lastDay : '[Yesterday]',
    lastWeek : '[last] dddd',
    sameElse : 'L'
};

Thanks for the response. I understand the concern about the filesize and I agree that it might not be worth it. But apparently we need both formats and we'll be needing other string format  as well. Maybe we could pass an object that will contain the custom formats in calendar instead?
eg:
var cusFormats = {
    sameDay : '[Today]',
    nextDay : '[Tomorrow]'
};

moment().calendar(cusFormats)

ill send another pull request for this. Cheers.

You could just create a plugin for moment.js that does this. Here's an example I whipped up that only works in the browser.
https://gist.github.com/1771023


pdfjs-dist gives different results than demo

Hi, I have a custom app for viewing and working with PDF files. So far I had no problems except this file. If I open it in Firefox or in demo app it is rendered correctly. However if I render it in my app it seems like some texts are missing. I noticed that the version used in demo app is 2.0.757, but I can not find the build anywhere. This incorrect behavior is platform and browser independent (happens in Chrome, Windows and Safari, iOS).
Problem PDF:
Vánoční turnaj Unichess_2017.pdf
PDFJS demo viewer result:

Result from my application:

I did build of current master and it works perfectly.


Migrate autofill to new feature flag framework

Task/Issue URL: https://app.asana.com/0/488551667048375/1203928902316152/f
Description
Migrate autofill to new FFF
Steps to test this PR

[ ] install from this branch and smoke test autofill
[ ] verify the autofill.db file is created and contains the exceptions
[ ] comment out the entire AutofillFeatureExceptionStore.kt file
[ ] comment out the exceptionsStore = AutofillFeatureExceptionStore::class in UnusedAutofillRemoteFeatureCodegenTrigger.kt
[ ] build, install and launch the app
[ ] verify the autofill.db file is NOT created

Current dependencies on/for this PR:

develop

PR #2818

PR #2830   👈

This comment was auto-generated by Graphite.


Switch terminology from "OEM Repo" to "Image Repo".  Fix issue #10

Rename add_target_to_oemrepo() to add_target_to_imagerepo()

Rename demo_oem_repo.py to demo_image_repo.py
Rename add_target_to_oemrepo() to add_target_to_imagerepo()
Update modified names in README.md

I am moving this pull request from awwad/uptane

I am moving this pull request from awwad/uptane

Merged, along with a few other README tweaks to clarify repo names. Thanks (:


Remove std feaure.

Closes #1.

Note that this doesn't have to be a breaking change, you can leave in the feature.

Actually that's a great idea, I'll just leave the feature and ignore it.


Fixed issue 36:  Illegal argument: character must be non-negative

If the W3C violation occurs in the first character of the file, e.g. if the file references a legacy doctype, message.hiliteStart will be 0, and message.hiliteStart-1 will be negative, causing the error Illegal argument: character must be non-negative. This was a quick fix: instead of passing message.hiliteStart-1 directly as a parameter of vscode.Position, pass Math.max(message.hiliteStart-1,0), which forces the character to be nonnegative.

Thanks @rupumped, I'll deploy the fix as soon as possible.


fix #22 removre package-lock from git

fix #22 removre package-lock from git

no ! package-lock must be in git


refactor(util): remove unused variables in src/util/

Description:

This removes needless type parameters & declarations.
Just a minor refactoring.

Related issue (if exists):

I found this refactoring chance in playing with codes for #1969

Coverage remained the same at 97.217% when pulling 6f37e34163023c1928cf91bc26d844be0d60c937 on saneyuki:rm-needless-in-utils into 260d335f497a2de223b164ea9705699e7f306d7d on ReactiveX:master.

Coverage remained the same at 97.217% when pulling bb4dfcf5c4c1eb3b665e28341f2c6bc5b23fe12b on saneyuki:rm-needless-in-utils into 0271fab2f435d21fc64187a39143237208a54163 on ReactiveX:master.

Coverage remained the same at 97.217% when pulling 53de9f88336a93f623bf790af9669417af34aaaf on saneyuki:rm-needless-in-utils into 89612b2914fe0f48123ceaa66e3e63b5f7975873 on ReactiveX:master.

Coverage remained the same at 97.217% when pulling 53de9f88336a93f623bf790af9669417af34aaaf on saneyuki:rm-needless-in-utils into 89612b2914fe0f48123ceaa66e3e63b5f7975873 on ReactiveX:master.

LGTM


Autocomplete: get property from object

Following the material2 documentation for autocomplete, I can not make the control value a property of the object.
In the example, I need the value of the control to be the id of the user, not the complete object, and also display the user's name in the input
http://plnkr.co/edit/ShFxkogDjg45S8jQm8pt?p=preview
Is it possible to do this?

I feel like this is outside the scope of material.
As your plunker stands, your displayWith function is called with the selected option's id. Just use that id to find the correct option from the options array and return its name.

You're right, I did not realize I could do that :S
The only change is that in order to access the controller variables, I had to modify displayWith as follows:
<md-autocomplete #auto="mdAutocomplete" [displayWith]="displayFn.bind(this)">
Thank you.


Dockerfiles fix

Fixed dockerfiles for ubuntu, debian and fedora.
Moved all the deps inside distro-deps.sh
Removed sudo from original installation scripts

git is the only package required in the dockerfiles (in order to fetch the repo).
This PR includes https://github.com/binpash/pash/pull/439 and should fix https://github.com/binpash/pash/issues/438
Signed-off-by: dkarnikis

So all the tests have passed. I also deployed a fresh installation of pash on an ubuntu 20.04 and had no issue as well (all the tests passed too). @angelhof should we merge?

Great job! Yes, we should merge :)


Provide basic error handling and user notifications

Taken from description of exercise 3:

The web application is considered to be finished if it supports at least four distinct use cases which are specific to your business model. Furthermore, in addition to those use cases, your web application has to:

provide basic error handling and user notifications

Currently we do not notify the user in an appropriate way. Sometimes we just log the error or call alert() - this is not user-friendly and not informative. I suggest that we build upon the error component that was introduced by @HmG248 a new more generic component that is called when one of the service calls fails. Here is an example:
Component: ClassListView
Method: componentWillMount
Service call: UserService
ClassService.getClassesOfUser().then((data) => {
            this.setState({
                classes: [...data],
                loading: false
            });
        }).catch((e) => {
            console.error(e);
        });

Suggestion:

Add error to state:

error : {
                fired: false,
                errorCode: 0,
                errorMsg: ''
            },

If error occured, setState in catch-block

ClassService.getClassesOfUser().then((data) => {
            this.setState({
                classes: [...data],
                loading: false
            });
        }).catch((e) => {
            this.setState({error: {fired: true, errorCode: e.Code, errorMsg: e.Msg}});
        });

Check the error state make the generic ErrorComponent visible:

<ErrorComponent
                    visible={this.state.error.fired}
                    error={this.state.error}
                    closeDialog={()=>{this.setState(
                        {error:
                                {
                                    fired:false,
                                    errorCode:0,
                                    errorMsg:''
                                }
                        })
                    }}
                />

The error component can use the error object to display an informative message to the user:

I made some checks and this approach is even working within modal dialogs:

I like it! Will implement the error component for login and register! :-)

@andrelandgraf You actually don't have to - I create a generic Exception component which will be invoked from App.js so you basically call this.props.handleException(error) - that's it 😃

So this is working as a snackbar now. Please have a look at the ExceptionComponent to see how you call it


Support fullscreen command

https://i3wm.org/docs/userguide.html#_toggling_fullscreen_mode_for_a_window
Status: use cases collection. Please post your use cases

I use it a lot to focus on a single app in fullscreen, do stuff, and then go back to my tilings. An alternative is stacking, but then you have to find the window you want in all the stacked windows; so it's easier to go fullscreen on the window having the focus.

I use fullscreen a ton. It's often easier to use fullscreen than to changed to stacked/tabbed modes or move the app to an empty workspace.
To me this feature is high priority, because it's an important part of my workflow.

Fixed in 0.3.0-Beta


When running "trunk serve" getting error wasm-bindgen version, but only inside of workspace

I am running a Rust Leptos project that uses a CSR build (wasm on client). When this project is run standalone by running trunk serve it works without issue. However when I move it into a rust workspace it gives this error below. I tried installing the wasm-bindgen-cli 0.2.87 but still get same error.
Is there a way to config the project crate such that it can run trunk without this issue?
it looks like the Rust project used to create this wasm file was linked against
version of wasm-bindgen that uses a different bindgen format than this binary:
rust wasm file schema version: 0.2.87
this binary schema version: 0.2.86 (2d882c97b)
Currently the bindgen format is unstable enough that these two schema versions
must exactly match. You can accomplish this by either updating this binary or
the wasm-bindgen dependency in the Rust project.
You should be able to update the wasm-bindgen dependency with:
cargo update -p wasm-bindgen --precise 0.2.86 (2d882c97b)

don't forget to recompile your wasm file! Alternatively, you can update the
binary with:
cargo install -f wasm-bindgen-cli --version 0.2.87

I just tried doing a cargo build and that works. So it appears that this is an issue where trunk insists on using wasm-bindgen 0.2.86

I deleted at cargo.lock files and did a cargo clean, which resolved the issue.

I've been fighting this issue too - can confirm deleting Cargo.lock fixes it!


Add ability to specify mamba dependencies

Pull Request Description
What issue does this change request address?
Closes  #2236
What are the significant changes in functionality due to this change request?
This adds a new --subset mamba for things that should be installed when mamba is installed.
This also uses this to switch to fmt version 10.1 in windows, because version 10.2.0 is broken in windows.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[ ] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[ ] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[ ] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[ ] 8. If an analytic test is changed/added is the the analytic documentation updated/added?
[ ] 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

Job Test qsubs sawtooth on 1e6aa2e : invalidated by @joshua-cogliati-inl
Failed with: DirectoryNotACondaEnvironmentError: The target directory exists, but it is not a conda environment.

LGTM. It looks like this will not require any changes to our testing scripts or machine environments, correct?

LGTM. It looks like this will not require any changes to our testing scripts or machine environments, correct?

No. This basically reverts fmt to an older version and otherwise should run in the same environment.


Unable to find subscription from the management subscription drop down

Hi there,
We have about 200 subscriptions in our environment when we try to deploy hub and spoke template we unable to see desired subscription from the drop down list in both the places (management subscription and landing zone) we are trying with account which got Ower access on the root

I think we are seeing this issue only after giving access on root ( / ).. also we have lot of " access to Azure Active Directory" subscriptions. if we try with the user ID ,doesn't have root access, seems to list all the subscriptions just fine..

Thanks for reaching out. The field should also allow you to filter, so you can start typing the displayName of the subscription. Is this not showing the subscription(s) you are looking for?

no field doesnt list all of my subscriptions however if I go to the portal -> subscriptions I am see all of my subscriptions there but not using this deployment template

Thanks - we are investigating this

Quick update; the issue is related to server-side pagination, and we are exploring how to fix it. If this is blocking you right now, feel free to send me an email so I can provide a workaround so you can get it deployed.


Cannot build project with 'panic = abort'

Seems like a bug in #2687
alexbool@alexbool-osx ~/D/I/nounwind> cat Cargo.toml
[package]
name = "nounwind"
version = "0.1.0"
authors = ["Alexander Bulaev<EMAIL_ADDRESS>
[dependencies]
kernel32-sys = "*"

[profile.release]
panic = "abort"

alexbool@alexbool-osx ~/D/I/nounwind> cat src/main.rs
extern crate kernel32;

fn main() {
}

alexbool@alexbool-osx ~/D/I/nounwind> cargo build --release
   Compiling kernel32-sys v0.2.2
error: the crate `build` is compiled with the panic strategy `abort` which is incompatible with this crate's strategy of `unwind`
error: aborting due to previous error
error: Could not compile `kernel32-sys`.

To learn more, run the command again with --verbose.

Meta:
alexbool@alexbool-osx ~/D/I/nounwind> rustc -vV
rustc 1.10.0-nightly (476fe6eef 2016-05-21)
binary: rustc
commit-hash: 476fe6eefe17db91ff7a60aab34aa67a0a750a18
commit-date: 2016-05-21
host: x86_64-apple-darwin
release: 1.10.0-nightly

alexbool@alexbool-osx ~/D/I/nounwind> cargo -vV
cargo 0.11.0-nightly (259324c 2016-05-20)

If I add
[profile.dev]
panic = "abort"

to Cargo.toml, cargo build --release works fine

Whoa now that's a weird bug! I believe I've fixed in https://github.com/rust-lang/cargo/pull/2730 though


Duplicated call to /v1/steps endpoint

Describe the bug
When opening the step catalog, there is a duplicated call to the /v1/steps endpoint.
To Reproduce
Steps to reproduce the behavior:

Go to kaoto
Open Developer Tools
Click on step catalog
See the network tab

Expected behavior
I guess only one call should be needed. Both have the same parameters.


Unexpected result of model_eval (inconsistent with keras implementation)

What I noticed is that cleverhans.utils_tf.model_eval function gives unexpected result,
i.e. result is not consistent with result of keras.models.Model.evaluate.
Here is my code so that result can be reproduced:
import tensorflow as tf
from keras.models import load_model
from cleverhans.utils_keras import KerasModelWrapper
from cleverhans.utils_tf import model_eval
from cleverhans.utils_mnist import data_mnist
from keras.optimizers import SGD
from keras.layers import Dense, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.models import Sequential

num_classes = 10
batch_size = 128
nb_epochs = 2 # to reproduce results faster
img_size = 28
nb_channels = 1

def main():
    # Create TF session
    sess = tf.Session()

    # load data
    x_train, y_train, x_test, y_test = data_mnist()

    # load model
    # model_path = 'same result if we load a model instead of training it'
    # model = load_model(model_path)
    # print("Model loaded.")

    # train a model
    model = get_keras_model(num_classes, (img_size, img_size, nb_channels))
    sgd = SGD(lr=0.1, momentum=0.9, decay=0.5)
    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
    model.fit(x_train, y_train,
              batch_size=batch_size,
              epochs=nb_epochs,
              validation_data=(x_test, y_test),
              shuffle=True)

    # Score trained model via keras impl of evaluate method
    scores = model.evaluate(x_test, y_test, verbose=1)
    print('Test accuracy:', scores['acc'])

    # initialize placeholders
    x = tf.placeholder(tf.float32, shape=(None, img_size, img_size, nb_channels))
    y = tf.placeholder(tf.float32, shape=(None, num_classes))

    # wrap it
    model = KerasModelWrapper(model)

    # initialize session
    init_op = tf.global_variables_initializer()
    sess.run(init_op)

    # Score trained model (again) vie keras implementation of evaluate
    scores = model.model.evaluate(x_test, y_test, verbose=1)
    print('Test accuracy (again):', scores[1])

    # evaluate wrapper via model_eval
    predictions = model.get_logits(x)
    accuracy = model_eval(sess, x, y, predictions, x_test, y_test, args={'batch_size': batch_size})
    print('Test accuracy (cleverhans impl): ' + str(accuracy))

def get_keras_model(num_classes, input_shape):
    model = Sequential()
    model.add(Conv2D(64, kernel_size=(3, 3), strides=(1, 1),
                     activation='relu',
                     input_shape=input_shape))
    model.add(Conv2D(64, kernel_size=(3, 3), strides=(1, 1),
                     activation='relu',
                     input_shape=input_shape))
    model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Flatten())
    model.add(Dense(256, activation='relu'))
    model.add(Dense(256, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))
    return model

if __name__ == '__main__':
    main()

and output (unimportant messages are omitted) it produces is the following:
Test accuracy: 0.9633

Test accuracy (again): 0.9633

Test accuracy (cleverhans impl): 0.1708

How is it the case that those accuracies are not the same?

It's likely to be a bug in KerasModelWrapper. Have you tried running the keras Sequential model on one batch and checking that its output is the same as the output of KerasModelWrapper?
Is there any chance that Keras has made its own session in the backend? CleverHans might be reading the values of the model's Variables out of one Session while Keras reads them out of another.

What version of keras are you running? I'm using keras 2.2.2 and when I run your code I get an exception when trying to access scores['loss']. For me, scores is a list.


xrdp-sesman reports wrong policy subject to agent when user is ldap

The bug has been reported here as well: https://bugs.centos.org/view.php?id=14485
polkit 0.112-12.el7_3 fails to register any agent when machine is puppeted and user is retrieved via ldap.Example agent (xfce but kde and mate fail the same)/usr/libexec/xfce-polkit ** (xfce-polkit:15429): CRITICAL **: polkit_agent_listener_register_with_options: assertion 'POLKIT_IS_SUBJECT (subject)' failed(xfce-polkit:15429): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
I have no idea why this could be failing. It looks like get peername correctly assumes a socket but gets a file.
It can always be reproduced. However I do not exactly know how puppet and ldap works, so I cannot give instructions how to reproduce outside of my company.However I will supply any additional information that might help.
I did the following troubleshooting:
Checked with other desktops, polkit agents always fail.
User is not in /etc/passwd, nor are the groups.
Access is via xrdp, no display manager, it isn't even installed.
pamtester authorises fine against sshd, xrdp-sesman and polkit-1
Local installed user that is not retrieved via ldap fails as well, with exactly the same error. authconfig
System is puppeted with user as ldap user, not in /etc/passwd.
However a local installed user fails as well.
Source of polkit is here:
https://github.com/wingo/polkit/blob/4c9a813f3fc1ada4fcce508d286e95f965a3002a/src/polkitagent/polkitagentlistener.c
/etc/pam.d/system-auth-ac:
#%PAM-1.0
This file is auto-generated.
User changes will be destroyed the next time authconfig is run.
auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        sufficient    pam_ldap.so use_first_pass
auth        requisite     pam_succeed_if.so uid >= 500 quiet
auth        required      pam_deny.so
account     sufficient    pam_ldap.so
account     sufficient    pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 500 quiet
account     required      pam_permit.so
password    requisite     pam_cracklib.so try_first_pass retry=3 type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    sufficient    pam_ldap.so use_first_pass
password    required      pam_deny.so
session     required      pam_mkhomedir.so skel=/etc/skel umask=0077
session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     sufficient    pam_unix.so
session     sufficient    pam_ldap.so
strace:
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
open("/proc/9527/cgroup", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f84828fa000
read(5, "11:freezer:/\n10:memory:/\n9:cpuac"..., 1024) = 192
close(5)                                = 0
munmap(0x7f84828fa000, 4096)            = 0
open("/proc/1/cgroup", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f84828fa000
read(5, "11:freezer:/\n10:memory:/\n9:cpuac"..., 1024) = 148
close(5)                                = 0
munmap(0x7f84828fa000, 4096)            = 0
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
getpeername(2, 0x7ffc19e92230, 0x7ffc19e9222c) = -1 ENOTSOCK (Socket operation on non-socket)
futex(0x7f84826da528, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
ioctl(2, TCGETS, {B38400 opost isig icanon echo ...}) = 0
open("/usr/lib64/charset.alias", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "\n** (xfce-polkit:9527): \33[1;35mC"..., 140
** (xfce-polkit:9527): CRITICAL **: polkit_agent_listener_register_with_options: assertion 'POLKIT_IS_SUBJECT (subject)' failed
) = 140
11:freezer:/
10:memory:/
9:cpuacct,cpu:/
8:blkio:/
7:net_prio,net_cls:/
6:cpuset:/
5:perf_event:/
4:hugetlb:/
3:devices:/system.slice/xrdp-sesman.service
2:pids:/
1:name=systemd:/system.slice/xrdp-sesman.service

There is nothing related to puppet, puppet only apply configuration asked by the ssyadmin. I think that the user source isn't related too (/etc/passwd or ldap). There can be probably missing bits for systemd  because opening sessions remotely isn't really prepared on systemd and xrdp doesn't implement any systemd hooks.

Hi @moobyfr, I installed the same exact version on a AWS machine and it worked seamlessly. Do you have any hints I could look for what the difference between the two system could be?

I am not a good c-programmer, however willing to learn. I have some python skills. If someone could point me into the right direction how to track this down (debugger etc), I'll do the work. Thank you


Explain routerScaleSize

I have a question about the new parameter routerScaleSize.
From what I understand, it is designed to distribute peers using the same room over different mediaSoup workers. Right?
Why the default value of 20? In the post about the performance, it was written that there could be like 100 users in 1 room, and this was at a time when a room was bound to a single mediaSoup worker.

The reason for being conservative is that we want to support a worst case scenario, both in hardware and usage. 100 users works fine when you have a setting where a few people talk and a lot of people are stopping video and muting themselves. If you have a setting where everybody has video and audio the count goes down. You are correct though, 20 is probably a too low setting for most hardware, and we could reasonably assume something like 40-50.
To have an example, let us assume that (in a worst case) 10 people are unmuted at any given time in a meeting. Also let us assume that lastN for all participants in the call is on average 5. These assumptions are a bit high, but that is on purpose.
If we set routerScaleSize to 40, this would give us:
(10 audio + 5 video) * routerScaleSize ~ 600 active consumers
In other words, let us increase it to 40 by default, but newer hardware should have no trouble in having it higher. Also note that when all routers (cores) have been filled up to routerScaleSize, it continues to fill them up based on least load.

Depends also on Last-N settings

Yes, in the example above I assumed lastN ~ 5. This will be your average of mobile users, normal users, and if someone has manually changed their settings.

OK. So now it is more less a load balancer on the cores of the machin, not to overload 1 core with 1 room when there are a lot of users in this room. But then, is it not better to scale with core load from the beginning and not the number of peers?
So we assume when the core load is at 75%, then route peers to the next mediaSoup worker. If we route by number of peers like now, the cut off number varies depending on the hardware (faster or slower CPU). If we route based on the core load, it is then independent of the hardware.

I think we will have a hard time matching worker/router to CPU core to be able to check the load, and also, it would be very difficult to make an algorithm that can "allocate" a core for a router based on load. When you have a router you want to fill it up to some extent before moving to the next, the piping between routers is in no way free (from a CPU standpoint).
If we do in naively, this would potentially result in a room with 5 users on 5 different routers, and that is in no way optimal.

OK, I understand.

One more question, I thought that new room start always on a new router, but from TOP I see that for 7 rooms only 4 workers are loaded and 4 are idle (8 core machine). So you load a worker to the limit set in config, whatever the room is?

The routers are only shuffled, so random. That means that what you see can happen.


Fix year in CHANGELOG.md

The year for the 0.5.0 version is wrong in the changelog.

time flies 😭


CI: fix ci not running on all targets

Since #379 CI doesn't run all tests, only the tree construction and tokenizer ones
For example here
Many tests also don't compile anymore, because the use statements aren't working anymore. I noticed this where I added more tests for V8 (#383)
fixes #390

What we maybe could also look into, I think, caching doesn't really work correctly here for the registry. It always needs to download packages. Maybe we can also update that.

@jaytaph Can you maybe look at this? https://github.com/gosub-browser/gosub-engine/actions/runs/8086610458/job/22096738161?pr=391#step:8:895
It's failing somewhere in the tree iterator test. Probably a rerun should fix it, but maybe there is something we can fix.


Specifying HttpClient without JsonSerializer on ClientBuilder

Should be able to override one without the other.

Actually not a bug, side effect of #37. Fixed by #38 and verified by b786109


The find command in the GithubMiner produces Null for the lastCheckedAt time

When we run the find command, the newly added records in the found_repositories.json file has Null as the lastCheckedTime.

This is not an issue, in the miner it replaces the Null values, So this can be closed.


Triplet matrix conversion to compressed storage with given Iptr

It would be good to convert triplet matrix to a cs with chosen Iptr type.
There might be a way to let Iptr = I as default?

Not sure how to fix CI / test (nightly). Doesn't seem to be related to my change

Not sure how to fix CI / test (nightly). Doesn't seem to be related to my change

CI can be fixed by removing -Zpackage-features from line 128 in ci.yml. This was a check for consistency of build-flags, but has now been stabilised

CI can be fixed by removing -Zpackage-features from line 128 in ci.yml. This was a check for consistency of build-flags, but has now been stabilised

CI can be fixed by removing -Zpackage-features from line 128 in ci.yml. This was a check for consistency of feature-flags, but has now been stabilised

Should I do this in this PR though?

CI can be fixed by removing -Zpackage-features from line 128 in ci.yml. This was a check for consistency of feature-flags, but has now been stabilised

Should I do this in this PR though?

We can avoid the extra type by changing TripletMatBase  to have separate types for row and columns with a default type, as done with CsMatBase. This is a ton more work with little benefit, tri matrices are only intended for construction as of now.
I'm happy with type annotations, they should be elided by type coercion in most cased as mentioned by vbarrielle

We can avoid the extra type by changing TripletMatBase  to have separate types for row and columns with a default type, as done with CsMatBase. This is a ton more work with little benefit, tri matrices are only intended for construction as of now.
I'm happy with type annotations, they should be elided by type coercion in most cased as mentioned by vbarrielle

I'm sorry but I'm not able make cargo build sprs-ldl with sprs_suitesparse_ldl and sprs_suitesparse_camd enabled. Could you help me?

I'm sorry but I'm not able make cargo build sprs-ldl with sprs_suitesparse_ldl and sprs_suitesparse_camd enabled. Could you help me?

Did nigthly break the -p option? Seems you have to specify package now, use --features sprs-ldl/sprs_suitesparse_ldl

Did nigthly break the -p option? Seems you have to specify package now, use --features sprs-ldl/sprs_suitesparse_ldl

All right, thanks @kolstae and @mulimoen for figuring out the CI fix. Specifying features of dependencies is clearly a pain point in rust...

All right, thanks @kolstae and @mulimoen for figuring out the CI fix. Specifying features of dependencies is clearly a pain point in rust...


use latest alpine from docker hub, update readme

Looks good, before merging I would need a PR to update the internal mirror and relative Jenkinsfile.

@Raffo yup. I remember ;)

👍


indent-blankline doesn't seem to have any effect on empty lines

The main reason I wanted to use indent-blankline was to fill empty lines but it isn't doing that. I'm using indent-Blankline with indentLine as well. Did I configure something wrong or?

I probably configured something wrong. Can I see the code for obtaining the With custom listchars and g:indent_blankline_space_char screenshot example in the readme

yeah looks like something is not working.
Can you set let g:indent_blankline_debug = v:true and see if you get any errors?

I don't seem to get any errors. Here is a screenshot of my config:

I'll explain some lines.
Line 10 is from this issue https://github.com/Yggdroot/indentLine/issues/315
Lines 2 and 1 is a map I did for completely toggle indent lines, I also use  LeadingSpaceToggle because if I don't all spaces appear with the g:indentLine_leadingSpaceChar symbol and only works normally after toggling at least once (I don't know if it is a bug or not but it doesn't seem like it, LeadingSpaceToggle solves it for me)

Okay, can you give me a bit more information? What's the output of these:
:echo g:indent_blankline_namespace
:echo jobpid(g:indent_blankline_nvim_instance)
:echon rpcrequest(g:indent_blankline_nvim_instance, 'nvim_eval', '"hello"')

weird...

:echo g:indent_blankline_namespace outputs:
E121: Undefined variable: g:indent_blankline_namespace
:echo jobpid(g:indent_blankline_nvim_instance) outputs:
E121: Undefined variable: g:indent_blankline_nvim_instance
E116: Invalid arguments for function jobpid
:echon rpcrequest(g:indent_blankline_nvim_instance, 'nvim_eval', '"hello"') outputs:
E121: Undefined variable: g:indent_blankline_nvim_instance
E116: Invalid arguments for function rpcrequest

Okay, so the plugin is never even initialized.
Can you run :IndentBlanklineRefresh manually and see what happens? Then try the above again

Same output...

What is the output of echo indent_blankline#BufferEnabled()

v:true

What is the output of echo len(nvim_list_uis()) and echo g:indent_blankline_enabled?

echo len(nvim_list_uis()) outputs 1
echo g:indent_blankline_enabled outputs 0

echo g:indent_blankline_enabled outputs 0

that would be why then
I didn't see that in your config, g:indentLine_enabled will set this too.
0 means false.  Try setting let g:indentLine_enabled=1 in your config

The thing is I want it disabled by default and then toggle it when I want to. Is it possible?

you can disable it by default as well.
let g:indentLine_enabled=0 disables it globally, and there is no way to toggle it on or off globally.
But you could disable it per buffer, and then turn it on/off per buffer.
this should be all you need
augroup AutogroupDisableIndentline
    autocmd!
    autocmd Syntax * IndentBlanklineDisable
augroup END

So now my config looks like this:
" --- For indentLine and indentBlankLine --------------------------------"
autocmd VimEnter * if bufname('%') == '' | IndentLinesDisable | endif
let g:indentLine_enabled = 1
let g:indentLine_char = '¦'
let g:indentLine_leadingSpaceEnabled = 1
let g:indentLine_leadingSpaceChar = '·'
let g:indentLine_showFirstIndentLevel = 1
let g:indentLine_color_gui = '#444444'
let g:indentLine_bgcolor_gui = 'NONE'
let g:indent_blankline_debug = v:true
nnoremap <silent><leader>i :LeadingSpaceToggle<CR>
      \:IndentBlanklineToggleAll<CR>

But the issue of not filling the lines persists and I got an error when I source my init.vim:
indent-blankline encountered an error...92: Not an editor command: viminit.vim

can you show me your whole config?

Here is what I get now when I do the commands you asked me before...

:echo g:indent_blankline_namespace outputs: 3
:echo jobpid(g:indent_blankline_nvim_instance) outputs:
E900: Invalid channel id
0
:echon rpcrequest(g:indent_blankline_nvim_instance, 'nvim_eval', '"hello"') outputs:
Error invoking 'nvim_eval' on channel 33:
Invalid channel: 330
:echo len(nvim_list_uis()) outputs: 1
:echo g:indent_blankline_enabled outputs: 1

can you show me your whole config?

the whole init.vim?

the whole init.vim?

yes
the Vim:E492: Not an editor command: viminit.vim error is very likely due to some configuration

Alright, here it is https://github.com/sp1n0r/configs/blob/main/init.vim

hm yeah I don't see anything either.
Maybe the abbreviations, but they shouldn't.
cabbrev init e ~/AppData/Local/nvim/init.vim
cabbrev ginit e ~/AppData/Local/nvim/ginit.vim

But I see you are using treesitter? So you are on nvim 0.5?
If so, you can use the lua branch of this plugins instead. It's a lot more stable.

hm yeah I don't see anything either.
Maybe the abbreviations, but they shouldn't.
cabbrev init e ~/AppData/Local/nvim/init.vim
cabbrev ginit e ~/AppData/Local/nvim/ginit.vim

Yeah this shouldn't cause any issue.

But I see you are using treesitter? So you are on nvim 0.5?
If so, you can use the lua branch of this plugins instead. It's a lot more stable.

Yup, got one of the latest nvim nightly versions. How to I get that 'lua' branch? I thought I was using that.

Do I have to change my init.vim to init.lua?

Do I have to change my init.vim to init.lua?

no, just change the plug call to this Plug 'lukas-reineke/indent-blankline.nvim', { 'branch': 'lua' }

Cool. Not on my pc anymore. I'll let you know the result in an hour or so

I think it is working fine now. There is only a problem that I'd like to fix in indent-blankline.nvim in order to leave the indentLine plugin altogether.

Other thing. I changed my config in order to use standalone indent-blankline.nvim but it doesn't seem to be working in my init.vim.

But it works in help files. That got me thinking. I use these settings in my init.vim, could it be related?
set tabstop=2 softtabstop=2
set shiftwidth=2
set expandtab
set smartindent

I should probably close the issue though. The original problem seems to be solved

in order to leave the indentLine plugin altogether

this plugin only fills in empty lines, you need both. At least until neovim supports better virtual text
#12 is currently not possible. I hoped the bugfix will allow conceal on line endings, but it doesn't.

Oh that's unfortunate. I'll continue to use both then, and I'll probably disable the first level. Nonetheless I think it's a great plugin and has a lot of potential since it is compatible with lua


Update Test.sh

References user story #47
Definition of done
test.sh runs after pull request and successful build, and is ready for tests to be added (if any ran here)

initial test.sh file created. Research for it done along with research done on build shell script. To be updated
[t:15]


Can not find RKE state file: open /var/lib/rancher/management-state/rke/rke-156711546/cluster.rkestate: no such file or directory

Hi. I cleaned all nodes on my cluster (as the guide  https://rancher.com/docs/rancher/v2.x/en/cluster-admin/cleaning-cluster-nodes/ ), then I re-installed rancher server using the command:
sudo docker run -d --name rancher_server --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher:v2.2.7
Then, I created a custom cluster and after adding the new node (include both master and etcd), I got the error:
Can not find RKE state file: open /var/lib/rancher/management-state/rke/rke-156711546/cluster.rkestate: no such file or directory
Do you know how to fix this ?

Useful
Info

Versions
Rancher v2.2.7 UI: v2.2.93

Route
global-admin.clusters.index

http://bbs.xuegod.cn/thread-2555-1-1.html
According to this Chinese forum post (I just looked at the pictures, I don't speak Chinese), it might have something to do with your node's hostname.
I changed my hostname from having underscores "_" to having hyphens "-" and this error went away.

I'm facing the same issue while setting up locally with hostname as <IP_ADDRESS>

Thanks for reporting, I am merging this into https://github.com/rancher/rancher/issues/25376

Thanks @zeidoo for the hint with the node name! My problem was a . at the end of the node names...
(In a Rancher version 2.4.5 RKE HA K8s 3 node setup.)
Regards,
Philip

I have a similar problem, I try to use rke for the cluster but I get:  FATA[0003] Failed to write state file: open ./rancher-cluster.rkestate: permission denied
I could solve the problem.


Expose SDK Sidecar GRPC Server as HTTP+JSON

Overview
Provide a REST / HTTP+JSON interface to the SDK gRPC Sidecar service.
Motivation

The tooling around C++ on Windows is basically a deal breaker. It involved installing 7 things and deviated quite considerably from the tooling around installing on Linux/MacOS
gRPC on Unreal and Unity tends to be difficult to impossible. But both can do HTTP+JSON.  This will make integration with these commercial engines much easier.
Any other language that gRPC doesn't support out of the box, this provides an easy to use alternative.

Design
I see that there are two options, each with it's own pros and cons:
Envoy Proxy

Pro: It's less tightly integrated with the SDK process.
Pro: it has a lot of development and backing behind it.
Con: It would be a separate process / separate Pod, which adds complexity.
Pro: It also exposes metrics around usage and consumption. They may come in handy down the line.

We could do local development with a docker run command that runs both Envoy and the SDK in local mode.
gRPC Gateway

Pro: It looks like we can run this in the same process as the gRPC server. This means we can keep the single binary for the sidecar as well as for local development.
Pro; it's simpler. It's built for just the one purpose that we want.
Con: I'm not sure that the grpc-gateway has as much contribution / backing at Envoy.
(but looks like releases are fairly frequent)
Question: How well does the grpc-gateway support streaming rpcs?

Research
Envoy

https://www.envoyproxy.io/docs/envoy/v1.6.0/configuration/http_filters/grpc_json_transcoder_filter
https://www.envoyproxy.io/docs/envoy/v1.6.0/configuration/http_filters/grpc_http1_bridge_filter.html?highlight=grpc

grpc-gateway

https://grpc-ecosystem.github.io/grpc-gateway/docs/usage.html

@Kuqd I believe you've got some experience with grpc-gateway. What are your thoughts?

My thoughts regarding grpc-gateway:

it can run in a sidecar also, it basically connect as a client to your grpc server so both works in or out of the process. (which means you also get metrics if you have some on your grpc server)
it's pretty quick to implement, I've got some piece of code ready to go.
Stream is implemented over newline-delimited JSON streams, so if you adapt your client you also get streams but not bidirectional. (no idea how envoy works on this subject )
it can generate swagger-spec which is good for documentation but also for generating client code and all the goodness that comes with the swagger ecosystem.  (not the case with envoy)
It's declarative, so you add more info to your proto files and the code is generated, this also means you get documentation for developer that want to create a custom client, they just have to look at either the proto file or the swagger generated. (not the case with envoy)
you have possibility to extend/hook via code into the gateway to make it fit your needs (not the case with envoy)

On our project we've stopped using it because we've dropped grpc and not vise-versa.
I don't have any experience with envoy so I can't give feedback, but I've heard it's good.
I would start with grpc-gateway and if required improve via a POC using envoy. If we have more time I'm happy to see someone take a shot at envoy to compare the end result before taking the decision. As @EricFortin would say:

we should timebox it

For me the client gen is neat, and I feel like with grpc-gateway we keep that.

@Kuqd I think you have basically 100% echoed my thoughts.
It's also far better documented than Envoy in this regard as well.

Copy pasted from discussion on chat for posterity:
Cyril Tovena [10:14 AM]
:stuck_out_tongue: (edited)
now the question is do you want to go with sidecar in the pod or in-process

Mark Mandel [10:15 AM]
I think in process
it's far simpler

Cyril Tovena [10:15 AM]
ok should it be on by default ?

Mark Mandel [10:15 AM]
so that's a good question

Eric Fortin [10:15 AM]
I agree with in process being better

Mark Mandel [10:15 AM]
start with yes?

Eric Fortin [10:15 AM]
I wanted to put my thought in GH but can do so here

Mark Mandel [10:16 AM]
or start with it always on, and then we can tighten it down
yeah true - probably better on github, then there is a record
(or I can copy paste)
yeah, I worried about the additional resources/complexity of adding _another_ sidecar
which is then a proxy to a proxy... it feels a little messy
and the local SDK server would maybe need to shift to being a `docker-compose` setup maybe? it was not as clean as I would like

Eric Fortin [10:18 AM]
yeah docker dependency for day to day scares me, because Windows.

Mark Mandel [10:18 AM]
yeah, whereas having it in process means we can still compile a .exe that can be executed

Eric Fortin [10:18 AM]
exactly

Mark Mandel [10:19 AM]
:thumbsup:  I think we are all in furious agreement.

Eric Fortin [10:19 AM]
Again haha

Cyril Tovena [10:20 AM]
@Mark Mandel I had one issue with the gateway it was related to the swagger-gen and it got fixed but it was a good 2month to wait
that's my only downside
but since it's google you might be able to speed up thing if required

Mark Mandel [10:21 AM]
that is true. I have abilities to bug people :smile:

Cyril Tovena [10:21 AM]
that's a good skill !

Work happening over here: https://github.com/markmandel/agones/tree/feature/http+json

Some directions for swagger scheme:
https://github.com/grpc-ecosystem/grpc-gateway/blob/a5b66c16bab6156b1cc525e76159bdff463123ba/examples/proto/examplepb/a_bit_of_everything.proto#L28

Also I doubt swagger will generate a client that support the stream in health GRPC method, you could - duplicate the health method (with a different name) without the stream and implement differently

remove the (fancy) stream.
implement a custom http client for each language.


Harbor App (Appimage) cannot load config

After making a clean install of harbor CLI (harbor doctor shows everything OK), I tried the Harbor App via AppImage. No AppImage wrapper was used, only chmod +x HarborApp.AppImage and ./HarborApp.AppImage.
The problem is only the Profiles tab which outputs an error "Unexpected error loading configuration".

Other tabs are unaffected.

Thank you so much for another report!
Unfortunately I can't reproduce it right away, so we'll have to debug
Just released v0.2.12 comes with a couple of things that should help:

harbor doctor now prints location of the Harbor Home, it should be present in the App in the "CLI" page, could you please check that the value there matches what you see when running the harbor doctor in the terminal?
Fixed one edge-case related to a deletion of a currently selected config profile. The error state looks slightly different compared to your screenshot, but worth trying it out with the new app build

Other than that - I have a suspicion that it might be related to either:

AppImage sandboxing which isn't kicking in on my system and is available by default on yours (Fedora SilverBlue, right?)
AppImage not bundling correct dependencies (unlikely, but not impossible)

As before thank you for quick update of harbor, however the problem persists. I followed the troubleshooting you have suggested and here are the results.
Harbor App doctor (the values match between App and CLI):
✔ Docker is installed and running
✔ Docker Compose (v2) is installed
✔ Harbor home: /var/home/user/.harbor
✔ Default profile exists and is readable
✔ Current profile (.env) exists and is readable
✔ CLI is linked
✔ NVIDIA Container Toolkit is installed
CLI:
14:57:09 [INFO] ✔ Docker is installed and running
14:57:09 [INFO] ✔ Docker Compose (v2) is installed
14:57:09 [INFO] ✔ Harbor home: /var/home/user/.harbor
14:57:09 [INFO] ✔ Default profile exists and is readable
14:57:09 [INFO] ✔ Current profile (.env) exists and is readable
14:57:09 [INFO] ✔ CLI is linked
14:57:09 [INFO] ✔ NVIDIA Container Toolkit is installed
I tried to disable sandboxing for the AppImage by running it with --no-sandbox flag, however the problem persists (the output remains the same).
I have also tried to make clean manual install instead of automated. And the error remains.
What I found are some fixed issues concerning Fedora Silverblue and broken support of AppImage. For example https://github.com/fedora-silverblue/issue-tracker/issues/589. Yet none of the problems reported there seem to be connected to this one. I plan to upgrade to Fedora Silverblue 41 soon (probably next weekend), so I can test if there is anything different.
Also, can I ask if you have any plans for Flatpak release? Flatpak in my experience has fewer (user side) problems than AppImage.

Yeah, nothing in these looks suspicious in any way. The --no-sandbox seems to be related to the Electron apps (Harbor runs with Tauri). Granted the homedir location, could you please try running it with: --filesystem=/var? Very thin chance, but still
I'm certain that this happens because of some sort of discrepancy in the FS access permissions between the systems with a very short possible fix. As another attempt - could you please try running it via AppImage launcher?

any plans for Flatpak release?

Unfortunately in Harbor's case, the Flatpak would be just a repackaged .deb, I'm not 100% it'll work on Silverblue (just learned about immutable FS), It looks like you won't be able to use the provided rpm due to exactly the same reason

I've added a "guess" fix to https://github.com/av/harbor/releases/tag/v0.2.14 with an explicit permission to read/write to the User's home folder (previously was under the wildcard). Also a thin chance that it fixes the problem for you, but please try running the AppImage from that release

Thank you for incredibly fast update, I updated CLI and the App. Nevertheless, the problem remains the same in version 0.2.14 with or without --filesystem=/var flag.
I noticed that the button create profile is clickable despite the error. The profile creation dialog opens and upon clicking create profile the screen flashes and dialog closes, simultaneously the following error is produced in command line:
** (harbor-app:364900): WARNING **: 19:10:21.768: atk-bridge: get_device_events_reply: unknown signature

Hopefully I will be able to install the rpm version on my test PC tomorrow (running the same Fedora Silverblue 40, installation via layering of the package).

No worries!
I'll try to find some time to test the AppImage in a VM during my next chunk of Harbor time (next weekend or a weekend after that)
Thanks for sticking it out through the early versions

Sorry for taking long with a reply, I am still having issues with upgrading to Fedora 41 (Nvidia drivers), so for now I tried to install the Harbor App into Fedora toolbox (which is should have access to the system as if I installed on non-immutable Fedora). The App is installed without any errors, but the problem remains the same as with the AppImage. Now the only thing that these different installations have in common is that home directory is under /var and security policy of Fedora.

I was working on a somewhat related feature (that'll eventually also let me test the app in the Fedora easier) and found out a possible culprit.
I think that the desktop entry in Fedore will run in a custom shell without the $PATH modification added by Harbor. Theoretically, when starting the App from a terminal directly, it should inherit the correct env and have access to Harbor's executable.
So, could you please try launching the app from the terminal instead of the desktop entry and let me know if it makes the app functioning?
# App's binary is named "harbor-app" and should be made available
# on the host during install
which harbor-app
# Should be something like: /usr/bin/harbor-app

# If not found - look into the .desktop entry for the app:
cat /usr/share/applications/Harbor.desktop

# Launch the app binary directly:
harbor-app

I'm currently trying to find any workarounds, but still curious to know if this was the reason in your instance.
Thanks!

Thank you for coming back to my issue. I am sorry I did not reply immediately, however I am still figuring out how to properly install RPM package on the immutable Fedora Silverblue (without breaking anything else). I updated to the latest version of harbor (CLI and App Appimage) and the issue persists. As for the debugging steps you suggest for the RPM, I tried to install RPM through Distrobox which shares the system configuration partially, yet it cannot detect Docker. I will have to layer the RPM directly.

Absolutely no worries!
I'm hoping to get my hands on testing this next weekend. Recent Harbor release included webtop service with the Harbor App pre-installed, could be used as a workaround until we get it to work natively


feat: able to always show slider labels

Describe your changes

Adds possibiblity to always show labels for thumbs, not only on hover.

Also adjust so thumb on focus and its label are visible on hover by increase its z-index, good when using multiple thumbs (else thumb dragging could end up hidden behind other thumbs/labels)

Checklist before requesting a review

[x] I have performed a self-review of my own code
[x] I have verified that the code builds perfectly fine on my local system
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have verified that my code follows the style already available in the repository
[x] I have made corresponding changes to the documentation

not needed now, closing


Detect Dropbox Link

Dropbox version 3.0.3
OSX Version 10.10.1
I am having an issue where Upshot is not detecting the link from Drop Box.
Not sure if this has anything to do with it but the screen shot you have on that step: http://screencast.com/t/SWIMUhWdv  is different from what the dropbox menu looks like on my computer: http://screencast.com/t/nlL6jBMuBs
P.S. sorry for the screencast links. I am trying to get upshot installed to replace it :)

here is a link that dropbox is generating. https://www.dropbox.com/s/7zxcgzfao1am5fx/Untitled-1.png?dl=0

You're not using the Dropbox "Public" folder. In the Preferences window there is a (?) button next to the detect button, which leads here: https://www.dropbox.com/help/16
Please use the instructions there for Dropbox to give you a Public folder, then generate a URL (Right click, "Copy Public Link") in that folder, which will look something like this: https://dl.dropboxusercontent.com/u/18779383/How to use the Public folder.rtf

(The difference between the Public folder and regular Dropbox sharing is that the Public folder lets you link directly to a file, as if it was a straight up web server; instead of an interstitial Dropbox website that has the shared file on it)

Thank you for your help! It looks like I have to upgrade to Pro or Business account for this to work.

"It looks like I have to upgrade to Pro or Business account for this to work."
Oh? That would be complete news to me. I do not have a paid account, nor does anyone I know.

(Unrelated, I'll have you know I am working on Issue #70 but it's not trivial, so not quite done yet. But when it is done, you'll be able to upload to different places, not just Dropbox).

Dropbox writes:

Creating a Public folder
Dropbox links give you everything you need to share and preview files and folders. However, if you’re just a diehard fan of Public folders, click here to enable one on your account. Currently only Pro and Dropbox for Business users can enable Public folders, so you may have to upgrade first.

and the link points here: https://www.dropbox.com/enable_public_folder
Try that out?

That means, upload functionality outside of Dropbox has just become a lot more important.

I just tweeted that out to my followers. Sorry that I can't help you right now, and many thanks for pointing it out! I'll quickly get back to implementing better upload targets!
First will be imgur because it has a fairly easy API. After I get that straight, architecturally, I should be able to add many more over time (like SFTP, S3, and so on).

thank you for your follow up! I look forward to seeing more upload targets. +1 for S3
p.s. you should get a donation link ;) unless i am missing it...

Oh thank you for the suggestion, I shall put one on upshot.it, it would certainly be encouraging if someone bought me a coffee for this ;)
If you have bitcoin, I have a tipping button on my blog ;) http://fredericiana.com/2014/11/27/bitcoin-tipping/


Change to junit5 methods for some tests

Thanks for the review.
I'm planning to work on the feedback in the next couple of days. If a problem arises with the Hamcrest Matchers, I think I'll keep it out of scope for this PR since this was meant as a quick & easy win.

Thanks for the review.
I'm planning to work on the feedback in the next couple of days. If a problem arises with the Hamcrest Matchers, I think I'll keep it out of scope for this PR since this was meant as a quick & easy win.

That's fine, I just wanted to highlight it!


Fix an issue Swift optional properties no longer ignored

This PR to fix the problem that the properties would be added to the schema even marked as ignored if the properties are optional.
It will happen only on Swift optional properties.
Related issue: https://github.com/realm/realm-cocoa/issues/2732

retest this please

/cc @mrackwitz @bdash

:+1:
This seems like a good candidate for a 0.96.2 release.

:+1:

CI failure is unrelated (documentation).

I have this kind of property how can i ignore this property
var photos = Array()

@macvinod please open a new issue. Also, Array is not an allowed property type for Realm models. See https://realm.io/docs/swift/latest/#models