Parsing issue of quoted values in large CSV file

I ran into a CSV parsing issue.
The source file I'm uploading is a CSV file (2.5 million rows) that only uses "quotes" when needed. This is the default readr::write_csv() behaviour, e.g. to escape commas in values. Note that I am not indicating Field Quotes: " in the IPT, as that is reserved for when all values are quoted.
Snippet of source file. Notice quoted "BIG FLOCK, WIDESPREAD FORAGING ON SPRAT" in occurrenceRemarks
eventID,basisOfRecord,occurrenceID,individualCount,sex,lifeStage,behavior,occurrenceStatus,associatedTaxa,occurrenceRemarks,scientificNameID,scientificName,kingdom
110000125_110000125_110003829,HumanObservation,110000125_110000125_110003829_1100024895,200,,,Deep pluging,present,,"BIG FLOCK, WIDESPREAD FORAGING ON SPRAT",urn:lsid:marinespecies.org:taxname:137156,Rissa tridactyla,Animalia

In the generated Darwin Core Archive, the resulting file is the following. Notice how "BIG FLOCK ...  is now spread over multiple fields:
id	basisOfRecord	occurrenceID	occurrenceRemarks	individualCount	sex	lifeStage	behavior	occurrenceStatus	associatedTaxa	eventID	scientificNameID	scientificName	kingdom
110000125_110000125_110003829	HumanObservation	110000125_110000125_110003829_1100024895	"BIG FLOCK	200			Deep pluging	present		110000125_110000125_110003829	WIDESPREAD FORAGING ON SPRAT"	urn:lsid:marinespecies.org:taxname:137156	Rissa tridactyla

Any idea what might be causing this? It's the first time I encounter this, even though I have uploaded many such CSV files (with only quoted values when necessary) to the IPT before, without ever running into issues. See e.g. https://www.gbif.org/occurrence/3795234906, where some values contain commas:
UvA-BiTS tag attached by harness to free-ranging animal | Found dead, possibly predated by peregrine falcon.

@peterdesmet Thank you Peter for reporting this and sorry for the late reply.
Looks like Field Quotes resolves this issue, but their description is not correct in that case

Oh great, so it Field Quotes = " also works file files that are quoted where needed. Is there a reason why Field Quotes doesn't use " by default? It is the default in the CSV readers and writers I'm aware of:

quote in https://readr.tidyverse.org/reference/read_delim.html
quote in https://readr.tidyverse.org/reference/write_delim.html
quoteChar in https://specs.frictionlessdata.io/csv-dialect/#specification

Yes, it works. Also I've just checked creating a couple of file sources, and they all have Field Quotes = " by default (txts though)

Thanks! I notice now that other csv or even csv.zip files I uploaded as source (in other resources) all got Field Quotes = ". So seems like an isolated case.
In any case, would indeed be good if description of the field was updated.

Did some more tests. Looks like the IPT is assessing the source data to figure out whether to use quotes values or not:

No quotes values found, use Field Quotes = <empty>
Quoted values found, use Field Quotes = "

So far so good. The issue seems to be that it only looks in the first 20 lines of a file. Compare:

line_10.csv: quoted behavior value in line 10, will set Field Quotes = "
line_21.csv: qouted behavior value in line 21, will set Field Quotes = <empty>

This will lead to parsing issues down the line. The user can off course always set Field Quotes to resolve the issue, but then they need to remember doing so. So I'd like to know if there are any downsides to always setting Field Quotes = "?

@peterdesmet Thank you for spending your time testing this issue.
I was also thinking about setting Field Quotes to ". I don't see any issues right now, but I assume they may appear.

Let's just set " as a default parameter #2190


openness blink soo tightly

Hi, i tried to use this one, but checking from neos and code, openness value just check status of gaze, and no threadings, so blinks so tightly. could i get the openness more softly or directly get openness from Varjo Eye Sdk? (i heard about this things from discord.)

Checked from Varjo Native SDK, but this one could be check eyestatus while blinking.
leftStatus and rightStatus return a value for each eye as follows:

0 – eye is not tracked and not visible (e.g., the eye is shut)
1 – eye is visible but not reliably tracked (e.g., during a saccade or blink)
2 – eye is tracked but quality is compromised (e.g., the headset has moved after calibration)
3 – eye is tracked

VarjoCompanion is this one:
        public enum GazeEyeStatus : long
        {
            Invalid = 0,
            Visible = 1,
            Compensated = 2,
            Tracked = 3
        }

so, if openness returns from like this one is better: GazeEyeStatus = smoothvalue (tracked ? 1f : ( Compensated? 0.5f : (Visible? 0.25f : 0f ) )

This is a good idea! Currently, Varjo's SDK only tracks eye openess as an enum like you've mentioned, I'll make the the changes and let you know when I do

Ok, I've made a fork of the mod and have poked around with Neos's built-in smoothing. Here is my implementation and the methods we have available:

Likewise, I have created a pre-release of the mod if you want to try it out. I don't own the HMD myself, so I need people to test this on!
https://github.com/dfgHiatus/NeosVarjoEyeTracking/releases/tag/v1.0.5

Thanks, I tried it.
but It seems not worked in Local, and It seems already works without SmmothStep in Remote.
for detail, I issue at #3 .

Since we did not know the source code, we decompiled it and compared it to the actual operation.
First, SmmothStep did not seem to work properly. [Failed to ping] was displayed.
Second, the Openness of RightEye was assigned "num2" when it should have been "val2".
As a result, the left eye did not move, but the right eye did.

Oops, it looks like I didn't push the new branch of the changes. I'll push this first thing in the morning tomorrow, thanks for the input!

Added in latest build


ERR_CONNECTION_REFUSED: Data Hub 2.3 developer edition with docker

Dear SAP support team,
I try to start Data Hub 2.3 developer edition with docker.
https://developers.sap.com/tutorials/datahub-docker-v2-setup.html
It seems that my docker build was fine and the system is up and running. However, my browser (Chrome) says ERR_CONNECTION_REFUSED for:

http://localhost:8090 (SAP Data Hub Modeler)
http://localhost:9225 (SAP Vora Tools)

I also tried urls from cmd output:

Data Hub Modeler:           http://<IP_ADDRESS>:8090
Vora Tools:      http://<IP_ADDRESS>:9225

What might be wrong with my environment?
Thanks
Marc

cmd input:
D:\DatahubDevEdition>docker run -ti --env VORA_USERNAME=vora --env VORA_PASSWORD
=SomeNicePassword19920706 --publish <IP_ADDRESS>:8090:8090 --publish <IP_ADDRESS>:9225
:9225 --publish 30115:30115 --name datahub --hostname datahub --network dev-net
datahub run --agree-to-sap-license

cmd output:
========================================================
   Welcome to the SAP Data Hub Dev Edition (provided as Docker image)
                         (c) 2018 by SAP SE
            non-productive usage, local deployments only
========================================================

2019-01-11T13:20:28+00:00 -------- executing set_proxy_env --------
2019-01-11T13:20:28+00:00 proxy settings remain unchanged
2019-01-11T13:20:28+00:00 -------- executing DATAHUB_install --------
2019-01-11T13:20:29+00:00 -------- executing VSYSTEM_install --------
creating user/group vsystem
groupadd: group 'vsystem' already exists
installing vsystem
2019-01-11T13:20:31+00:00 -------- executing JAVA_install --------
2019-01-11T13:20:31+00:00 -------- executing VREP_init --------
extracting vrelease_21_delivery.tar.gz
extracting vsolution_shared_ui.tar.gz
extracting vsolution_vsystem_ui.tar.gz
extracting velocity-data-tools-ui.tar.gz
extracting vsolution_vora-tools-linuxx86_64.tar.gz
extracting datahub-app-base-vsolution-2.3.99.tar.gz
extracting datahub-app-base-vsolution-db-2.3.99.tar.gz
2019-01-11T13:20:46+00:00 -------- executing VFLOW_start --------
nohup: redirecting stderr to stdout
.
2019-01-11T13:20:47+00:00 -------- executing add_hostname_connectionmgr --------
2019-01-11T13:20:47+00:00 -------- executing CONNECTIONMGR_start --------
2019-01-11T13:20:47+00:00 Connection manager not present. Cannot start
2019-01-11T13:20:47+00:00 -------- executing add_hostname_vora --------
2019-01-11T13:20:47+00:00 -------- executing DQP_configure --------
2019-01-11T13:20:47+00:00 -------- executing DQP_start --------
...........
removing user SYSTEM in /etc/vora-security/default/htpasswd
User system created in /etc/vora-security/default/htpasswd.
2019-01-11T13:20:59+00:00 -------- executing VORATOOLS_start --------
nohup: redirecting stderr to stdout
.
2019-01-11T13:21:00+00:00 -------- executing DQP_wait --------

2019-01-11T13:21:00+00:00 -------- executing VORA_set_credentials --------
2019-01-11T13:21:00+00:00 creating user: vora in userstore
removing user vora in /etc/vora-security/default/htpasswd
User vora created in /etc/vora-security/default/htpasswd.
2019-01-11T13:21:00+00:00 creating user: vora in vora
connecting to <IP_ADDRESS> at port 2202 (<IP_ADDRESS>:2202) ...
switched to existing session ""
query send...
query executed in 1.84493 s.
cease to connect to <IP_ADDRESS>:2202...
connecting to <IP_ADDRESS> at port 2202 (<IP_ADDRESS>:2202) ...
switched to existing session ""
query send...
query executed in 0.0704851 s.
query send...
query executed in 0.0909435 s.
cease to connect to <IP_ADDRESS>:2202...
connecting to <IP_ADDRESS> at port 2202 (<IP_ADDRESS>:2202) ...
switched to existing session ""
query send...
query executed in 0.06366 s.
query send...
query executed in 0.0382611 s.
query send...
query executed in 0.0362352 s.
cease to connect to <IP_ADDRESS>:2202...
2019-01-11T13:21:04+00:00 -------- executing services_dump --------
2019-01-11T13:21:04+00:00 TLS is set to false
2019-01-11T13:21:04+00:00 Vora Transaction Coordinator   vora://<IP_ADDRESS>:2202
2019-01-11T13:21:04+00:00                                hana://<IP_ADDRESS>:30115

2019-01-11T13:21:04+00:00 Data Hub Modeler:              http://<IP_ADDRESS>:8090

2019-01-11T13:21:04+00:00 Vora Tools:                    http://<IP_ADDRESS>:9225

docker container error logfile output
datahub:/tmp # less v2node.ERROR

Log file created at: 2019/01/11 13:23:03
Running on machine: datahub
Binary: Built with gc go1.8.3 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0111 13:23:03.367392    2582 reflector.go:205] sap.com/vora/v2node/v2node/inter
nal/watch.go:190: Failed to list *v1.Pod: Get http://<IP_ADDRESS>:8555/api/v1/name
spaces/default/pods?limit=500&resourceVersion=0: dial tcp <IP_ADDRESS>:8555: getso
ckopt: connection refused
E0111 13:23:03.368430    2582 reflector.go:205] sap.com/vora/v2node/v2node/inter
nal/resolve.go:289: Failed to list *v1.Pod: Get http://<IP_ADDRESS>:8555/api/v1/na
mespaces/default/pods?limit=500&resourceVersion=0: dial tcp <IP_ADDRESS>:8555: get
sockopt: connection refused

Thank you for your feedback. We haven't heard from you in the past 30 days, so we are closing the issue.
If you still have questions, feel free to reopen the issue.

Hi Anish,
I did not come back here, since the same image (dev edition 2.3) worked on my Mac.
I gave up trying on Windows for now.
Best regards
Marc

Hi i am having this issue. I have checked the my cmd log and docker build seems to have worked. When I try the following in my browser (Chrome):
http://localhost:8090 (SAP Data Hub Modeler)
http://localhost:9225 (SAP Vora Tools)
The same error message is returned as in previous post
Any suggestions ?
Thanks,
Rochelle

Hi,
Are you using the new image for the developer edition?
Regards,
Anish

Hi Yes I am using the latest image


New Hope FFTs

I've been looking at New Hope for my company, so I've been testing your code, specifically rlwe_1024_12289_binomial. I noticed that the FFTs that it uses, FFT_twisted_forward_1024_12289 and FFT_twisted_backward_1024_12289 are not inverses of each other, multiplying everything by a factor of 1024.  Did you forget to divide by 1024 (or m, really) at the end?
When I run your code, the key agreement seems to work. When I try to simulate the same functions in sage though, that constant factor increases the error terms, making the key agreement fail.
Thanks,
Rino

@vscrypto I was wondering if you saw this question.

Hi Rino
Thank you for your comment. Please note that the key agreement only uses FFT_forward and FFT_backward. These are defined in rlwe_includes.h in terms of their twisted versions and a division by 1024 (implemented as a multiplication by 12277, which is the inverse of 1024 modulo 12289):
/*Use the twisted Fourier Transform in the Power of 2 case for multiplication
in the ring F_q[x] / <x^m+1>*/

#define FFT_forward FFT_twisted_forward_1024_12289
#define FFT_backward(_x)\
do {\
	FFT_twisted_backward_1024_12289(_x);\
	for (uint16_t _i = 0; _i < m; ++_i) {\
		MUL_MOD((_x)[_i], (_x)[_i], 12277, (q));\
	}\
} while(0)

I wish you the best with testing the code.
Best Regards
Vikram

Thanks for the response, I completely missed that section of the code.  I had implemented the code in sage, but didn't see that bit.
Best wishes,
Rino


How does pylance check for imports

I'm running remote development with singularity (the only option we have available).
While it works perfectly specifying the python interpreter, pointing to the interpreter running in the singularity image, when executing code. It seems that pylance is not picking up the installed dependencies from the specified python interpreter. (And I also don't understand which it actually uses to check the code, since I did not specify another one.)
So I end up with a lot of import errors and no autocompletion, e.g.
Import "numpy" could not be resolvedPylance (reportMissingImports),
while the script executes without any issue.

Can you provide trace logs so we can see what files are being accessed and their schemes? See: https://github.com/microsoft/pylance-release/blob/master/TROUBLESHOOTING.md#filing-an-issue (also in the bug template)
I'm not sure how singularity works, but one of the problems with "remotes" in VS Code is that they don't actually run the language server on the remote host, instead creating fake filesystems that are only accessible in the editor. It could be similar here where Pylance is running on the host and can't access the actual environment.

Some more information:
Pylance Version: v2020.8.3
Some more specifics for the remote setup:
settings.json:
{     "terminal.integrated.shell.linux": "./get_worker.sh",     "python.pythonPath": "./singularity_python.sh",     "python.analysis.logLevel": "Trace",     "python.linting.pycodestyleEnabled": true,     "python.linting.enabled": true, },
where ./get_worker.sh requests a new shell on a compute node.
and ./singularity_python.sh, specifies the python interpreter from a singularity containter running in the compute node:
singularity exec --bind . --pwd $PWD  docker://some-python-image python "$@"
Which output tab from the "Python Language Server":
[Info  - 8:55:42 AM] Pylance language server 2020.8.3 (pyright 8c5415fd) starting [Info  - 8:55:42 AM] Server root directory: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server [Info  - 8:55:42 AM] No configuration file found. [Info  - 8:55:42 AM] Setting pythonPath for service "docker_remote": "*******/**********/singularity_python.sh" [Error - 8:55:43 AM] stubPath *******/**********/typings is not a valid directory. [Info  - 8:55:43 AM] Assuming Python version 3.8 [Info  - 8:55:43 AM] Assuming Python platform Linux [Info  - 8:55:43 AM] Searching for source files [Info  - 8:55:43 AM] Found 1 source file [Info  - 8:55:43 AM] Background analysis root directory: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server [Info  - 8:55:43 AM] Background analysis started [BG] analyzing: *******/**********/test.py ... [BG]   parsing: *******/**********/test.py (36ms) [BG]   parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/builtins.pyi (206ms) [BG]   binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/builtins.pyi (71ms) [BG]   binding: *******/**********/test.py (2ms) [BG]   checking: *******/**********/test.py ... [BG]     parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/sys.pyi (21ms) [BG]     binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/sys.pyi (5ms) [BG]     parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/os/__init__.pyi (64ms) [BG]     binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/os/__init__.pyi (25ms) [BG]     parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/typing.pyi (26ms) [BG]     binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/3/typing.pyi (26ms) [BG]     parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (17ms) [BG]     binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (6ms) [BG]   checking: *******/**********/test.py (238ms) [BG] analyzing: *******/**********/test.py (555ms) [FG] parsing: *******/**********/test.py (22ms) [FG] parsing: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/builtins.pyi (201ms) [FG] binding: *******/.vscode-server/extensions/ms-python.vscode-pylance-2020.8.3/server/typeshed-fallback/stdlib/2and3/builtins.pyi (67ms) [FG] binding: *******/**********/test.py (1ms)
I'm aware that this is a rather complicated setup (SSH Remote -> Compute Node -> Python interpreter running in a container), however, the script execution works like a charm, it is really the code analysis that is having trouble.

How can I see from the looks which python path/interpreter is actually checked?

You sorta cut out the most important part of the paths; the prefix. Do they start with sshfs:// or similar? That's what I'm looking for. I'm not sure that this is SSHFS, though, given it's using .vscode-server (which is a good sign that we should be working).
We pull that pythonPath config in order to run python, so that path needs to exist where the VS Code server is running, otherwise we may not end up working. I would expect other Python extension features to dependent on that; so there might be more insight in the Python output window.

I have the same problem and filed a question on stackoverflow. I initially had absolutely no code completion / docs popup working and after switching to Pylance (previously it was Microsoft), I had them working. But as mentioned in this issue, it couldn't recognize the pip modules inside the container.

Mine's a very similar setup too and here are my logs, I replaced the original dirs with homepath and projpath and left the rest just as it is:
[Info  - 6:11:32 PM] Pylance language server 2020.9.8 (pyright a0620921) starting
[Info  - 6:11:32 PM] Server root directory: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist
[Info  - 6:11:32 PM] No configuration file found.
[Info  - 6:11:32 PM] Setting pythonPath for service "test_pyproj": "/projpath/test_pyproj/python"
No search paths found for configured python interpreter.
[Error - 6:11:32 PM] stubPath /projpath/test_pyproj/typings is not a valid directory.
[Info  - 6:11:32 PM] Assuming Python version 3.7
[Info  - 6:11:32 PM] Assuming Python platform Linux
[Info  - 6:11:32 PM] Searching for source files
[Info  - 6:11:32 PM] Found 1 source file
[Info  - 6:11:32 PM] Background analysis(1) root directory: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist
[Info  - 6:11:32 PM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: setTrackedFiles
[FG] parsing: /projpath/test_pyproj/test.py (12ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/builtins.pyi (210ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/builtins.pyi (67ms)
[FG] binding: /projpath/test_pyproj/test.py (1ms)
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: analyze
[BG(1)] analyzing: /projpath/test_pyproj/test.py ...
[BG(1)]   parsing: /projpath/test_pyproj/test.py (22ms)
[BG(1)]   parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/builtins.pyi (215ms)
[BG(1)]   binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/builtins.pyi (67ms)
[BG(1)]   binding: /projpath/test_pyproj/test.py (1ms)
[BG(1)]   checking: /projpath/test_pyproj/test.py ...
[BG(1)]     parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (12ms)
[BG(1)]     binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (3ms)
[BG(1)]     parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/third_party/2and3/typing_extensions.pyi (7ms)
[BG(1)]     binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/third_party/2and3/typing_extensions.pyi (2ms)
[BG(1)]     parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/typing.pyi (54ms)
[BG(1)]     binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/typing.pyi (17ms)
[BG(1)]   checking: /projpath/test_pyproj/test.py (120ms)
[BG(1)] analyzing: /projpath/test_pyproj/test.py (427ms)
Background analysis message: getSemanticTokens
Background analysis message: getSemanticTokens
Background analysis message: resumeAnalysis
IntelliCode model /homepath/.vscode-server/extensions/visualstudioexptteam.vscodeintellicode-1.2.10/cache/E61945A9A512ED5E1A3EE3F1A2365B88F8FE_E4E9EADA96734F01970E616FAB2FAC19
Loading ONNX runtime...
Loaded ONNX runtime. Creating IntelliCode session...
2020-10-07 18:11:34.354809256 [I:onnxruntime:, inference_session.cc:174 ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
2020-10-07 18:11:34.357154873 [I:onnxruntime:, inference_session.cc:840 Initialize] Initializing session.
2020-10-07 18:11:34.357366831 [I:onnxruntime:, inference_session.cc:865 Initialize] Adding default CPU execution provider.
2020-10-07 18:11:34.359518865 [I:onnxruntime:, reshape_fusion.cc:37 ApplyImpl] Total fused reshape node count: 0
2020-10-07 18:11:34.360469490 [I:onnxruntime:, reshape_fusion.cc:37 ApplyImpl] Total fused reshape node count: 0
2020-10-07 18:11:34.366590864 [V:onnxruntime:, inference_session.cc:679 TransformGraph] Node placements
2020-10-07 18:11:34.366716825 [V:onnxruntime:, inference_session.cc:681 TransformGraph] All nodes have been placed on [CPUExecutionProvider].
2020-10-07 18:11:34.366858635 [V:onnxruntime:, session_state.cc:71 CreateGraphInfo] SaveMLValueNameIndexMapping
2020-10-07 18:11:34.367015788 [V:onnxruntime:, session_state.cc:116 CreateGraphInfo] Done saving OrtValue mappings.
2020-10-07 18:11:34.368384271 [I:onnxruntime:, finalize_session_state.cc:173 SaveInitializedTensors] Saving initialized tensors.
2020-10-07 18:11:34.442090834 [I:onnxruntime:, finalize_session_state.cc:225 SaveInitializedTensors] Done saving initialized tensors
2020-10-07 18:11:34.446610849 [I:onnxruntime:, inference_session.cc:954 Initialize] Session successfully initialized.
Created IntelliCode session.
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: setFileOpened
Background analysis message: markFilesDirty
Background analysis message: getSemanticTokens
[BG(1)] parsing: /projpath/test_pyproj/test.py (8ms)
[BG(1)] binding: /projpath/test_pyproj/test.py (1ms)
Background analysis message: analyze
[BG(1)] analyzing: /projpath/test_pyproj/test.py ...
[BG(1)]   checking: /projpath/test_pyproj/test.py (2ms)
[BG(1)] analyzing: /projpath/test_pyproj/test.py (2ms)
Background analysis message: resumeAnalysis
Background analysis message: setFileOpened
Background analysis message: markFilesDirty
[FG] parsing: /projpath/test_pyproj/test.py (3ms)
[FG] binding: /projpath/test_pyproj/test.py (0ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (16ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/2and3/_typeshed/__init__.pyi (3ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/third_party/2and3/typing_extensions.pyi (5ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/third_party/2and3/typing_extensions.pyi (2ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/typing.pyi (61ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/typing.pyi (23ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/abc.pyi (6ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/abc.pyi (1ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/ast.pyi (19ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/ast.pyi ...
[FG]   parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/_ast.pyi (10ms)
[FG]   binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/_ast.pyi (7ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/ast.pyi (24ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/types.pyi (11ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/types.pyi (7ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/os/__init__.pyi (36ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/os/__init__.pyi (24ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/io.pyi (10ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/io.pyi (3ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/_importlib_modulespec.pyi (2ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/_importlib_modulespec.pyi (0ms)
[FG] parsing: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/posix.pyi (3ms)
[FG] binding: /homepath/.vscode-server/extensions/ms-python.vscode-pylance-2020.9.8/dist/typeshed-fallback/stdlib/3/posix.pyi (1ms)
IntelliCode: current invocation did not produce any meaningful tokens.
Time taken to get recommendations: 3 ms, Memory increase: 108 KB.
Background analysis message: analyze
[BG(1)] analyzing: /projpath/test_pyproj/test.py ...
[BG(1)]   parsing: /projpath/test_pyproj/test.py (1ms)
[BG(1)]   binding: /projpath/test_pyproj/test.py (0ms)
[BG(1)]   checking: /projpath/test_pyproj/test.py (4ms)
[BG(1)] analyzing: /projpath/test_pyproj/test.py (5ms)
Background analysis message: setFileOpened
Background analysis message: markFilesDirty
Background analysis message: getSemanticTokens
[BG(1)] parsing: /projpath/test_pyproj/test.py (1ms)
[BG(1)] binding: /projpath/test_pyproj/test.py (0ms)
Background analysis message: setFileOpened
Background analysis message: markFilesDirty
[FG] parsing: /projpath/test_pyproj/test.py (0ms)
[FG] binding: /projpath/test_pyproj/test.py (0ms)
IntelliCode: current invocation did not produce any meaningful tokens.
Time taken to get recommendations: 1 ms, Memory increase: 15 KB.
Background analysis message: getSemanticTokens
[BG(1)] parsing: /projpath/test_pyproj/test.py (1ms)
[BG(1)] binding: /projpath/test_pyproj/test.py (0ms)
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: analyze
[BG(1)] analyzing: /projpath/test_pyproj/test.py ...
[BG(1)]   checking: /projpath/test_pyproj/test.py (3ms)
[BG(1)] analyzing: /projpath/test_pyproj/test.py (3ms)
Background analysis message: resumeAnalysis
Background analysis message: setFileOpened
Background analysis message: markFilesDirty
[FG] parsing: /projpath/test_pyproj/test.py (0ms)
[FG] binding: /projpath/test_pyproj/test.py (0ms)
Background analysis message: analyze
[BG(1)] analyzing: /projpath/test_pyproj/test.py ...
[BG(1)]   parsing: /projpath/test_pyproj/test.py (1ms)
[BG(1)]   binding: /projpath/test_pyproj/test.py (0ms)
[BG(1)]   checking: /projpath/test_pyproj/test.py (2ms)
[BG(1)] analyzing: /projpath/test_pyproj/test.py (3ms)
Background analysis message: resumeAnalysis
Background analysis message: getSemanticTokens
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange

I have a similar issue to what has been reported so far. Although, I am simply importing a python file I created and not a module.

As with other's experience, the script runs fine regardless.

@mikeysan what comes to mind is a wrong import. the directory phoducation-bot is your root module, hence the config import should be phoducation-bot.config. it will still work at runtime, because python will discover something called config in your config root directory, but pylance expects things a bit differently, i.e. import config will look at a config package (folder or single file) in your virtualenv (should you use one), i.e. .venv/lib/python3.8/site-packages/config. Python module structure.
retweet_anything
|
+---phoducation-bot
|   |
|   +---module
|   |   |
|   |   +---__init_.py
|   |   +---foo.py
|   |
|   +---config.py
|
...

To import phoducation-bot/config.py in phoducation-bot/module/foo.py you must use from phoducation-bot.config import .... In your case, it's looking for config in the native or installed packages, of which there are none.

Thanks for the explanation @felixhammerl. I give your suggestion a try as soon as possible.

I think I was able to track down what went wrong.
Seems like pylance gets the interpreter path out of the singularity container and then tries to uses this path locally, see the bold log below.
However, I don't know how this could be fixed @jakebailey
`
User belongs to experiment group 'AlwaysDisplayTestExplorer - experiment'
User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'LocalZMQKernel - experiment'
User belongs to experiment group 'CollectLSRequestTiming - experiment'
User belongs to experiment group 'CollectNodeLSRequestTiming - experiment'
User belongs to experiment group 'EnableIPyWidgets - experiment'
User belongs to experiment group 'RunByLine - experiment'
User belongs to experiment group 'CustomEditorSupport - control'
User belongs to experiment group 'pythonaa'

conda --version
pyenv root
python3.7 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python3.6 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python3 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
python2 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
Error 2020-11-25 11:15:12: Failed to check if file needs to be fixed [EntryNotFound (FileSystemError): Unable to read file 'vscode-remote://ssh-remote+inhsshnode01/home/xxxxxx/.config/Code/User/settings.json' (EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, open '/home/xxxxxx/.config/Code/User/settings.json')
at _handleError (/home/xxxxxx/.vscode-server/bin/e5a624b788d92b8d34d1392e4c4d9789406efe8f/out/vs/server/remoteExtensionHostProcess.js:754:838)
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at async y.readText (/home/xxxxxx/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:9:152841)
at async p.doesFileNeedToBeFixed (/home/xxxxxx/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:469606)
at async /home/xxxxxx/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468733
at async Promise.all (index 0)
at async p.getFilesToBeFixed (/home/xxxxxx/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468679)
at async p.updateTestSettings (/home/xxxxxx/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468306)] {
code: 'FileNotFound',
name: 'EntryNotFound (FileSystemError)'
}
python ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
~/code/yyyyyyy/.vscode/singularity_python.sh ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
conda info --json
Error 2020-11-25 11:15:13: Detection of Python Interpreter for Command /home/xxxxxx/code/yyyyyyy/.vscode/singularity_python.sh and args  failed as file /usr/local/bin/python does not exist
conda env list
pyenv root
pyenv root
pyenv root
pyenv root
conda env list
pyenv root
pyenv root
pyenv root
Starting Pylance language server.
Python interpreter path: ./.vscode/singularity_python.sh
pyenv root
pyenv root
pyenv root
`

For Pylance to work, Pylance needs to be running where the code lives; this is how VSC's typical remote systems work (SSH, WSL) by running VS Code in server mode on the other system. Since we'd run there, the paths are valid. I don't know how Singularity works under the hood, but it'd need to do that as well as there's no abstraction for the file system that works with language servers (and it's expected that LSs read straight from disk). VS Code itself has fancy APIs to do that, but the LSP does not.

Closing issue. Please reopen if necessary.


Improve tabular kerning

Description
Eliminated some false positives in Ubuntu Sans related to digraphs.
A few other false positives still remain, don't get why currently.
Checklist

[ ] update CHANGELOG.md
[ ] wait for the tests to pass
[ ] request a review

I'm reviewing this now.

I see that you changed existing font binaries from the data/tests directory. We usually avoid that because it can break other code-tests, but in this case it is OK because the UbuntuSans files you changed are (at least for now) only used by this single code-test.


attach databricks compute via python sdk2

I recently upgraded my machine learning code from azure-core to azure-ai-ml. However, I couldn't find documentation or example code on how to attach Databricks compute using the Python SDK v2. I would appreciate it if the ability to attach Databricks compute could be added as a feature in the latest version. This would allow for a consistent Azure ML experience without the need to maintain both azure-core and azure-ai-ml dependencies. Thank you for considering this enhancement in future updates.

Thanks for reporting the issue, currently Azure Data brick compute is unsupported in SDK V2, I am adding it the internal list of asks and closing the ticket.


Is it possible to include the new query based rules in this language

It'd be awesome wi be able to use this feature while still using the Bolt language, any ideas about how to implement this feature?

Duplicate of #224


[Bug] Tags missing in GitHub

Reproducible in vscode.dev or in VS Code Desktop?

[X] Not reproducible in vscode.dev or VS Code Desktop

Reproducible in the monaco editor playground?

[X] Not reproducible in the monaco editor playground

Monaco Editor Playground Code
/

Reproduction Steps
See changelog: https://github.com/microsoft/monaco-editor/blob/main/CHANGELOG.md
Actual (Problematic) Behavior
It looks like the latest created tag is 0.32.0. Several newer versions have been published to npm since and the changelog is listing each new version.
Is it possible to create the respective missing tags for each new versions? It's a requirement for us to being able to point a version to a specific tag in order for us to use it.
Expected Behavior
Each new version has its own tag associated.
Additional Context
/

@alexdima @hediet: Please help us to resolve, maybe ping the proper person? Thank you!

I just added the missing tags. We also publish tags now automatically.


Beta Release

Introduction
For this release we integrated several new features as outlined in our Sprint 4 Deliverables.
Comprehensive Functionality Overview:

removal of block menu
introduction of new swim
Creation of Main Menu interface with three buttons to: create new swim lane, play back audio, or remove looping lane
By default application starts with one swim lane loop
ability to upload audio blocks
ability to remove audio blocks
ability to adjust iterations and interval for blocks

Notes:
Due to limitations with selected technology, the interface required a significant overhaul. As of right now there is still refactoring that needs to be completed

looks good


Update opensuse.md

additional Infos:
https://en.opensuse.org/Lifetime
https://en.opensuse.org/Portal:15.4

Thanks a lot for the update @itssct 🥇


Update README.md

updated video link in "Links" section
typos
added small description about loading files in UI

Looks good, thanks Rich.


Add ggplot2 package yes/no (test pull requests)

This package is not necessary in this script (test pull requests)


ENH: Adding simulated annealing method (and related benchmark update)

Dear All,
A PR proposing the new simulated annealing including code additions in the benchmark section.
Thanks a lot !

Original PR is #6197

Do you have the results of the benchmarks, e.g. a hundred trials? Can you put those on a gist, or similar?

For me to review this PR I'm going to have to read the original articles because I find the code isn't all that self explanatory.

For me to review this PR I'm going to have to read the original articles because I find the code isn't all that self explanatory.

I agree, it's quite hard to follow because there are a lot of internal variables, long functions, too few blank lines and almost no comments. It would be really helpful if there would be comments with equations, ideally referring to a paper that the implemention is based on.

@andyfaff: Thanks a lot for your comments. I started addressing some of them in the last commit and I am working on the others comments.

@andyaff, @rgommers: thanks a lot for your comments, I think that I have addressed all of them.
[X] remove profile line (x2)
[X] For the parameter list documentation I would just copy from another minimizer, for familiarity. The following is from differential_evolution.
[X] space after the colon (x2)
[X] Bounds for variables. (min, max) ...
[X] args : tuple, optional...
[X] There are no guides or notes on how to choose this value, or visitparam, or acceptparam...
[X] If bounds are required shouldn't the method be LBFGSB?
[X] I don't think there is a need to say it was originally implemented in C++/R... Reference 2 has to be used
[X] Two blank lines are probably required for PEP8. pep8 is now passing successuly
[X] assert can get optimised out of code. Use...
[X] Probably not necessary to create a custom exception. Can't we just use ValueError, etc?
[X] There don't need to be any changes made to asv.conf.json, especially not your specific directories.
[]  Do you have the results of the benchmarks, e.g. a hundred trials? Can you put those on a gist, or similar? I'll do it ASAP and comment on it
[X] This is a big multiply nested loop. Is there any chance of factorising / simplifying...
[X] Is it necessary that this attribute (and several others) are private?
[X] Don't set private variables like this. These variables in L577-> L583...
[X] It's not a dict, it should be a tuple.
[X] Why args in gensa, but fargs here? Can we use identical naming?
[X] Consider having a single bounds variable, accepting (min, max)...
[X] with all these types of variables use init_error, in_constraint instead..
[X] vectorise loops (x2)
[X] here doesn't seem todo be a stopping criterion that can be set here... maxtime, niter, maxcall are stopping conditions. If the user
wants to stop on reaching the known solution, know_real and real_threshold public attribures of the GenSARunner have to be used
[X] There's no way of setting a random seed for reproducibility. Random state is now use in GenSARunner
[X] ave a look at how basinhopping and differential_evolution implement it.
[X] add test that checks that gensa still works if any of the upper bounds are less than the lower bounds
[X] add test that checks that setting a random seed gives reproducible minimization.
[X] Can you vectorize this loop? Yes
@rgommers  comments:
[X] I agree, it's quite hard to follow because there are a lot of internal variables, long functions, ... Comments added, equations and explaination added in notes
[X] What we currently consider the best way of doing this is using a random_state=None Addressed in with @andyfaff  comments

This now needs a rebase, there's a conflict in the benchmarks.

@andyfaff , @rgommers : I have created a gist with the benchmark results tables and attached the poster presented at EuroSciPy 2016. Thanks a lot.
https://gist.github.com/sgubianpm/7d55f8d3ba5c9de4e9f0f1ffff1aa6cf
EuroSciPy-2016.pdf

@andyfaff, @rgommers
We are about to submit a paper about extensive benchmarks in Python of global optimization methods. It includes SciPy ones and some others (like PSO). I know that you guys are very busy with the big chunk of PRs. I have addressed all of you comments/suggestions. Do you think you would foresee a merge for this PR. Thanks a lot for you time.

If you use lbfgs for the local search, does that mean that the method will only really work with smooth functions?  If so you might want to add that to the documentation.

@js850 Thanks for your question! In principle, pure generalized simulated annealing works for smooth and non-smooth problems, so our algorithm should cover both types of problem. However, current version doesn't give users the option to choose different local search algorithm (i.e. Nelder-Mead), which will be implemented in a future version, as we have done in the R version. For current version, the objective function has to be smooth in principle, however, we found that it gives reasonable results for non-smooth functions depending on how non-smooth this function is. Please refer to the financial example in the R Journal paper

@js850 Thanks a lot for your comments. I have addressed them in the recent commits.

@js850 thanks a lot for your additional comments. I will address them and let you know when done.

A new PR created for SDA.


Add support for custom auth backend path

Current implementation only supports one kubernetes backend in vault, registered in default path such as this:
vault auth enable kubernetes
#aka
vault auth enable -path="kubernetes" kubernetes

To support multiple kubernetes clusters in one vault, multiple auth backends can be configured as follows:
vault auth enable -path="kubernetes/cluster-one" kubernetes
vault auth enable -path="kubernetes/cluster-two" kubernetes

This pull request allows specifying an additional KUBERNETES_VAULT_PATH env variable which defaults to kubernetes.
To configure the example clusters, one would set KUBERNETES_VAULT_PATH=kubernetes/cluster-one and KUBERNETES_VAULT_PATH=kubernetes/cluster-two respectively.

It currently fails when the env is undefined

Could not resolve placeholder 'kubernetes.vault.path' in value "${kubernetes.vault.path}"

Could you please advise if there is any annotation to pass in a default value instead?

I've fixed the vault path being required by using the builtin defaults in Spring Annotations.

LGTM

Hi @Mikulas,
thank you very much for the pull request, I'll create a new release for it in few minutes.

https://github.com/DaspawnW/vault-crd/releases/tag/v1.2.2

Awesome, thank you :)


memory.swappiness may not be writeable on some kernels

PR #622 adding the capability for writing to the memory.swappiness setting within a cgroup.  Because of the check (>= 0 and <= 100), even if the cgroup object has no intended setting, by default, I believe it will be 0 (which is a Go default for int64, and, if using from Docker, there is no initial setting in docker's "execdriver" template), and therefore, every container start will try and write to the cgroup sysfs entry.
This has two problems: 1) I'm going to assume that the intent of adding the behavior was not to always set swappiness to zero on every container/cgroup (which is I think what happens right now), and 2) setting memory.swappiness in hierarchical cgroups was disabled (you get EINVAL) until this patch: torvalds/linux@3dae7fec5e884a4e72e5416db0894de66f586201 which appears to be in 3.16 and newer kernels.
On Ubuntu 14.04.02 LTS for example, using Docker with an updated libcontainer, even without the "UI" elements for interacting with memory.swappiness, every container start fails because of the cgroup write EINVAL result from trying to write zero to memory.swappiness on that older kernel.
A fix for problem 1 would be to somehow default an "unset" MemorySwappiness to -1 so writes aren't inadvertently done every time with a zero value.  I think we need to discuss problem 2 as far as either graceful fallback on failure with EINVAL or removing the feature until it is available in the lowest supported kernel.

-1 for 1) sounds good.
For the second part, I think libcontainer should return an error if a user asks for swappiness when its not available. Docker daemon can ignore requests when swappiness is not enabled, just the way it does for mem swap.

WDYT about changing the default type for cgroups config value to *int64. It's a bit more troublesome to consume, but it allows us to properly detect whether values are set by user.

For 2), I'm not sure how we can properly detect this for older kernel. If 1) is fixed, then i think that failing to set user-defined swappiness value is a fatal error and the container should not be started (i.e., there's no approriate fallback)

@estesp @dqminh @cpuguy83
It is correct that we do not want to set the mem swappiness value to zero for everybody.
Assuming that failing to set the swappiness value when user has asked to do so..
we have
case A: explicit setting of memory swappiness value on a supported kernel
Container does not start
case B: explicit setting of memory swappiness on a  unsupported kernel
Container does not start
case C: Implicit setting of memory swappiness value on a supported kernel
This should not be done.
case D: Implicit setting of memory swappiness value on a unsupported kernel.
once case C is fixed D is not a problem
How to handle (D and container starts normally) if (C) is not fixed.
We could check (if root memory swappiness exists and root use_hierarchy is set) then
failure in trying to set implicit value should be allowed.
@estesp with https://github.com/docker/docker/pull/14030, do we avoid setting value of swappiness and fix (C) ?

@ktraghavendra yes, the Docker PR means there will be no inadvertent write to swappiness until a PR is merged which allows the user to specify swappiness (your "UI PR" as I called it).  Once that PR is merged, then the only failures on the Docker side will be when a user explicitly tries to set a swappiness value, but the kernel is old enough to predate the loosened restriction on cgroup hierarchy use + swappiness writes.
That case (failure due to older kernel) we could handle only on the Docker side (similar to what we do when OOM and memory restrictions don't work because those features aren't turned on via kernel command line, etc.
The only case that remains that isn't solved is regarding a "new" libcontainer consumer (outside of nsinit and docker).  Using the "out of the box" implementation for configs, that consumer of libcontainer will get automatic writes attempted with swappiness = 0 unless they have a "template init" style that sets up a better default value that matches the 2 PRs I've submitted for docker and nsinit.
I believe this is where @dqminh is suggested possibly moving to an *int64 as it will require explicit initialization to determine that a write is requested.

@estesp @dqminh , Thanks .. yes moving to *int64 is a good option. So in libcontainer we do not return error if point is still a null. Will work on that patch. From that we do not have to even check the older kernel failure stuff.


Byebug 11 performance decreases when breakpoint set

Problem description
Byebug 11.0.1 slows down code drastically when a breakpoint is set.
Expected behavior
I expect it to slow down the code somewhat - 2x or even 3x.
Actual behavior
It slows down much much more than that.
Steps to reproduce the problem
# test.rb
require 'benchmark'
puts Benchmark.measure {
  big = []
  10000000.times do |i|
    big << rand(36**16).to_s(36)
  end
}

Baseline ruby test:
ruby test.rb
14.670000   0.670000  15.340000 ( 19.430349)

Baseline byebug test (no breakpoints):
byebug test.rb
=> 1: require 'benchmark'
...
(byebug) c
 15.000000   0.210000  15.210000 ( 15.215282)

Byebug test with added breakpoint:
byebug test.rb
=> 1: require 'benchmark'
...
(byebug) b 3
(byebug) c
=> 3:   big = []
   4:   10000000.times do |i|
...
(byebug) c
110.460000   0.590000 111.050000 (111.636692)

So about a 7x slowdown just by setting a breakpoint. If you remove the breakpoint after you hit it, things are fine.  So the workaround is to clear your breakpoints before you do the final continue.
I first noticed this in a Rails app, where leaving a breakpoint set after a request causes every other request to run a very long time. I just mention that to show I don't think it's an artifact of this kind of calculation, I think it just slows down any Ruby code by a factor of 7+.
I tested with both Ruby 2.4.4 and 2.5.1. I also tested with byebug going back to 6 and all exhibited the issue. Anecdotally, I've been using byebug for years and just started noticing this problem so it could be related to the newer Ruby versions. I will try to find time to go back and check 2.3 and earlier.
Thanks for looking into this.

Hi @bdeterling.
This is expected. After you run continue, byebug needs to check whether to stop or not for every line run by your program. This has an overhead, unfortunately. If you don't have any breakpoints enabled, there's nothing to check, so byebug optimizes that particular case but disabling itself before letting the program run.

Ok, that does make sense, but just to make sure I wasn't crazy I went all the way back to Ruby 2.0 and byebug 2.0 and found that byebug used to slow the process down by about 4x, but it didn't matter whether a breakpoint was set or not. The current situation is better: <2x slowdown without breakpoints and 7x slowdown with breakpoints versus the old 4x slowdown either way.
Perhaps some day there could be a run to end command that clears breakpoints and then continues. It's not a big deal to type del/y, but it's a pain when you forget. Or maybe a ctrl-something to stop at the current line so you can delete them when you forget.
Thanks for the response.

Ok, that does make sense, but just to make sure I wasn't crazy I went all the way back to Ruby 2.0 and byebug 2.0 and found that byebug used to slow the process down by about 4x, but it didn't matter whether a breakpoint was set or not. The current situation is better: <2x slowdown without breakpoints and 7x slowdown with breakpoints versus the old 4x slowdown either way.

That's helpful to know. I've seen this anecdotally, but it's helpful to know someone's confirmed it's the case and got some rough numbers on the difference. Thanks @bdeterling. Before reading this I wasn't sure of the distinction in performance between adding a byebug statement, and manually adding breakpoints during execution.


Union Conversion/Type Checking

A small project that will let us 1) move the unsafe union conversion out of the dispatcher and into the interface with other unsafe code 2) will give us another layer of sanity checks on the types passed in by the union RPC
You should make a new file in the interface folder called types.rs. In here will be a bunch of functions (1 per type) that takes in the union as a parameter, converts to the correct type via an unsafe block, does some sanity checks on what types have been passed, and returns the variable for use in safeposix.
@TristanB22 As far as what checking is done, you should brainstorm a bit and you can bounce some ideas off of us. You can probably get some ideas by going through the Repy source code in files like namespace.py to see what was being done there.

This was merged and is quite good!
We still need to untangle the dependencies a bit though. Let's try to make this not import stuff from "safeposix" into the interface.

No, this is good now.


upgrade scala 2.13.5 remove unused boundaries

probably should be on 21 but appeasing mama is a horrorshow

I tried a cherry-pick, and it already doesn't pick cleanly.  We can just leave that branch stuck on 2.13.4.


Enable enforcing turn transport protocol in p2p

Description
Summary:

Related Issue:

Testing

Screenshots/GIFs (if applicable)

Checklist

[x] My code follows the project's coding standards.
[x] Prefixed the PR title and commit messages with the service or package name
[ ] I have written unit tests (if applicable).
[ ] I have updated the documentation (if applicable).
[x] By submitting this pull request, I confirm that my contribution is made
under the terms of the MIT license.

Additional Information

/canary

/canary

/canary


Fix capitalization

What's changed, or what was fixed?

Capitalize first word of sentence

Fixes: None
Target Live Date: TBA

[ ] This has been reviewed and approved by (NAME)
[ ] I have run npm test locally and all tests pass.
[ ] I have added the appropriate type-something label.
[ ] I've staged the site and manually verified that my content displays correctly.

CC: @petele

I signed it!

:tada: This has been pushed live to https://developers.google.com/web/


Update azure-to-azure-tutorial-enable-replication.md

Fixed typo "manager" to "manage" in permissions section.

@joeemt : Thanks for your contribution! The author, @rayne-wiselman, has been notified to review your proposed change.

@joeemt Thanks! #sign-off


[DEV] Registry contract for MultiSig

as an "good-to-have" purpose

[ ] who can add the contract to the registry ?


Add type stubs

This adds type hints to .pyi files to avoid changing/cluttering up the source files.
I've gone through the paho code to verify the types used, but not tested it beyond running mypy --strict asyncio_mqtt.

Hi Ellis. Thanks for your pull request. Let me have a look. :)
Great to see some typing. :+1: I wanted to add this at one point or another myself. As for the types themselves, I have no objections. It looks good.
I'm wondering if we should simply inline the types instead of using stub files. Minimum Python version is 3.7 so that isn't a barrier. Likewise, we can use the built-in typing module.
What do you think? I know that you mention "avoid changing/cluttering" as the main reason. Are there other reasons for this preference?
Again, thanks for this pull request.

I actually prefer doing it inline, as you don't need to keep two files in sync. I can move them over when I get a moment. I might also add hints for the private methods too, since it's helpful for development on this project as well as for projects using this client.

I actually prefer doing it inline, as you don't need to keep two files in sync.

I agree. :)

I can move them over when I get a moment. I might also add hints for the private methods too, since it's helpful for development on this project as well as for projects using this client.

I would really appreciate that, thank you. :+1:

I can merge this right away. Let me know if you prefer that, or, if you prefer to change it into inline typings first.

I'll change into inline tomorrow hopefully, so don't bother merging just yet. Cheers

With a large change like this, it's super helpful to use auto-formatting (I can't imagine working without it any more). It may be a separate issue to raise, but what do you feel about using https://github.com/psf/black to format the code?
I'll make a draft PR in a minute to show you what the changes would be when using black to format the codebase.

It's a great idea. :+1: See my reply in #43 for details.
I feel that we align on all these recent issues. Do you want to co-maintain asyncio-mqtt with me? This way, you can freely make changes/improvements without delay. It's still the early days (pre v1.0.0), so there's plenty of opportunity to put a personal touch on the project (give you a feeling of ownership). :)

I feel that we align on all these recent issues

Yeah, it's slightly uncanny!

Do you want to co-maintain asyncio-mqtt with me?

Sure! Thanks :grin: . I was part-way through writing my own wrapper around paho-mqtt before I found this one. I thought https://github.com/mossblaser/aiomqtt was the only one, and I thought (hoped, ignorantly) that I might be able to make something that integrates paho more closely with the asyncio event loop. Then I saw that you had beaten me to it!
I can't promise to always be engaged with the project; I tend to dip in and out of things a bit, but I'd be happy to help push it along.

I had a real close look at aiomqtt (and the other alternative) as well before I created asyncio-mqtt. Great minds think alike. :))

I can't promise to always be engaged with the project; I tend to dip in and out of things a bit, but I'd be happy to help push it along.

No worries; I do the same. Put in the time that you can afford and no more.

You can merge this PR yourself now when you see fit. :+1:

Cool, thanks!
I wonder how you feel about enabling the new GitHub Discussions feature? I've just done it on one of my repos and it seems like a nice way to discuss things without making a bunch of issues.

I never tried GitHub Discussions but let's give it a go. :) I enabled it.

I've added inline type hints throughout, and split up the _pending_calls list into:
self._pending_subscribes: Dict[int, "asyncio.Future[int]"] = {}
self._pending_unsubscribes: Dict[int, asyncio.Event] = {}
self._pending_publishes: Dict[int, asyncio.Event] = {}

so that we don't have Dicts with different types of Futures and Events in. The correct dict gets passed in to _pending_call() so that the Future/Event can be added and removed.
I added a property that returns a set of mids that have pending calls, so that we're still able to get a count of them and check if the mid's already got a pending call:
    @property
    def _pending_calls(self) -> Set[int]:
        """
        Return a set of all message IDs with pending calls.
        """
        mids: Set[int] = set()
        mids.update(self._pending_subscribes.keys())
        mids.update(self._pending_unsubscribes.keys())
        mids.update(self._pending_publishes.keys())
        return mids

_cb_and_generator() was a bit tricky to follow through, but I think I managed to end up with the right types for everything!
I know I can merge myself, but I'll leave it for you to review beforehand, in case there's something you want to discuss.

Thanks for the huge effort on this one. 👍 👍
Really nice to have types throughout the API (and internally). That'll make maintenance way easier going forward. :)


fix bad performance of resolve queueable entity

Illuminate Str startsWith function uses mb_strpos which has a very bad performance when the haystack is huge. When the size of haystack is larger than 100M it will take more than 5 mins. There no need use mb_strpos, I change it to strpos can improve performance significantly.

5.2 :scream: is no longer supported. You can send a PR to either 5.5 or master

5.5 is not implemented in this way. I'm closing this request 😞

There is a need for mb_strpos() because there are many languages out there that require the use of multibyte strings. I don't know which PHP version you're using but I guess you can improve this by just using a recent stable release. 👍


Numeric sort isn't intelligent. Sorts to "100, 25, 50, 75"

I've got symbols for, say, "Avatars", named after their sizes: "25, 50, 75, 100", but the organizer puts them in order as 100, 25, 50, 75, which is pretty irksome for someone who's gone and installed a plugin to organize their symbols 😝

@aaronmw Thanks for bringing this to my attention, should be resolved in v14.0 which was just published - https://github.com/sonburn/symbol-organizer/releases/tag/14.0.

Closing the issue as the issue is resolved.


Fixed bug, duplicated cms links in sitemap file on prestashop multistore.

Thank you for your report! This issue is fixed here: https://github.com/PrestaShop/gsitemap/pull/53


95% of what? (When average working set memory usage per container is greater than 95%)

Can you explain what the average working set memory per container is compared to? It says greater than 95%, but I don't understand 95% of what?
95% of the overall VM node memory, e.g. if a node has 8Gb, then a container must reach 7,9x G?
95% of predefined requests/limits in the pod definition? What about containers without such specified?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f09aeae6-06a7-d9e6-36ed-f26b91ef4df8
Version Independent ID: 786ddb6f-70a2-2c9c-37b9-21cfeaf3a22e
Content: Metric alerts from Container insights - Azure Monitor
Content Source: articles/azure-monitor/containers/container-insights-metric-alerts.md
Service: azure-monitor
Sub-service: containers
GitHub Login: @bwren
Microsoft Alias: bwren

@deyanp Thanks for your comment. We will review the issue and get back to you shortly.

@bwren Can you please provide your insights on this issue?

any clue?

Hi, @deyanp , In case of container metrics, compared against the limits of the container. If there is no limit specified for the container, then metric will be compared against the capacity of the node where the container running since container can take up to the Node's capacity.  Best practice to define the limits on the containers.

#please-close


[WFLY-8113 10.x] Upgrade netty-xnio-transport to 0.1.2.Final

Related to WFLY-7517, WFLY-8113, JBEAP-8827.

@jmesnil Please approve. This is for the 10.x branch.


fix accessibility issue caused by not using table

This PR fixes #9345
previously the elements are organized like a table but not really using table element, it caused issues. the fix is to put it inside a real table to solve the problem, also i am adding aria-label for all the text inputs.
@Charles-Gagnon and @llali , i am removing the default color related styles in the declarativeTable component css, I don't need these styles and if i want to overwrite it, i can only make it work with one theme. i believe you both ran into similar issues. please let me know if you have any concerns.

@llali and @Charles-Gagnon
before

after


kubeless CLI should respect current implicit namespace

When kubectl is invoked without an explicit "--namespace" option, it respects the current namespace setting in current kubeconfig context.
So does Openshift's oc command which respect current project/namespace.
This saves a lot of typing, and less surprise, kubeless CLI should do this too.
More information:
https://github.com/ahmetb/kubectx
https://github.com/kubernetes/kubernetes/issues/40968

👍 I agree


post create hook not work in docker

I created a directory and it name is hooks in project root

and this is my dockerfile
FROM *******/dockerhub/golang:1.17.6-alpine AS builder
WORKDIR /go/src/github.com/tus/tusd

# Add gcc and libc-dev early so it is cached
RUN set -xe \
	&& apk add --no-cache gcc libc-dev

# Install dependencies earlier so they are cached between builds
COPY go.mod go.sum ./
RUN set -xe \
	&& go mod download

# Copy the source code, because directories are special, there are separate layers
COPY cmd/ ./cmd/
COPY internal/ ./internal/
COPY pkg/ ./pkg/
COPY hooks/ ./hooks/

# Get the version name and git commit as a build argument
ARG GIT_VERSION
ARG GIT_COMMIT

RUN set -xe \
	&& GOOS=linux GOARCH=amd64 go build \
        -ldflags="-X github.com/tus/tusd/cmd/tusd/cli.VersionName=${GIT_VERSION} -X github.com/tus/tusd/cmd/tusd/cli.GitCommit=${GIT_COMMIT} -X 'github.com/tus/tusd/cmd/tusd/cli.BuildDate=$(date --utc)'" \
        -o /go/bin/tusd ./cmd/tusd/main.go

# start a new stage that copies in the binary built in the previous stage
FROM ******/dockerhub/alpine:3.15.0
WORKDIR /srv/tusd-data

RUN apk add --no-cache ca-certificates jq \
    && addgroup -g 1000 tusd \
    && adduser -u 1000 -G tusd -s /bin/sh -D tusd \
    && mkdir -p /srv/tusd-hooks \
    && chown tusd:tusd /srv/tusd-data

COPY --from=builder /go/bin/tusd /usr/local/bin/tusd
COPY hooks/ /srv/tusd-hooks

EXPOSE 8010
USER tusd

ENTRYPOINT ["tusd"]
CMD [ "--hooks-dir", "/srv/tusd-hooks" ]

When i run this command :  go run cmd/tusd/main.go  --hooks-dir ROOT_PROJECT/hooks everything works correctly.
but in docker hooks not work.
need help for this.

here is tusd-hooks directory

when i call the api i get 201 response:

and in logs :

How do you know the pre-create hook is not working? Can you elaborate? I don't see any errors.

Closing this issue due to inactivity. Feel free to leave a comment if you want to continue the discussion :)


Return grouping of contacts

In old version of skype I had some groups of contacts with 10-100 contacts in each. In new version grouping was deleted and see all some hundreds of contacts in one list. This is terrible. I can't filtering it and I don't remember from which group some contacts. Please return this functionality!

@Alexandr63 Thanks for your feedback. Please try the Skype developer community on Stack Overflow (https://stackoverflow.com/questions/tagged/skype) where you can get help from actual developers. If you find a bug in our documentation, please let us know. Thanks!


Source Array Help

Hi,
Can anyone help with an example of how to use an array for multiple sources?
I'd like to add multiple iCloud album sources to my config file but struggling on the syntax.
Thanks!

Something like this should work:
modules: [
  {
    module: "MMM-Wallpaper",
    position: "fullscreen_below",
    config: { // See "Configuration options" for more information.
      source: [
        "icloud:12345678",
        "icloud:87654321",
      ],
      slideInterval: 60 * 1000 // Change slides every minute
    }
  }
]```

Great thanks!


insim_core / insim: string conversion assumes everything should be escaped

I've made the exact same mistake that Martin raised in node-insim yesterday in the LFS Discord.
Which is the exact same mistake I made in https://github.com/theangryangel/XI4N years ago 😨

https://github.com/simbroadcasts/node-insim/issues/24
https://github.com/simbroadcasts/node-insim/issues/22
https://github.com/simbroadcasts/node-insim/pull/25

Options:

Do not escape and leave that to the user of the library, but give them a really good unescaping/escaping method (preferred)
Add a wrapping type around string (we dropped these for better developer erognomics, I'd prefer not resurrect these, unless we can do some clever generics gynastics where the user of the library can provide either)
Some smart escaping/unescaping based on detecting the field or the content (doomed for failure probably)
Some magic context thingy that
Something else?

Opted for 100% manual unescape and escape :)


assign product types to sale or voucher

hi.
sorry for my poor English
i want to assign vouchers or sales to product type!
what should i do?
i think in saleor we should create a range and assign to the voucher.
range can inclouded product types , categories , collections ,....etc

Hey @javadfarah! Thank you for your suggestion. It's not possible to assign product types to sales or vouchers. You can find more information about creating the vouchers here: https://docs.saleor.io/docs/dashboard/discounts/vouchers/#docsNav
If you have any questions, you can also reach us at Gitter or Spectrum:
https://gitter.im/mirumee/saleor
https://spectrum.chat/saleor

hi @karolkielecki thanks for your reply.
i solved my issue !
add new field to the sale model.
types = models.ManyToManyField("product.ProductType", blank=True)
and just eddited _fetch_products in utils.py:
    types = Sale.types.through.objects.filter(producttype_id__in=sale_pks).values_list(
        "sale_id", "producttype_id"
    )
    for sale_id , producttype_id in types:
        Products_ = Product.objects.filter(product_type__pk=producttype_id).values_list(
        "pk"
    )
        for pk in Products_:
            product_map[sale_id].add(pk[0])

if there is any problem and mistake or any security issue.please tell me:)
regards.


Adding srtp1

This formula was copied from homebrew-core at
e3d6d868156ccc9d7592db3206d165712710d8da:Formula/srtp.rb

@BrewTestBot test this please

We need a test added. Ideally, can a test be added to do something more substantial than e.g. --version or --help? See cmake.rb for an example of an application formula with a good test and tinyxml2.rb for an example of a library formula with a good test. Thanks!

Test added; bottles removed; renamed to srtp15.

@MikeMcQuaid Look good now?

Thanks for your contribution to Homebrew!  You rock!


Wsc master

update to sound v0.1.1-alpha.2

something went wrong as I get oriented to jumping around github accounts/organisations.


reject duplicate static mut variables

after #140 landed the entry, exception and interrupt attributes started
accepting code like this:
 #[entry]
fn main() -> ! {
    static mut FOO: u32 = 0;
    static mut FOO: i32 = 0;
}

because that code expands into:
fn main() -> ! {
    let FOO: &'static mut u32 = unsafe {
        static mut FOO: u32 = 0;
        &mut FOO
    };
    // shadows previous variable
    let FOO: &'static mut u32 = unsafe {
        static mut FOO: i32 = 0;
        &mut FOO
    };
}

this commit adds a check that rejects static muts with duplicated names to
these three attributes.

bors r+