[jvm-packages] Issue running XGBoost Spark example

For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.
Environment info
Operating System: OS X El Capitan
Compiler: Apple LLVM version 7.0.2 (clang-700.1.81)
Package used (python/R/jvm/C++): jvm
xgboost version used: 0.7
'spark' - 2.0.0
'scala' - 2.10.6
If installing from source, please provide

The commit hash (git rev-parse HEAD)
Logs will be helpful (If logs are large, please upload as attachment).

If you are using python package, please provide

The python version and distribution
The command to install xgboost if you are not installing from source

If you are using R package, please provide

The R sessionInfo()
The command to install xgboost if you are not installing from source

Reposting the question here as this is
Steps to reproduce

Compile and install of jvm-package
Running the XGBoost-Spark example

What have you tried?
I am encountering a type mismatch problem at the following step when running the XGBoost-Spark
val model = XGBoost.train(trainData3, paramMap, round=2, nWorkers=2, useExternalMemory=true, missing=0.0f )
error: type mismatch org.apache.spark.rdd.RDD[org.apache.spark.mllib.regression.LabeledPoint] required: org.apache.spark.rdd.RDD[org.apache.spark.ml.feature.LabeledPoint]
It seems that val trainData3 = MLUtils.loadLibSVMFile(sc, inputTrainPath) earlier produces a mllib.regression.LabeledPoint rather than a ml.feature.LabelPoint that XGBoost expects.
To test my overall setup, I manually created a ml.feature.LabelPoint via
val trainRDD = sc.parallelize(Seq(
|   LabeledPoint(1.0, new DenseVector(Array(2.0, 3.0, 4.0))),
|   LabeledPoint(0.0, new DenseVector(Array(5.0, 5.0, 5.0))),
...
),4)
This as input to xgboost.train runs without any issues. trainRDD is a org.apache.spark.ml.feature.LabeledPoint
Any thoughts on why I am seeing this type mismatch? thanks!

wait....you mean you can compile the examples but cannot run with a type mismatch issue?

yes.
On Sunday, February 26, 2017 10:58 AM, Nan Zhu<EMAIL_ADDRESS>wrote:

wait....you mean you can compile the examples but cannot run with a type mismatch issue?—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

can you post the complete log trace with the error? It is impossible for the scala compiler to let a program with type mismatch error go

Hi Nan,
Thanks for your response.
I misspoke about the compile error. Sorry.   I do have compiler errors.
Error:(38, 31) type mismatch; found   : org.apache.spark.rdd.RDD[org.apache.spark.mllib.regression.LabeledPoint] required: org.apache.spark.rdd.RDD[org.apache.spark.ml.feature.LabeledPoint]    val model = XGBoost.train(trainRDD, paramMap, numRound, nWorkers = 5, useExternalMemory = true)
There are also errors stating nWorkers, useExternalMemory and saveModelToHadoop unresolved. I am wondering whether I missed out something in my XGBoost compile ? (scala 2-10.6, spark_2.11-2.0.0 xgboost4j-spark-0.7, Apple LLVM version 7.0.2 (clang-700.1.81)
I ran the same code in a spark-shell.  There again, I was able to see the type mismatch error. However, when I changed the trainRDD to below (type ml.feature.LabeledPoint)  it ran without any issue.
val trainRDD = sc.parallelize(Seq(
|   LabeledPoint(1.0, new DenseVector(Array(2.0, 3.0, 4.0))),
|   LabeledPoint(0.0, new DenseVector(Array(5.0, 5.0, 5.0))),
...
),4)
Any thoughts/suggestions you can provide much appreciated.
thanks--priyan
import ml.dmlc.xgboost4j.scala.Booster
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.mllib.util.MLUtils
import ml.dmlc.xgboost4j.scala.spark.XGBoost
object XGboostSparkHello {
def main(args: Array[String]): Unit = {
if (args.length != 3) {
println(
"usage: program  num_of_rounds training_path model_path")
sys.exit(1)
}
// if you do not want to use KryoSerializer in Spark, you can ignore the related configuration
val sparkConf = new SparkConf().setMaster("local[*]").setAppName("XGBoost-spark-example")
.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
sparkConf.registerKryoClasses(Array(classOf[Booster]))
val sc = new SparkContext(sparkConf)
val inputTrainPath = args(1)
val outputModelPath = args(2)
// number of iterations
val numRound = args(0).toInt
val trainRDD = MLUtils.loadLibSVMFile(sc, inputTrainPath)
// training parameters
val paramMap = List(
"eta" -> 0.1f,
"max_depth" -> 2,
"objective" -> "binary:logistic").toMap
// use 5 distributed workers to train the model
// useExternalMemory indicates whether
val model = XGBoost.train(trainRDD, paramMap, numRound, nWorkers = 5, useExternalMemory = true)
// save model to HDFS path
model.saveModelToHadoop(outputModelPath)
}
}
On Sunday, February 26, 2017 12:58 PM, Nan Zhu<EMAIL_ADDRESS>wrote:

can you post the complete log trace with the error? It is impossible for the scala compiler to let a program with type mismatch error go—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

so, it is not Spark example code, it is your own program, right?
Please update the issue title accordingly
and also, please check the example code SparkWithRDD where it has an explicit conversion

Thanks.  It is the Spark example code listed on
 https://github.com/dmlc/xgboost/tree/master/jvm-packages.  Not really my code.
Thanks for the suggestion on using SparkWithRDD.  I tried out https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j-example/src/main/scala/ml/dmlc/xgboost4j/scala/example/spark/SparkWithRDD.scala that includes a conversion that resolves the type mismatch problem.  That works!
Appreciate your help Nan.  I will update my ticket.
cheers--priyan
On Sunday, February 26, 2017 5:01 PM, Nan Zhu<EMAIL_ADDRESS>wrote:

so, it is not Spark example code, it is your own program, right?Please update the issue title accordinglyand also, please check the example code SparkWithRDD where it has an explicit conversion—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Would you mind sharing your code, I also couldn't get the spark example working for the same type mismatch error.


Use tslint.loadConfigurationFromPath() to load tslint.json

https://github.com/panuhorsmalahti/gulp-tslint/blob/master/index.ts#L130
tslint.loadConfigurationFromPath()
https://github.com/palantir/tslint/blob/master/src/tslint.ts#L32
will resolve rulesDirectory as well as handling extends (coming soon).
I think this is the cause for #55
cc/ @JKillian

Specifically for #55,  https://github.com/panuhorsmalahti/gulp-tslint/blob/master/index.ts#L153
rulesDirectory: pluginOptions.rulesDirectory || null

should be
rulesDirectory: pluginOptions.rulesDirectory || fileOptions.rulesDirectory || null

But the file loading should still come from loadConfigurationFromFile()
My problem is the extends does not get resolved. The rules are not merged.

Yeah, replacing rcloader would be nice. However, we need to check there's no regression related to configuration merging (e.g. related to https://github.com/panuhorsmalahti/gulp-tslint/issues/49 ).
Any link to the upcoming extends feature?

We can't use tslint.loadConfigurationFromPath() directly, as the function reads the file from the supplied path directly instead of looking for it: https://github.com/palantir/tslint/blob/f4b3064a167226b52ee8027797a2c3942f5e6e7a/src/configuration.ts#L131
A "hack" workaround would be to implement a new callback parameter to rcloader which returns the location of tslint.json, which we can then load using tslint.loadConfigurationFromPath(). However, there should be better alternatives.
tsloader's fix can also be utilized (with the rcloader change) and is straightforward enough: https://github.com/wbuchwalter/tslint-loader/commit/2cd55a00aa8990b9955142956e36038c4d2191b6
However, the ideal fix would probably utilize findConfigurationPath in https://github.com/palantir/tslint/blob/f4b3064a167226b52ee8027797a2c3942f5e6e7a/src/configuration.ts#L87

Can you use findConfiguration? This is how I'll imagine that most 3rd-party tools will want to load configurations.

Extends feature: https://github.com/palantir/tslint/pull/1016
I'm work on it today or tomorrow if I get some time. It is now unblocked.

for a quick stop-gap until gulp-tslint is using findConfiguration internally, you can inject the config when piping.
import tslint from 'gulp-tslint'

gulp.task('lint:ts', () => {
  gulp
    .src('*.ts')
    .pipe(tslint({ configuration: require('tslint').findConfiguration() }))
    .pipe(tslint.report('verbose', { emitError: true, summarizeFailureOutput: true }));
});

// tslint.json
{
  "extends": "tslint/docs/sample.tslint",
  "rules": {
  }
}

The extends feature is released in 3.7.0

+1
this is extremely valuable feature
https://npmjs.org/tslint-config-valorsoft
// tslint.json
{
  "extends": "./node_modules/tslint-config-valorsoft/tslint.json",
  "rulesDirectory": "./node_modules/codelyzer/dist/src"
}

this works fine with tslint (in WebStorm)
but gulp-tslint fails to load it

These are the changes I made to grunt-tslint so new features like extends work correctly: https://github.com/palantir/grunt-tslint/pull/47/files
I don't know the gulp model very well / how it interacts with the filesystem, but perhaps something similar would be possible here?


Feature Request: Show the last 3-5 commands used on command palette launch.

This makes it easy to access the last command, if not the last few, instead of having to type it out each time.
Example, changing themes. Each time a new theme is applied, you have to go back and type 'Change Theme'  from the command palette.
As a follow up to this example, the currently selected theme should show when the theme picker is launched. I presume the theme picker itself is ripe for further development where you can preview on the fly.
I'd love to jump in and contribute but would love to see what the team has decided about this (and whether the architecture allows buffering previously use commands).

Commands are now sorted according to usage


vapi-vmc-client as an undeclared dependency on vapi-common-client

[x] I am using the latest SDK version
[x] This API is compatible with my vCenter version (You can get the info from 'vcenter_version' in each sample)
[x] I have searched existing issues

Environment

sdk package version:
(vapi-runtime is an explicit dependency of vapi-vmc-client)

$ pip freeze | grep vapi
vapi-runtime==2.9.0
vapi-vmc-client==2.9.0

python version:

$ python --version
Python 2.7.13

wheel package used: https://github.com/vmware/vsphere-automation-sdk-python/tree/346f5b1/lib/vapi-common-client

Steps or code snippet to reproduce + Actual behavior
$ ipython -c 'from vmware.vapi.vmc.client import create_vmc_client'
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-fad65e76a271> in <module>()
----> 1 from vmware.vapi.vmc.client import create_vmc_client

/home/lucas_cimon/.local/share/virtualenvs/infralib-py2/lib/python2.7/site-packages/vmware/vapi/vmc/client.py in <module>()
     10 from vmware.vapi.bindings.stub import ApiClient, StubFactory
     11 from vmware.vapi.lib.connect import get_requests_connector
---> 12 from vmware.vapi.stdlib.client.factories import StubConfigurationFactory                                                                                13                                                                                14 from .csp_filter import CSPSecurityContextFilter                                                                                /home/lucas_cimon/.local/share/virtualenvs/infralib-py2/lib/python2.7/site-packages/vmware/vapi/stdlib/client/factories.py in <module>()                                          9 from vmware.vapi.bindings.error import VapiError                                                                                10 from vmware.vapi.bindings.stub import StubConfiguration                                                                                ---> 11 from com.vmware.vapi.std import errors_client                                                                                12
     13
ImportError: No module named com.vmware.vapi.std

Expected behavior
No ImportError raised
Resolution
Installing also the vapi-common-client==2.9.0 wheel

@Lucas-C How did you install the modules? com.vmware.vapi.std is part of vapi-client-bindings. If you use the install cmd in README, all dependencies should be installed.
We recently integrated NSX bindings which may have introduced some duplicated dependencies, we will revisit and cleanup the redundant dependencies.

In my company, our internal process to handle Python packages that are not published on http://pypi.org is to put them in our repository manager, that handles all our build artifacts in all the languages we use (think JFrog artifactory, SonaType Nexus, etc).
We do handle Python dependencies by git clone-ing repositories.
Hence, to answer your question, I pip-installed the following wheels directly: vapi_client_bindings-1.3.1-py2.py3-none-any.whl, vapi_runtime-2.9.0-py2.py3-none-any.whl, vapi_vmc_client-2.9.0-py2.py3-none-any.whl
Whereas vapi-vmc-client had an explicit dependency on vapi-runtime, and refused to get installed without this wheel, not installing vapi_common_client-2.9.0-py2.py3-none-any.whl resulted in a runtime error.

Thanks for the clarification. Yes, vapi_common_client should also be a dependency of vapi-vmc-client. We will work on a fix to add this dependency to the wheel.


Make EnsoOutputStream fast again!

Pull Request Description
Will fix #10550 by properly using @TruffleBoundary annotation
Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:

[x] All code follows the
Java,
[ ] Benchmark have been written (first run) and are improved (improved run).

Two benchmark runs were scheduled:

(first run) with the old version of the code
(second run). with 4d1fe31 fix

We'll see the results tomorrow. However locally I see speedup of the Write_csv_file benchmark from 341ms to 9ms.

I am constantly getting failures in GUI Tests, however I am not dare to say the failures are unrelated as they are too close to the area touched by this PR:
1) widgets.spec.ts:205:1 › Selection widgets in Data.read node
  Error: expect.toBeVisible: Error: strict mode violation: locator('.DropdownWidget:not([data-transitioning])') resolved to 2 elements:
      1)
…
 aka locator('div').filter({ hasText: /^Choose file…File 1File 2$/ }).nth(2)
      2)
…
 aka locator('div').filter({ hasText: /^IgnoreReport_WarningReport_Error$/ }).nth(2)
  Call log:
    - expect.toBeVisible with timeout 5000ms
    - waiting for locator('.DropdownWidget:not([data-transitioning])')
    34 |
    35 |   async expectVisible(): Promise<void> {
  > 36 |     await expect(this.dropDown).toBeVisible()
       |                                 ^
    37 |   }
    38 |
    39 |   async expectNotVisible(): Promise<void> {
      at DropDownLocator.expectVisible (/runner/_work/enso/enso/app/gui2/e2e/widgets.spec.ts:36:33)
      at DropDownLocator.expectVisibleWithOptions (/runner/_work/enso/enso/app/gui2/e2e/widgets.spec.ts:24:16)
      at /runner/_work/enso/enso/app/gui2/e2e/widgets.spec.ts:251:22

on the other hand, this PR only changes OutputStream, it shouldn't affect _Selection widgets in Data.read node....

Original run:

New Benchmark run:


rename factory VideoWriter.open to VideoWriter.fromFile, add support …

fix: #197

factory VideoWriter.open -> VideoWriter.fromFile
VideoWriter: add apiPreference option

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.19%. Comparing base (097fbc4) to head (f9359e2).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #203   +/-   ##
=======================================
  Coverage   91.19%   91.19%
=======================================
  Files          57       57
  Lines        8371     8374    +3
=======================================
+ Hits         7634     7637    +3
  Misses        737      737

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add best practices to user guide

The user guide should contain best practices for writing robot tests. I have an initial set of best practices which can be improved and enhanced.

Write all tests in Behaviour Driven Development (BDD) style i.e using Given, When and Then.
Take advantage of robot framework's tracing i.e write as much behaviour as possible in robot instead of python
Use appropriate abstraction level in keywords and test case names
The tests should show WHAT is being tested, not HOW!
Tests should not contain complex logic; no FOR loops or IF-ELSE blocks. These can be used in the keywords though.
Use Join Path and avoid many slashes in the path
One test case should test one thing
Use Consistent variable names, avoid hard coding
Document reusable keywords
Use proper indentation
Reuse and extend the keywords already defined in common resource files
Use Suite Setup and Tear Down to speed up execution
Use Test Setup and Tear Down
Maximum 10 tests in a suite are recommended if they are not data driven
Maximum 10 steps in a test case are recommended, preferably less
Avoid sleeping, use polling where possible

Guidelines are important but I don't think the User Guide is the right place for them. We actually already have HowToWriteGoodTestCases which could be enhanced or even recreated if needed.
Notice also that it is hard or even impossible to create string guidelines that match all context. For example, I disagree all tests should be written using Given/When/Then style and think it is very often a good idea to move logic to Python.


CLI fails on Multiple Roles

first run works fine. But, if I run the commands again to get keys for a different role it fails unless I remove the session file ~/.okta-aws-cli-session
./awscli sts get-caller-identity
Exception in thread "main" java.lang.NullPointerException
at com.okta.tools.OktaAwsCliAssumeRole.run(OktaAwsCliAssumeRole.java:109)
at com.okta.tools.awscli.main(awscli.java:31)

I'll second the problem - same error:
awscli sts get-caller-identity
Exception in thread "main" java.lang.NullPointerException
	at com.okta.tools.OktaAwsCliAssumeRole.run(OktaAwsCliAssumeRole.java:109)
	at com.okta.tools.awscli.main(awscli.java:31)

Confirmed. I can reproduce the issue on master. Thank you for your report, @alamir1. Thank you for confirming, @emalenfant-fuze.
The fix is easy. This regression was accidentally introduced with the named profile support.
A quick fix is this:
export OKTA_PROFILE=“ “

I’ll get a real fix for this soon. Sorry 😊


macos: lacking automatic detection of serial port device path

In 0.0.2 we decided to remove the automatic detection of TKey serial port device path on Macos. The implementation (https://github.com/bugst/go-serial/blob/master/enumerator/usb_darwin.go) was not in pure Go and required the use of Golang's Cgo -- that is, calls to C code. To compile it, Macos/Darwin libraries/headers/source code needs to be available (and C compiler is used). Because of this, it became complicated to reproducibly build release binaries (and also to build them all on one platform). We decided to prioritize reproducible builds of these binaries, which are used by customers to verify that their TKey is genuine.
The consequence of removing automatic TKey detection on Macos is that after plugging in the TKey, user has to first list the available serial ports using something like ls -l /dev/cu*, and then run the verification using the relevant serial port like: tkey-verification verify --port /dev/cu.usbmodemN).  There will typically only be 1 /dev/cu.usbmodem present, unless the user has some other similar serial-port based devices plugged in. Other platforms does not (usually) require the use of the --port option.
One idea to work around this was to just have the program look for any /dev/cu.usbmodem* devices and if exactly 1 is found, then use it. This runs into the risk of talking to a serial port which does not belong to a TKey. However unlikely, this does not seem like a nice thing to do. The point of using the enumeration of the go-serial library is that it can list serial ports that belong to a USB device with a specific, known VendorID:ProductID (1207:8887 being Tillitis').
The path forward to enable auto-detection on Macos is probably to think through and implement a way to conveniently build reproducible Macos releases (of both architectures) when using CGO (ideally getting the two other platforms built at the same time). Or, could there be a pure-Go way to find out about the origins of a device-node on Macos?

It is probably possible to call /usr/bin/ioreg with the right options and parse the output, probably having to dig a second time as well. Not very nice and maybe not reliable. And that's parsing some kind of tree hierarchy realized in ASCII (or if using -a, parsing xml...).

FYI system_profiler can provide JSON output which might be more attractive than parsing XML  :wink:
So something along the lines of system_profiler -json SPUSBDataType

@udf2457 Great suggestion, we will look into it!

Been looking into this a bit now;
There seems to exists a few different options, (1) ioreg or (2) system_profiler.
(1) ioreg -r -c IOUSBHostDevice -l -x
One can look for the key IODialinDevice which provides the path to the serial device, for example /dev/tty.usbmodem1101. It is also possible to find the USB PID and VID. However, once you add the -aflag to get the output in XML, the VID and PID seems to be removed?! Not sure why. The XML output is huge, and needs to be parsed well, and it is not completely trivial how to know it is the path to the right serial device.
(2) system_profiler SPUSBDataType -json
This gives nice output in json, however does not seem to provide the path to the serial port. So, this is not so useful on its own. It is however very useful to confirm that a device of a certain PID and VID is connected to the computer.
So a solution could be to combination of the two above, even if it will be a lot of parsing and XML/JSON structs.
Maybe a good enough solution is to use (2), see that the correct device is connected to the computer. If there is only one, try to connect to it. If there are two or more devices, print them out for the user to select using --port.
There might be other options as well...


[v1.45] Cannot authenticate proxies

While using vscode version 1.45, I cannot submit the username and password of the proxy. The following popup prompted as expected, but both the OK button and the Enter key cannot submit the login info.

Confirm! The same issue for me, i can't submit it.

Same issue for me.
I rollback to install version 1.44 temporarily
https://code.visualstudio.com/updates/v1_44

Confirm.
And one more thing, I cannot setup proxy in settings.json file to bypass popup. Is there anyone?
My setup is: http://username:password@proxy.server:port

Confirm.
And one more thing, I cannot setup proxy in settings.json file to bypass popup. Is there anyone?
My setup is: http://username:password@proxy.server:port

In my case,
I tried

set proxy credential in settings.json
set proxy command line in VSCode shortcut
Those methods still can't work.
So I still keep input my credential in first launching VSCode.

Confirm.
And one more thing, I cannot setup proxy in settings.json file to bypass popup. Is there anyone?
My setup is: http://username:password@proxy.server:port

In my case,
I tried

set proxy credential in settings.json
set proxy command line in VSCode shortcut
Those methods still can't work.
So I still keep input my credential in first launching VSCode.

It's annoyed. Hope there are some work around for this.

Could you clear the http.proxy setting and then restart VS Code (File > Quit to make sure all windows are gone) and retry?

Could you clear the http.proxy setting and then restart VS Code (File > Quit to make sure all windows are gone) and retry?

I have no proxy settings in json config, but I couldn't submit proxy credentials in popup window.

@shamarin Could you check if this older 1.45-insiders build has the same problem:

Windows 64
MacOS
.deb

@shamarin Could you check if this older 1.45-insiders build has the same problem:
* [Windows 64](https://az764295.vo.msecnd.net/insider/b83339a3a7bd8e3ec8a8b79deda9c37e0d91f094/VSCodeUserSetup-x64-1.45.0-insider.exe)

* [MacOS](https://az764295.vo.msecnd.net/insider/b83339a3a7bd8e3ec8a8b79deda9c37e0d91f094/VSCode-darwin-insider.zip)

* [.deb](https://az764295.vo.msecnd.net/insider/b83339a3a7bd8e3ec8a8b79deda9c37e0d91f094/code-insiders_1.45.0-1587447574_amd64.deb)

Just checked - the same issue with this insider version.

@shamarin Do you have any of the HTTP_PROXY, HTTPS_PROXY or NO_PROXY environment variables set?

@shamarin Do you have any of the HTTP_PROXY, HTTPS_PROXY or NO_PROXY environment variables set?

No, settings as is (default values after first install)

@shamarin Can you launch 1.45.0 with --log-net-log=netlog.json, retry (is it with the extension viewlet, the update check or just by itself?) and attach the resulting netlog.json here?

@shamarin Can you launch 1.45.0 with --log-net-log=netlog.json, retry (is it with the extension viewlet, the update check or just by itself?) and attach the resulting netlog.json here?

How to do that? I don't know.

@shamarin Take the following steps:

Close all VS Code windows.
Open a command line window.
Launch code --log-net-log=netlog.json.
Trigger the auth dialog (what triggers it?).
Close VS Code.
Upload the netlog.json on this issue through the GitHub web UI (mail won't let attachments through).

/cc @deepak1556

@shamarin Take the following steps:
* Close all VS Code windows.

* Open a command line window.

* Launch `code --log-net-log=netlog.json`.

* Trigger the auth dialog (what triggers it?).

* Close VS Code.

* Upload the `netlog.json` on this issue through the GitHub web UI (mail won't let attachments through).

/cc @deepak1556

Did it. Thank you for helping me!
netlog.zip

@shamarin I see some requests successfully authenticate. Can you search for and install extensions when you keep entering the credentials?
@deepak1556 I ruled out the removal of header forwarding (03a7223ee2a086bbcf028801da9370d5deff5093) with the build in https://github.com/microsoft/vscode/issues/97199#issuecomment-625860342 and the session.setProxy() call with the netlog.json (that only shows a PAC file being configured). Are you aware of any other changes we should check?
/cc @joaomoreno In case you are aware of anything with regard to the auth dialog. Where do we store the credentials?

@shamarin I see some requests successfully authenticate. Can you search for and install extensions when you keep entering the credentials?

I couldn't search and install any extensions because to make any action in program I need to close proxy auth popup. After that even if I pressed OK in popup window with correctly entered credentials extension installer and search showing dialog that it couldn't connect to internet.

@chrmarti the net-log shows there is an explicit request for authentication from the proxy server
HTTP/1.1 407 Proxy Authentication Required
  Server: squid/3.5.23
  Mime-Version: 1.0
  Date: Fri, 08 May 2020 16:00:48 GMT
  Content-Type: text/html;charset=utf-8
  Content-Length: 2256
  X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
  Proxy-Authenticate: Basic realm="Krista internet proxy server"
  X-Cache: MISS from proxy6.krista.ru
  X-Cache-Lookup: NONE from proxy6.krista.ru:8080
  Via: 1.1 proxy6.krista.ru (squid/3.5.23)
  Connection: keep-alive

If the authentication scheme is basic, it will trigger the login event on app module https://github.com/electron/electron/blob/master/docs/api/app.md#event-login
We are handling this in https://github.com/microsoft/vscode/blob/master/src/vs/code/electron-main/auth.ts which provides the dialog seen by users.

Reproduced and I misunderstood: The UI is non-functional in that the buttons do not do anything. The dev tools show (the dialog is modal so I can't copy the text):

@deepak1556 helped track it down to the addition of a content security policy that makes the script execution fail: https://github.com/microsoft/vscode/commit/a873a9cca89d733f92e39489e5121364da88f182
[14109:0508/220747.365803:INFO:CONSOLE(81)] "Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-gwG0y+iDKxFvH2kBXU4EiwnPdGFE1C6ygbR24+UxvhI='), or a nonce ('nonce-...') is required to enable inline execution.
", source: file:///Users/chrmarti/Development/repos/vscode/out/vs/code/electron-browser/proxy/auth.html?config=%7B%7D (81)
[14109:0508/220747.466220:INFO:CONSOLE(1)] "Uncaught ReferenceError: promptForCredentials is not defined", source: file:///Users/chrmarti/Development/repos/vscode/out/vs/code/electron-browser/proxy/auth.html?config=%7B%7D (1)

Removing that policy fixes it for me.

Removing lines 7 and 8 from <install folder>/Contents/Resources/app/out/vs/code/electron-browser/proxy/auth.html appears to be a workaround (if you cannot go back to 1.44).

But can I fix the proxy setting for not showing popup? I don't want to enter username and password every time opening vscode.

If your proxy server is only configured for basic HTTP authentication, you will have to enter the credentials on the first run and the credentials will be cached, successive runs will delegate to the cache unless credentials were rejected then auth dialog will be prompted again.
For any further proxy configuration, I would suggest to contact your network administrator to try other proxy authentication schemes.

If your proxy server is only configured for basic HTTP authentication, you will have to enter the credentials on the first run and the credentials will be cached, successive runs will delegate to the cache unless credentials were rejected then auth dialog will be prompted again.
For any further proxy configuration, I would suggest to contact your network administrator to try other proxy authentication schemes.

I mean I would like to enter the credential only one time. Can I? If I close the vscode and reopen, the dialog is display. Is there any mechanism to set vscode use the Windows proxy setting?

here are screen shots from my VSCODE
https://github.com/microsoft/vscode/issues/97233

Removing lines 7 and 8 from <install folder>/Contents/Resources/app/out/vs/code/electron-browser/proxy/auth.html appears to be a workaround (if you cannot go back to 1.44).

Yep, this is a working workaround, at least for now.

Removing lines 7 and 8 from /Contents/Resources/app/out/vs/code/electron-browser/proxy/auth.html appears to be a workaround (if you cannot go back to 1.44).

The file location for me (Windows 10 default install location) was here:
C:\Users\<username>\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\code\electron-browser\proxy\auth.html

And yes it works! Removing lines 7 and 8 from that file fixed the issue for me :D

I couldn't find such code in my auth.html file. I have "" on line 7 and 8

Removing lines 7 and 8 from <install folder>/Contents/Resources/app/out/vs/code/electron-browser/proxy/auth.html appears to be a workaround (if you cannot go back to 1.44).

Same issue, and improved.
(environment)
Windows10 1909 64bit
Visual Studio Code 1.45.0 (system setup)
The target is
"C:\Program Files\Microsoft VS Code\resources\app\out\vs\code\electron-browser\proxy\auth.html"
Comment out lines 7 and 8
meta http-equiv="Content-Security-Policy"
content="default-src 'none'; img-src 'self' https: data:; media-src 'none'; child-src 'self'; object-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; connect-src 'self' https:; font-src 'self' https:;"
I solved this issue.

Confirm, commenting those lines solve the problem.

I mean I would like to enter the credential only one time. Can I? If I close the vscode and reopen, the dialog is display. Is there any mechanism to set vscode use the Windows proxy setting?

@quyleanh by default VSCode will respect system proxy settings. But based on the logs I saw in this issue, proxy settings are mostly set over PAC and they are configured for basic auth, VSCode has no choice but to explicitly request the credentials. The authenticated proxy connection will be valid in an open VSCode session as long as the Proxy server allows it, but once you close and reopen VSCode, authentication is required for new connection. You might want to check NTLM or negotiate authentication schemes for your proxy server.

Sorry for the breakage, all! We'll include the fix in the May recovery build, out soon.


Use nikic/FastRoute for routeparsing

Have you considered using nikic/FastRoute (https://github.com/nikic/FastRoute) for route parsing? Since Flight doesn't support route groups or such, it shouldn't be much work but probably improve the routing performance a lot. Would it be worth submiting a PR or just a waste of time?
Thanks for a great little gem (and keep it small!)!

The main problem is that Flight's url parameter format is completely different, /user/@id versus /user/{id}. That would be a major switch for users. But I did read the blog post where he discussed performance and there are some good ideas in there which I can probably incorporate into Flight in the future. However, FastRoute by itself doesn't support a couple Flight features like optional groups /user(/name(/id)) or the splat /user/*. Flight's router is already really fast itself, so I don't think it's worth switching just for a small performance gain and lose functionality.


ci: set Cilium base version to v1.10.12 in v1.10 conformance tests

The agent health check port will be changed for v1.10.12, ref.
cilium/cilium-cli#869. Set the base version explicitly to already use
the correct port in PRs in preparation for v1.10.12.

Reviews are in and changes only affect workflows in the v1.10 branch. No need to wait for CI results on master. Marked as ready to merge.


Fix GPS reference altitude in case gps checks pass quickly

In SITL the gps reference altitude was always set to zero as the gps checks passed before the main filtered initialized.
This PR also sets the gps velocity and position derivative filter states to zero if they are not being updated (vehicle is in air or there is movement.)

@kamilritz Thanks for the review, I updated the PR.


Will not start

I cloned this down and ran npm install
when running node PiCamera_accessory.js I get the following output
/home/pi/HomeKitCam/PiCamera_accessory.js:25
let sessionIdentifier = uuid.unparse(sessionID);
^^^
SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:414:25)
at Object.Module._extensions..js (module.js:442:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:311:12)
at Function.Module.runMain (module.js:467:10)
at startup (node.js:134:18)
at node.js:961:3

What is your node version?

node -v
v4.2.1

Well known issue of older node, please update your node to version 7
https://www.google.com.hk/search?q=SyntaxError%3A+Block-scoped+declarations+(let%2C+const%2C+function%2C+class)+not+yet+supported+outside+strict+mode


In VPN mode instead of exporting environment variables write them into a .env file

This way they can be loaded by the IDE for debugging.
Currently the IDE used for debugger (like PyCharm) is running in is own context and is not aware of the environment variables set by telepresence in the shell. But if these were written into a file they can easily be loaded by python into the correct environment.
my workaround is to create a shell script to get the env and write it into a file.
sed -i 's|TELEPRESENCE_ROOT.*|TELEPRESENCE_ROOT='"$TELEPRESENCE_ROOT"'|' ./app/.env

https://plugins.jetbrains.com/plugin/7861-env-file

https://plugins.jetbrains.com/plugin/9525--env-files-support is the more popular one

If we give Telepresence the ability to output an env file, eventually we'll need to decide which variant we generate. Here are a few options:

Docker compose. No special treatment for quotation marks. Might be same code for docker run --env-file, which does not support multi-line values.
Node dotenv package. Fancy treatment of quotation marks.
VS Code Python plugin. Explicitly has no support for multi-line values.

Generating JSON avoids these concerns, but might not be as broadly useful.


adjust error for runcon & stdbuf to make tests/misc/invalid-opt.pl pass

Hm, it looks like tests/misc/invalid-opt.pl is not run in the CI :|

it passes
2024-08-21T18:00:11.5397460Z PASS: tests/misc/invalid-opt.pl
it doesn't show because we didn't have it before (i guess)


Revisit saga example

It throws an Exception currently
2021-06-30 08:05:07,219 WARN  [org.apa.cam.com.pla.htt.ver.VertxPlatformHttpConsumer] (vert.x-eventloop-thread-0) Failed handling platform-http endpoint /api/pay. Caused by: [org.apache.camel.NoTypeConversionAvailableException - No type converter available to convert from type: io.vertx.core.buffer.impl.BufferImpl to the required type: java.nio.ByteBuffer with value ]: org.apache.camel.NoTypeConversionAvailableException: No type converter available to convert from type: io.vertx.core.buffer.impl.BufferImpl to the required type: java.nio.ByteBuffer with value

and I think it should be related to CAMEL-16756 Improve handling of Vert.x Buffer payloads in platform-http-vertx and it looks like be fixed in camel-3.12.
It needs to revisit the saga example when upgrade the camel to 3.12

It needs to revisit the saga example when upgrade the camel to 3.12

It should be fixed when camel-k upgrades to Camel Quarkus 2.0.0, as I added a workaround until Camel 3.11.1 / 3.12.0 is released.


findAll bug

if the isar.chatGroupMemberEntitys.where().gidEqualTo(gid).build().countSync() == 0
findAll return nothing
isar.chatGroupMemberEntitys.where().gidEqualTo(gid).build().findAll().then((value){
logger.d(value);
}, onError: (e){
logger.d(e);
});
but findAllSync is work

ps, my entity contains the isarLink
is it don't work in findAll when my entity contains isarLink because the isolate?
how can i do?
only use findAllSync?

Do you have sample code reproducing the issue?


bet Vortex mechanic

http://cb.poedb.tw/us/gem.php?n=Vortex
40% more damage when cast on frostbolt.

Interesting; Ice Nova has the same stat too. I'll add a "Cast on Frostbolt?" option to the Configuration tab for both skills.


osx build with SDK 10.12

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

FIXES #5

Need approval from core team

@conda-forge/core Review requested for the use of MACOSX_SDK_VERSION for issue #5

Forgot to update fork after removing OSX -> conflict


Jitter causes bad images for unstructured grid volume rendering

Here's how to generate the data: Get the data at: /project/projectdirs/vacet/1TZ Upsample using visitconvert to be 512 domains / 4B cells. Here's how to make the picture:- run the script attached to this bug report. And it makes the image attached ... for me on Longhorn and for Christoph on the cloud. I found that if I disable jittering in avtExtractor.C, it makes the correct picture. I'm really not sure what is going wrong. Also, note that I can't reproduce this with two cores on my laptop.
-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.
Ticket number: 590
Status: Pending
Project: VisIt
Tracker: Bug
Priority: Normal
Subject: Jitter causes bad images for unstructured grid volume rendering
Assigned to: Hank Childs
Category: -
Target version: -
Author: Hank Childs
Start: 02/07/2011
Due date:
% Done: 0%
Estimated time:
Created: 02/07/2011 05:48 pm
Updated: 06/10/2011 06:30 pm
Likelihood: 3 - Occasional
Severity: 4 - Crash / Wrong Results
Found in version: 2.2.0
Impact:
Expected Use:
OS: All
Support Group: Any
Description:
Here's how to generate the data: Get the data at: /project/projectdirs/vacet/1TZ Upsample using visitconvert to be 512 domains / 4B cells. Here's how to make the picture:- run the script attached to this bug report. And it makes the image attached ... for me on Longhorn and for Christoph on the cloud. I found that if I disable jittering in avtExtractor.C, it makes the correct picture. I'm really not sure what is going wrong. Also, note that I can't reproduce this with two cores on my laptop.
Comments:

0601_ugrid_volren.py.zip


Radial annotator: citation count no longer updates

In Browse nomenclature: Every time a new citation was added, the number in the blue segment was updated. This is no longer happening (here displaying 2 instead of 3)

I still doesn't update.

It is not in production yet
https://github.com/SpeciesFileGroup/taxonworks/blob/development/CHANGELOG.md


[DOCS]:Text Alignment Issue on Contact Us Page

Describe the documentation update
The "Contact Us" page has a misalignment in the text under the Address, Phone, and Email sections. The icons and text should be aligned for better readability and user experience. The documentation should be updated with guidelines on how to fix the alignment issue in the HTML and CSS code.
Why is this needed?
Proper alignment of elements on the contact page is essential for better user interface and accessibility. It enhances readability and provides a cleaner look to the page, improving the overall user experience.
Additional context
As shown in the attached screenshot, the text next to the icons for Address, Phone, and Email appears misaligned. Proper alignment can be achieved through CSS adjustments.
Screenshot:

please assign @akbatra567


Add documentation for building a pickle

Buidling your own pickle is pretty easy, but you need instructions on how to do it.

There is a tutorial here: https://github.com/jgroschwitz/vulcan/wiki/Tutorial:-Creating-a-simple-visualization-input-file


Clicks being blocked by UI outside of elements

Don't know if this is possible to resove. But it feels like there is an invisible box that blocks interactions with the game in the area of the listed items.
If this can be addressed, great.. maybe it is a limitation of how the items are drawn?

Yes, there is an invisible box around the items. It's unfortunate drawback of the way we draw the recipes on screen. We need to react to the mouse events so we capture them all.
I think it should be possible to implement "visual" transparency and "mouse event" transparency for everything except the images, but I don't really know how to do it in tk.


[Help Request] Blink

I'm trying to update a busylight script I have from an old version of busylight-for-humans==0.15.0.  I'm a hobbyist and would appreciate some help for things I'm not finding in the documentation.
It's a simple script that if it detects the microphone in use icon in the tray, then I want it to make the light blink red/off.
With that version, I was able to accomplish that with something along the lines of
from busylight.lights.embrava import Blynclight

LIGHT = Blynclight.first_light()

RED = (255, 0, 0)
BLINK = 1

if mic_status:
    LIGHT.blink(RED, BLINK)

I dug through the various releases and notes, but I've not been able to parse how to use the new import in conjunction with my light.  first_light() still turns it on, but I cannot figure out how to make it blink with
from busylight.effects import Blink

Thank you.

Yeah, that was my fault. I removed blink as a generic busylight.lights.light.Light method because several of the lights I was trying to support didn't have a blink mode that didn't require software intervention.  And then I re-wrote it busylight-for-humans again to support the effects you found via asyncio instead of the previous threaded version. It's ok if none of this makes sense.  The easiest way to get your light blinking when you want is using the busyserve command:
$ busyserve >& busyserve.log &    # start the busylight server in the background and keep the log

Next, talk to your light, or lights, using HTTP:
# make the first light found blink for 10 seconds and then turn off

from time import sleep
import requests

response = request.get("http://localhost:8000/lights")

lights = response.json()

light_id = lights[0]['light_id']

response = request.get(f"http://localhost:8000/light/{light_id}/blink")

print(response.json())

sleep(10)

response = request.get(f"http://localhost:8000/light/{light_id}/off")

print(response.json())

I know it's more involved than the version you were using, I was trying to normalize how lights behaved and none of them knew how to blink very well (except the Embrava Busylight family).
Let me know how that works for you and I'll consider re-adding the blink method to the Light class.

Hi JnyJny,
Thanks for the detailed explanation!
I was able to make some progress using the manager per the README.md, although the Python script would get hung up at applying the Blink effect.  I need to configure logging to see if I can identify why it's hanging at that line, whenever I get some time.
I went super simple and just decided to toggle it off/on right now with some sleep commands, but will also look into the blinkserve/api as I think that will expand my use cases.
For my own understanding, is there some more documentation or code samples with using the manager or combining the Blynclight.first_light() and the Effects classes?
I appreciate your help!

The best example I have for using the Effects classes is in busylight/__main__.py.  The blink subcommand for the command line interface is implemented as:
@cli.command(name="blink")
def blink_lights(
    ctx: typer.Context,
    color: Optional[str] = typer.Argument(
        "red", callback=string_to_scaled_color, show_default=True
    ),
    speed: Speed = typer.Argument(
        Speed.Slow,
        show_default=True,
    ),
) -> None:
    """Blink light on and off."""
    logger.info("blinking lights")

    blink = Effects.for_name("blink")(color, speed.duty_cycle)

    try:
        manager.apply_effect(blink, ctx.obj.lights, timeout=ctx.obj.timeout)
    except (KeyboardInterrupt, TimeoutError):
        manager.off(ctx.obj.lights)
    except NoLightsFound as error:
        typer.secho("Unable to blink lights.", fg="red")
        raise typer.Exit(code=1) from None

The manager refers to an instance of busylight.manager.LightManager which mediates the discovery of lights and applying effects to them via asyncio.

Great, I'll work on this some more this weekend.  No need to re-implement the the blink method to the Light class.  Thank you!


TypeError: expected dtype object, got 'numpy.dtype[float64]' when running scanpy on scvelo objects

[x] I have checked that this issue has not already been reported.
[x] I have confirmed this bug exists on the latest version of scanpy.
[ ] (optional) I have confirmed this bug exists on the master branch of scanpy.

Apologies for all the edits, but I'm stuck on this so have been playing around with it. Basically I'm getting weird errors when running scanpy on scvelo objects. I first had this issue when running sc.pp.pca(adata_panc, n_comps=50), but managed to solve it by previously setting adata_panc.X = np.array(adata_panc.X.todense()). However, I'm now getting the exact same error when running sc.pp.neighbors(adata_panc) and I'm not sure which matrix to test. Any advice would be very much appreciated!
Minimal code sample (that we can copy&paste without having any data)
adata_panc = scv.datasets.pancreas()
scv.pp.filter_and_normalize(adata_panc, n_top_genes=3000, min_shared_counts=20)
del adata_panc.obsm['X_pca']
del adata_panc.obsm['X_umap']
del adata_panc.obsp['distances']
del adata_panc.obsp['connectivities']
adata_panc.X = np.array(adata_panc.X.todense())
sc.pp.pca(adata_panc, n_comps=50)
sc.pp.neighbors(adata_panc)

Filtered out 20801 genes that are detected 20 counts (shared).
Normalized count data: X, spliced, unspliced.
Extracted 3000 highly variable genes.
Logarithmized X.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: expected dtype object, got 'numpy.dtype[float32]'

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
/hps/scratch/lsf_tmpdir/hl-codon-10-04/ipykernel_2322052/531027197.py in <module>
      7 adata_panc.X = np.array(adata_panc.X.todense())
      8 sc.pp.pca(adata_panc, n_comps=50)
----> 9 sc.pp.neighbors(adata_panc)

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in neighbors(adata, n_neighbors, n_pcs, use_rep, knn, random_state, method, metric, metric_kwds, key_added, copy)
    137         adata._init_as_actual(adata.copy())
    138     neighbors = Neighbors(adata)
--> 139     neighbors.compute_neighbors(
    140         n_neighbors=n_neighbors,
    141         knn=knn,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in compute_neighbors(self, n_neighbors, knn, n_pcs, use_rep, method, random_state, write_knn_indices, metric, metric_kwds)
    806             # we need self._distances also for method == 'gauss' if we didn't
    807             # use dense distances
--> 808             self._distances, self._connectivities = _compute_connectivities_umap(
    809                 knn_indices,
    810                 knn_distances,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in _compute_connectivities_umap(knn_indices, knn_dists, n_obs, n_neighbors, set_op_mix_ratio, local_connectivity)
    388
    389     X = coo_matrix(([], ([], [])), shape=(n_obs, 1))
--> 390     connectivities = fuzzy_simplicial_set(
    391         X,
    392         n_neighbors,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/umap/umap_.py in fuzzy_simplicial_set(X, n_neighbors, random_state, metric, metric_kwds, knn_indices, knn_dists, angular, set_op_mix_ratio, local_connectivity, apply_set_operations, verbose)
    600     knn_dists = knn_dists.astype(np.float32)
    601
--> 602     sigmas, rhos = smooth_knn_dist(
    603         knn_dists, float(n_neighbors), local_connectivity=float(local_connectivity),
    604     )

SystemError: CPUDispatcher(<function smooth_knn_dist at 0x14a113bac160>) returned a result with an error set

time: 4.73 s (started: 2021-08-18 11:47:40 +01:00)

Versions

WARNING: If you miss a compact list, please try `print_header`!
The `sinfo` package has changed name and is now called `session_info` to become more discoverable and self-explanatory. The `sinfo` PyPI package will be kept around to avoid breaking old installs and you can downgrade to 0.3.2 if you want to use it without seeing this message. For the latest features and bug fixes, please install `session_info` instead. The usage and defaults also changed slightly, so please review the latest README at https://gitlab.com/joelostblom/session_info.
-----
anndata     0.7.6
scanpy      1.8.1
sinfo       0.3.4
-----
PIL                 8.3.1
autotime            0.3.1
backcall            0.2.0
bottleneck          1.3.2
cffi                1.14.6
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
decorator           5.0.9
defusedxml          0.7.1
h5py                2.10.0
igraph              0.9.6
ipykernel           6.0.3
ipython_genutils    0.2.0
jedi                0.18.0
joblib              1.0.1
kiwisolver          1.3.1
leidenalg           0.8.7
llvmlite            0.33.0
loompy              3.0.6
louvain             0.7.0
matplotlib          3.4.2
matplotlib_inline   NA
mkl                 2.4.0
mpl_toolkits        NA
natsort             7.1.1
numba               0.50.1
numexpr             2.7.3
numpy               1.20.3
numpy_groupies      0.9.13
packaging           21.0
pandas              1.3.0
parso               0.8.2
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prompt_toolkit      3.0.19
ptyprocess          0.7.0
pycparser           2.20
pygments            2.9.0
pyparsing           2.4.7
pytz                2021.1
scipy               1.6.2
scvelo              0.2.3
six                 1.16.0
sklearn             0.24.2
storemagic          NA
tables              3.6.1
texttable           1.6.4
tornado             6.1
traitlets           5.0.5
wcwidth             0.2.5
zipp                NA
zmq                 22.1.0
-----
IPython             7.26.0
jupyter_client      6.1.12
jupyter_core        4.7.1
notebook            6.4.0
-----
Python 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05) [GCC 9.3.0]
Linux-4.18.0-240.22.1.el8_3.x86_64-x86_64-with-glibc2.10
96 logical CPU cores, x86_64
-----
Session information updated at 2021-08-17 16:58

time: 402 ms (started: 2021-08-17 16:58:46 +01:00)

I just wanted to update that this issue does not depend on scvelo at all, but I can recreate it by just using scanpy. Any help would be much appreciated:
Minimal code sample (that we can copy&paste without having any data)
import os
import scanpy as sc
import numpy as np
import pandas as pd
import copy
import anndata
import matplotlib.pyplot as plt
adata_pbmc3k = sc.datasets.pbmc3k_processed()
#del adata_pbmc3k.obsm['X_pca']
#del adata_pbmc3k.obsm['X_umap']
del adata_pbmc3k.obsp['distances']
del adata_pbmc3k.obsp['connectivities']
#sc.pp.pca(adata_pbmc3k, n_comps=50)
sc.pp.neighbors(adata_pbmc3k)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: expected dtype object, got 'numpy.dtype[float32]'

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
/hps/scratch/lsf_tmpdir/hl-codon-13-02/ipykernel_2124423/1009160698.py in <module>
----> 1 sc.pp.neighbors(adata_pbmc3k)

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in neighbors(adata, n_neighbors, n_pcs, use_rep, knn, random_state, method, metric, metric_kwds, key_added, copy)
    137         adata._init_as_actual(adata.copy())
    138     neighbors = Neighbors(adata)
--> 139     neighbors.compute_neighbors(
    140         n_neighbors=n_neighbors,
    141         knn=knn,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in compute_neighbors(self, n_neighbors, knn, n_pcs, use_rep, method, random_state, write_knn_indices, metric, metric_kwds)
    806             # we need self._distances also for method == 'gauss' if we didn't
    807             # use dense distances
--> 808             self._distances, self._connectivities = _compute_connectivities_umap(
    809                 knn_indices,
    810                 knn_distances,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/scanpy/neighbors/__init__.py in _compute_connectivities_umap(knn_indices, knn_dists, n_obs, n_neighbors, set_op_mix_ratio, local_connectivity)
    388
    389     X = coo_matrix(([], ([], [])), shape=(n_obs, 1))
--> 390     connectivities = fuzzy_simplicial_set(
    391         X,
    392         n_neighbors,

/hps/software/users/marioni/Leah/miniconda3/envs/scvelo/lib/python3.8/site-packages/umap/umap_.py in fuzzy_simplicial_set(X, n_neighbors, random_state, metric, metric_kwds, knn_indices, knn_dists, angular, set_op_mix_ratio, local_connectivity, apply_set_operations, verbose)
    600     knn_dists = knn_dists.astype(np.float32)
    601
--> 602     sigmas, rhos = smooth_knn_dist(
    603         knn_dists, float(n_neighbors), local_connectivity=float(local_connectivity),
    604     )

SystemError: CPUDispatcher(<function smooth_knn_dist at 0x150524c6cdc0>) returned a result with an error set

time: 3 s (started: 2021-08-23 11:59:12 +01:00)


patchJsImports utility treats in-workspace packages as local imports

We have a pnpm workspace with several packages. When one package imports another which happens to come from the workspace, patchJsImports interprets it as a relative import. The result is that, during build, the import gets transformed from a module name into a local path, which then fails when the package is published.
Example
Suppose a monorepo holds @company/package-one and @company/package-two. Package-one has a dependency on package-two:
// package-one/src/index.ts
import { stuff } from '@company/package-two';

This import should be left as-is during the build, because @company/package-two is a normal package, not a relative import -- it just happens to be resolved from the workspace. Instead, however, it gets transformed to the resolved path:
// package-one/dist/index.js
import { stuff } from '../../package-two/dist/index.js';

So when package-one gets published, the import fails and nobody can use the package.
What's happening

The "@company/package-two" import target gets resolved into a path like "/path/to/repo/package-two/dist/index.js" (patchJsImports.ts:49)
This path gets tested against NODE_MODULES_DIR_REGEX (patchJsImports.ts:52), which specifically looks for "node_modules" in the path (patchJsImports.ts:18)
Because the regex test fails, it gets converted to a relative path, and the transformed path is used instead of the original module name.

Some possible solutions
A) Explicitly check for relative paths and aliases, and only resolve those, instead of resolving all imports. It looks like this would be as simple as combining resolveAliases with the relative-path check (patchJsImports.ts:56)
B) If the resolved file is outside of the directories passed in, do not transform its import. This could be combined with the isNodeModulePath check, so that any out-of-directory file is treated as an external module
I'd be happy to submit a PR, if one of these sounds reasonable

I think the correct way to do it is to indicate to patchJsImports what should be considered as external and should not be touched.
You can actually already do it by providing an alias that resolves to null (null meaning "leave this dependency untouched"):
patchJsImports(directories, [
  {
    find<EMAIL_ADDRESS>// all non-relative paths...
    replacement: null, // ... are left as-is
 },
]

It is debatable, but I don't think it should be the default behavior. You might want to resolve non-relative paths.
Same for B), you might want to transform external modules.
I would advise using the aliases property to get what you need.


application pod readiness probe and startup probe failed once upgrade to istio 1.10.0 from 1.9.0

Bug Description
I was following https://istio.io/latest/docs/setup/upgrade/in-place/ and https://istio.io/latest/docs/setup/upgrade/canary/ to upgrade my istio to 1.10.0 from 1.9.0.
After upgraded, I tried kubectl rollout restart deployment. However, the new pods of deployment are failed due to probes. I checked the istiod pod log, it shows:
2022-01-01T05:38:17.417607Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:35276: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-01T05:38:37.908647Z     warn    serverca        Authentication failed for <IP_ADDRESS>:33646: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: target JWT extraction error: no HTTP authorization header exists.

I also have redeployment the istio and istio-ingress deployment. Unfortunately it still doesn't work. Any anyone helps on this?
Version
client version: 1.10.0
control plane version: 1.10.0
data plane version: 1.10.0

Additional Information
No response
Affected product area

[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[X] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Control Plane Revisions

Is this the right place to submit this?

[X] This is not a security vulnerability
[X] This is not a question about how to use Istio

Logs from istio proxy would be helpful to identify the issue.

Logs from istio proxy would be helpful to identify the issue.

@bianpengyuan these are the logs from istio proxy. istio 1.10.0 is doing some extra AuthN to new pods.
2022-01-02T03:26:59.692179Z     info    ads     CDS: PUSH request for node:dataservice-runtime-7c9b65465f-v9vh2.dataservice resources:34 size:34.9kB
2022-01-02T03:26:59.692290Z     info    ads     EDS: PUSH request for node:dataservice-runtime-7c9b65465f-v9vh2.dataservice resources:30 size:6.9kB empty:0 cached:30/30
2022-01-02T03:26:59.694864Z     info    ads     LDS: PUSH request for node:dataservice-runtime-7c9b65465f-v9vh2.dataservice resources:26 size:82.4kB
2022-01-02T03:26:59.695483Z     info    ads     RDS: PUSH request for node:dataservice-runtime-7c9b65465f-v9vh2.dataservice resources:13 size:11.9kB
2022-01-02T03:27:11.328102Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:54790: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:27:14.530201Z     warn    serverca        Authentication failed for <IP_ADDRESS>:50472: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: target JWT extraction error: no HTTP authorization header exists.
2022-01-02T03:27:14.789232Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:54838: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:27:19.387578Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:54880: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:27:37.897769Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:55072: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:28:06.952163Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:55374: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:28:28.902615Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:55600: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:28:41.380423Z     warn    serverca        Authentication failed for <IP_ADDRESS>:50472: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: target JWT extraction error: no HTTP authorization header exists.
2022-01-02T03:28:45.332439Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:55764: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:29:09.748059Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:56032: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:29:20.608486Z     warn    serverca        Authentication failed for <IP_ADDRESS>:50472: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: target JWT extraction error: no HTTP authorization header exists.
2022-01-02T03:29:28.847952Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:56222: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:29:40.381549Z     info    ads     ADS: "<IP_ADDRESS>:57030" router~<IP_ADDRESS>~istio-ingressgateway-6976f87cf8-4n5sr.istio-system~istio-system.svc.cluster.local-181 terminated rpc error: code = Canceled desc = context canceled
2022-01-02T03:29:40.560522Z     info    ads     ADS: new connection for node:router~<IP_ADDRESS>~istio-ingressgateway-6976f87cf8-4n5sr.istio-system~istio-system.svc.cluster.local-189
2022-01-02T03:29:40.561819Z     info    ads     CDS: PUSH request for node:istio-ingressgateway-6976f87cf8-4n5sr.istio-system resources:31 size:32.0kB
2022-01-02T03:29:40.562179Z     info    ads     EDS: PUSH request for node:istio-ingressgateway-6976f87cf8-4n5sr.istio-system resources:30 size:6.9kB empty:0 cached:30/30
2022-01-02T03:29:40.562756Z     info    ads     LDS: PUSH request for node:istio-ingressgateway-6976f87cf8-4n5sr.istio-system resources:1 size:2.7kB
2022-01-02T03:29:40.586529Z     info    ads     SDS: PUSH for node:istio-ingressgateway-6976f87cf8-4n5sr.istio-system resources:1 size:7.5kB cached:1/1
2022-01-02T03:29:40.587081Z     info    ads     RDS: PUSH request for node:istio-ingressgateway-6976f87cf8-4n5sr.istio-system resources:1 size:5.0kB
2022-01-02T03:29:58.141632Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:56536: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists
2022-01-02T03:30:06.538333Z     error   ads     Failed to authenticate client from <IP_ADDRESS>:56632: Authenticator ClientCertAuthenticator: no verified chain is found; Authenticator KubeJWTAuthenticator: target JWT extraction error: no HTTP authorization header exists

Looks like the log you pasted are from istiod? I am actually looking for log from istio proxy.

@bianpengyuan thanks for clarifying. Here is the logs of istio proxy of the new pod.
2022-01-02T06:20:31.847339Z     warn    failed to fetch token from file: open ./var/run/secrets/tokens/istio-token: permission denied
2022-01-02T06:20:31.847637Z     info    xdsproxy        connected to upstream XDS server: istiod.istio-system.svc:15012
2022-01-02T06:20:31.848804Z     warn    xdsproxy        upstream terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2022-01-02T06:20:31.849353Z     warning envoy config    StreamAggregatedResources gRPC config stream closed: 16, authentication failure
2022-01-02T06:21:01.337841Z     warn    failed to fetch token from file: open ./var/run/secrets/tokens/istio-token: permission denied
2022-01-02T06:21:01.337881Z     info    xdsproxy        connected to upstream XDS server: istiod.istio-system.svc:15012
2022-01-02T06:21:01.338874Z     warn    xdsproxy        upstream terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2022-01-02T06:21:01.339418Z     warning envoy config    StreamAggregatedResources gRPC config stream closed: 16, authentication failure
2022-01-02T06:21:22.346106Z     warn    failed to fetch token from file: open ./var/run/secrets/tokens/istio-token: permission denied
2022-01-02T06:21:22.347235Z     warn    sds     failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure

according to the log, found a similar issue https://github.com/istio/istio/issues/31740
In this case, use the following command do solve the problem.
istioctl install --set values.pilot.env.ENABLE_LEGACY_FSGROUP_INJECTION=true


Deprecate StabilizerTable

Summary
This PR deprecates StabilizerTable, which is used in Clifford.
Related to https://github.com/Qiskit/qiskit-terra/pull/7245.
Details and comments
Performance regression occurs due to stacking of x and z and tracking of phase.
We need to check the regression is acceptable or not.
=> Performance check: small regression for small qubits (acceptable?)
       before           after         ratio
     [6c0c8d26]       [199370f9]
     <main>           <qinfo/deprecate-stabilizer-table>
+         465±4ms         842±10ms     1.81  quantum_info.CliffordComposeBench.time_compose('1,7000')
+         546±6ms         823±20ms     1.51  quantum_info.CliffordComposeBench.time_compose('2,5000')
+        58.9±2ms       84.6±0.8ms     1.44  quantum_info.CliffordDecomposeBench.time_decompose('1,1000')
+      1.11±0.01s       1.49±0.02s     1.34  quantum_info.CliffordDecomposeBench.time_decompose('2,500')
+      1.59±0.02s       2.03±0.02s     1.28  quantum_info.CliffordDecomposeBench.time_decompose('3,100')
+         878±7ms       1.09±0.01s     1.25  quantum_info.CliffordComposeBench.time_compose('3,5000')
+         593±4ms          670±9ms     1.13  quantum_info.RandomCliffordBench.time_random_clifford('3,2000')
+         658±5ms          744±3ms     1.13  quantum_info.CliffordComposeBench.time_compose('4,2500')
+         666±5ms          749±5ms     1.13  quantum_info.RandomCliffordBench.time_random_clifford('2,2500')
+        722±10ms          801±7ms     1.11  quantum_info.CliffordComposeBench.time_compose('5,2000')
+        753±20ms          835±5ms     1.11  quantum_info.RandomCliffordBench.time_random_clifford('1,3000')
+         508±5ms          560±4ms     1.10  quantum_info.RandomCliffordBench.time_random_clifford('4,1500')

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

148 of 166 (89.16%)  changed or added relevant lines in 5 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.005%) to 82.479%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/quantum_info/operators/symplectic/pauli_list.py
6
9
66.67%

qiskit/quantum_info/operators/symplectic/clifford.py
52
67
77.61%

Files with Coverage Reduction
New Missed Lines
%

qiskit/quantum_info/operators/symplectic/stabilizer_table.py
1
88.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.005%

Covered Lines:
49674

Relevant Lines:
60226

💛  - Coveralls

Perhaps it's also relevant to check the RB performance since it requires to generate and compose large sequences of random Cliffords.
There is a benchmark here: https://github.com/Qiskit/qiskit/blob/master/test/benchmarks/randomized_benchmarking.py
However, it's based on the old ignis code that is going to be deprecated soon, and not on the qiskit.experiments code
https://qiskit.org/documentation/experiments/tutorials/randomized_benchmarking.html

@ShellyGarion Thanks. I checked RB:
[  0.00%] ·· Benchmarking virtualenv-py3.8
[ 12.50%] ··· Running (randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile--).
[ 25.00%] ··· Running (randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile_single_thread--).
[ 37.50%] ··· randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile                                                                                ok
[ 37.50%] ··· =============== ============
                 rb_pattern
              --------------- ------------
                   [[0]]        846±20ms
                  [[0, 1]]     3.82±0.04s
               [[0, 1], [2]]   4.73±0.1s
              =============== ============

[ 50.00%] ··· randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile_single_thread                                                                                ok
[ 50.00%] ··· =============== ============
                 rb_pattern
              --------------- ------------
                   [[0]]        814±20ms
                  [[0, 1]]     3.84±0.05s
               [[0, 1], [2]]   4.72±0.1s
              =============== ============

[ 50.00%] · For qiskit-terra commit e02119b2 <main>:
[ 50.00%] ·· Building for virtualenv-py3.8.......
[ 50.00%] ·· Benchmarking virtualenv-py3.8
[ 62.50%] ··· Running (randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile--).
[ 75.00%] ··· Running (randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile_single_thread--).
[ 87.50%] ··· randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile                                                                                ok
[ 87.50%] ··· =============== ===========
                 rb_pattern
              --------------- -----------
                   [[0]]        821±10ms
                  [[0, 1]]     4.10±0.3s
               [[0, 1], [2]]    9.60±3s
              =============== ===========

[100.00%] ··· randomized_benchmarking.RandomizedBenchmarkingBenchmark.time_ibmq_backend_transpile_single_thread                                                                                ok
[100.00%] ··· =============== ============
                 rb_pattern
              --------------- ------------
                   [[0]]        881±60ms
                  [[0, 1]]     3.95±0.04s
               [[0, 1], [2]]   4.93±0.1s
              =============== ============

Thanks @ikkoham - but now I think that this benchmark is only for the transpilation process, and does not benchmark the actual generation of the RB sequences themselves...

I see. then, we need to add more benchmarks to Qiskit repository.

I'll

In clifford_circuits and stabilzer_state it may be good to add some comments to the reader of the code, since it's not the exact formulas from Aaronson-Gottesman.

@ShellyGarion Thank you for your comments. It is true that the previous codes are hard to read. So, I added methods: stabilizer_tableau() and stabilizer_list(), and so on. I think now it's more readable.

@ikkoham: we'd like to get this merged soon to ensure that both it and #7245 can get into 0.20, and hopefully even some of the Clifford-related performance improvements.  I know you're busy with the new primitives, so let us know if we need to ask somebody else to get this finalised.

Mergeable?

It looks to me on a quick glance that the new from_stabilizers method will alleviate most of my usability worries about the deprecation, but we'd need to swap our internal cases that set the stabilisers and destabilisers to that as well - right now we're still logically mutating the Clifford class in those cases, which is something the deprecation seems to be telling people not to do.
We talked about this PR in the Terra meeting on Tuesday, and if you're ok with it, we'd prefer to push this from 0.20, and for you, me, Shelly and potentially Matthew to sit together and make sure we're all agreed on why StabilizerTable and PauliTable need to go, and what (if any) performance regressions are acceptable.
The benchmarks at the top of this PR indicate relatively hefty regressions - is it definitely appropriate to use PauliList as the backing for Clifford if it's slower than StabilizerTable?  If the eventual plan is to win back the speed by rewriting parts of it in Rust, then is there a need to go through a painful deprecation period when we'd be likely to need another one soon after?  The speed of using compiled extensions will help in some cases, but if PauliList fundamentally requires more operations to calculate the necessary values, it still might not be the best choice in moving to it.

I believe we already discussed that issue at last years quantum_info meeting. I think at least @chriseclectic and @ShellyGarion were there, maybe you (@jakelishman) and @mtreinish too. The regression is only for small number of qubits, which is not bad for scaling, so I'm sure we agreed, the adoption of PauliList is due to the simplicity that comes from using one class. In the first place I thought that Clifford should be given a dedicated implementation, which is not the case. supported. If we are going to go into this issue, we will rebuild from scratch and target 0.21.

The process is:

Define the internal data structure for Clifford (based on tableau representation).
StabilizerTable and PauliTable will be deprecated and use PauliList instead.

If you agree this direction, I'll close this PR. @jakelishman @ShellyGarion @chriseclectic

FYI: the impact of _count_y might be mitigated with https://github.com/Qiskit/qiskit-terra/pull/7830

This is my summer vacation :gift:
Sorry for late, but could you review the PR?

OK, then let's remove deprecation.
In any case, the deprecation of PauliTable will not be ready in time for 0.22, and I would like to deprecate PauliTable and StabilizerTable at the same time.


<base> not detected for game in a root

Death Stranding Director's Cut on PCGW (Epic) defines the config file location as <path-to-game>\settings.cfg. In Ludusavi, this is defined as <base>/settings.cfg. The issue is that previewing Death Stranding Director's Cut in Ludusavi won't locate the settings.cfg file and I'm not sure why. The game is detected so it's in a defined root, but why can't Ludusavi determine its installation directory correctly and find the config file?

Creating a custom game entry for the game and swapping <base> for the exact location of settings.cfg detects the file just fine.
Perhaps there is a conflict somehow as I have the non-Director's Cut version's settings.cfg still present on my system (or rather, in a backup and therefore Ludusavi shows it as missing)?

(The folded folders also do not contain settings.cfg.)
Thanks.

Perhaps there is a conflict somehow as I have the non-Director's Cut version's settings.cfg still present on my system (or rather, in a backup and therefore Ludusavi shows it as missing)?

Temporarily deleting my backup only removed the settings.cfg from showing on the standard Death Stranding. It did not show up on the Director's Cut version.

Hi! DeathStrandingDC wasn't a known install directory in the manifest, and it wasn't close enough to the game title for Ludusavi to infer it.
I've added an override entry, so this should be fixed when you download the updated manifest :)

Thanks as always @mtkennerly! I'll reopen this in the unlikely event the fix didn't work.


Question: clarification of ResiDual implementation

From my understanding of the ResiDual paper, shouldn't L1143-1144:
 x = x + pre_norm(outer_residual)

be positioned outside of the for loop (so that it only occurs once at the end of the forward pass). Apologies in advance if this is a misunderstanding on my end!

@alstonlo :man_facepalming: yes, i totally misconstrued the diagram
thank you for pointing this out! do you want to do a quick code review ?

LGTM, thanks!

@alstonlo i'm kind of curious now if the way i did it would make a difference

haha yes! I'm sure not as performant, given extra normalization and linear per layer
maybe you can write the follow up paper 😅

If it helps microsoft has an official implementation:
https://github.com/microsoft/ResiDual/blob/main/resi_dual/resi_dual.py
Maybe it would make sense to write it in plain first to see if it follows the same logic.
ie. postnorm = ln(x+mha(x))


Nested initial states

Hey David,
This project is really interesting. Great work! I have a question though.
With the following machine (example from the readme) the machine.initialState will be method.
{
  initial: 'method',
  states: {
    method: {
      initial: 'cash',
      states: {
        cash: { on: { SWITCH_CHECK: 'check' } },
        check: { on: { SWITCH_CASH: 'cash' } }
      },
      on: { NEXT: 'review' }
    },
    review: {
      on: { PREVIOUS: 'method.$history' }
    }
  }
}

However I was expecting it to be method.cash. This is further solidified by the first transition you give in the example:
const checkState = paymentMachine
  .transition('method.cash', 'SWITCH_CHECK');

Can you explain why that is the case?

The initial specified state is 'method', but the initial resolved state is 'method.cash'. The process for resolving the initial state (for non-parallel machines) is:

Find the specified initial state config ('method' in this case)
Recursively find the initial state of that state
If no initial state exists (leaf node), you're done.

Hope that answered your question!

Yes it did, thank you!

Hey, @davidkpiano great work on this library! I think would be handy to have a small method for resolving the initial state inside this library, because is a very common task.

@carloslfu That would be machine.initialState. Is that what you're asking? That gives you the State object that represents the initial state of the machine.

Oh, I'm sorry I was confusing machine.initial with machine.initialState. Thanks, @davidkpiano.


update pool payment and extension checks for capacitypool using the get payment info

get payment info api got added on devnet, testnet. let's prepare the changes needed for better checks on pools extensions. code already exists on development_vdc

Verified

Verified


Repository cleanup

Removing unused files and lines.
enabling isort autoflake again and organizing imports.

I think I already removed it for now. If we need config, we can come back and check it out again. How does it sound for you?


don't show pools which don't have any VMs

It is a rare case that a pool has no VMs in it but it may happen - such pool is not a valid one, can not be started and the properties of it can not be loaded. So it will have an icon of a cake for example.
So, it would be best not to show them at all.

also when you open the details of this pool, you will get:
Defined Memory:    0 null


Tests passing on PR validation but failing on CI

https://github.com/microsoft/vscode-jupyter/runs/1810903063
https://github.com/microsoft/vscode-jupyter/runs/1810908818

Hmm on those earlier builds there were some old variable view tests that I worry that I regresed. But I'm not seeing those on my most recent run here:
https://github.com/microsoft/vscode-jupyter/runs/1816585506
I just see:
"before each" hook for "Notebook has 3 cells":1 | Timeout of 30000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (D:\a\vscode-jupyter\vscode-jupyter\out\test\datascience\uiTests\ipywidget.ui.functional.test.js)
And
"after each" hook for "Open notebook and interactive at the same time":1 | Timeout of 180000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (D:\a\vscode-jupyter\vscode-jupyter\out\test\datascience\interactiveWindow.functional.test.js)
Also looks like the failures all seem to be windows only (make sense since PRs run on ubuntu) to might need a windows box to investigate.


Conda install xgboost

For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.
Environment info
Operating System:
Mac OS X El Captain 10.11.6
Package used (python/R/jvm/C++):
Python
xgboost version used:
0.3.0
If installing from source, please provide

The commit hash (git rev-parse HEAD)
Logs will be helpful (If logs are large, please upload as attachment).

If you are using python package, please provide

The python version and distribution
2.7
The command to install xgboost if you are not installing from source
conda install -c akode xgboost=0.3

Steps to reproduce
I have successfully installed xgboost and it is shown at the root.  However, when i tried to import xgboost it said the package is not there.
OSError: dlopen(//anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
Referenced from: //anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
Reason: image not found

@ghostintheshellarise the version xgboost=0.3 seems very outdated. since you have conda, do you want to try pip install xgboost ?

I following this instruction

Open Terminal
type : git clone https://github.com/dmlc/xgboost.git
type : cd xgboost; make; cd wrapper; python setup.py install --user

But it doesn't work at the last command
python setup.py install --user
and return following error message
Traceback (most recent call last):
  File "setup.py", line 19, in <module>
    LIB_PATH = libpath['find_lib_path']()
  File "xgboost/libpath.py", line 47, in find_lib_path
    'List of candidates:\n' + ('\n'.join(dll_path)))
__builtin__.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candidate path, did you install compilers and run build.sh in root path?
List of candidates:
/Users/SimonTse/xgboost/python-package/xgboost/libxgboost.so
/Users/SimonTse/xgboost/python-package/xgboost/../../lib/libxgboost.so
/Users/SimonTse/xgboost/python-package/xgboost/./lib/libxgboost.so
//anaconda/xgboost/libxgboost.so

And I did try to install via pip instruction but it gave me the message that I am already installed with xgboost
Requirement already satisfied (use --upgrade to upgrade): xgboost in /anaconda/lib/python2.7/site-packages
I am no CS trained so everything looks so cryptic to me

I am experiencing the same issue:
Cannot find XGBoost Libarary in the candidate path, did you install compilers and run build.sh in root path?
Did you find any solution?
Thank you very much :)

@felipecruz91 have you tried with pip install xgboost after having necessary compilers (gcc 5 for example), instead of using conda install?

@phunterlau Yes I tried using pip install xgboost but I get the following error:
No files/directories in C:\Users\....\AppData\Local\Temp\pip-build-9j_99ys4\xgboost\pip-egg-info (from PKG-INFO)

Oh, you use windows, please install from GitHub

@phunterlau I tried installing from Github:
git clone --recursive https://github.com/dmlc/xgboost
git submodule init
git submodule update
alias make='mingw32-make'
cp make/mingw64.mk config.mk; make -j4
cd python-package
python setup.py install
And I get the following error:
Traceback (most recent call last):   File "setup.py", line 19, in <module>     LIB_PATH = libpath['find_lib_path']()   File "xgboost/libpath.py", line 46, in find_lib_path     'List of candidates:\n' + ('\n'.join(dll_path))) XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path? List of candidates: C:\Users\felipe.cruz\Source\Repos\xgboost2\xgboost\python-package\xgboost\libxgboost.dll C:\Users\felipe.cruz\Source\Repos\xgboost2\xgboost\python-package\xgboost\../../lib/libxgboost.dll C:\Users\felipe.cruz\Source\Repos\xgboost2\xgboost\python-package\xgboost\./lib/libxgboost.dll C:\Users\felipe.cruz\AppData\Local\Continuum\Miniconda3\xgboost\libxgboost.dll C:\Users\felipe.cruz\Source\Repos\xgboost2\xgboost\python-package\xgboost\../../windows/x64/Release/libxgboost.dll C:\Users\felipe.cruz\Source\Repos\xgboost2\xgboost\python-package\xgboost\./windows/x64/Release/libxgboost.dll
Any idea?
Thank you very much :)

@felipecruz91 please follow this instruction by IBM (as mentioned in the installation instruction too) , step by step, strictly
https://www.ibm.com/developerworks/community/blogs/jfp/entry/Installing_XGBoost_For_Anaconda_on_Windows?lang=en

To install the official Anaconda Distribution xgboost Python package please use conda install py-xgboost (on Windows, Linux or macOS).

@mingwandroid you are hero

@mingwandroid thump up


unable to run 1.5.2 on openshift 4.3.13

In the 1.5.x docs there's a section on installing to openshift with the cni plugin that says it only applies for 1.4.x. But I see the initContainers fail with connection refused when running the bookinfo sample with the privileges set for the namespace - the same as described in https://github.com/istio/istio/issues/21266
What I used to install istio 1.5.2 when getting the connection refused error was:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
  values:
    cni:
      excludeNamespaces:
       - istio-system
       - kube-system
      logLevel: info

(Applied with istioctl manifest apply -f.)
So then I tried to install istio using something similar to the 1.4.x settings (but still on 1.5.2):
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
  values:
    cni:
      chained: false
      cniBinDir: /var/lib/cni/bin
      cniConfDir: /etc/cni/multus/net.d
      cniConfFileName: istio-cni.conf
      excludeNamespaces:
       - istio-system
       - kube-system
      logLevel: debug
    sidecarInjectorWebhook:
      injectedAnnotations:
        "k8s.v1.cni.cncf.io/networks": istio-cni

Then the bookinfo pods get stuck in initalizing with the error:
  Warning  FailedCreatePodSandBox  4s         kubelet, ip-10-0-164-206.eu-west-1.compute.internal  Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_details-v1-78d78fbddf-wpsfl_bookinfo_fb6f83d6-7218-4335-a669-393d393e8c37_0(60059780d5ecf144be09dc7d9b644235448a1d31def4e315a4df262423fa9d5e): Multus: [bookinfo/details-v1-78d78fbddf-wpsfl]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in getting k8s network from pod: GetNetworkDelegates: failed getting the delegate: getKubernetesDelegate: cannot find get a network-attachment-definition (istio-cni) in namespace (bookinfo): the server could not find the requested resource

This is openshift 4.3.13 (k8s 1.16.2). Issue seems to be specifically for namespaces where sidecar injection is enabled

You need to follow the steps described here: https://istio.io/docs/setup/platform-setup/openshift/#additional-requirements-for-the-application-namespace

Have just tried putting the networkattachmentdefinition.k8s.cni.cncf.io/istio-cni in the bookinfo namespace (which has sidecar injection on) with the first configuration above and in the bookinfo pods I still get:
in new validator: <IP_ADDRESS>
Listening on <IP_ADDRESS>:15001
Listening on <IP_ADDRESS>:15006
Error connecting to <IP_ADDRESS>:15002: dial tcp <IP_ADDRESS>:0-><IP_ADDRESS>:15002: connect: connection refused
Error connecting to <IP_ADDRESS>:15002: dial tcp <IP_ADDRESS>:0-><IP_ADDRESS>:15002: connect: connection refused

Will try the second configuration.

With the second configuration it works. Thanks!

@ryandawsonuk I have the same issue. What is the second configuration you were referring to?

@bharaththiruveedula This one:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
  values:
    cni:
      chained: false
      cniBinDir: /var/lib/cni/bin
      cniConfDir: /etc/cni/multus/net.d
      cniConfFileName: istio-cni.conf
      excludeNamespaces:
       - istio-system
       - kube-system
      logLevel: debug
    sidecarInjectorWebhook:
      injectedAnnotations:
        "k8s.v1.cni.cncf.io/networks": istio-cni

@ryandawsonuk how does this work for you ? I am getting the following error during ingress installation. Did you do anything extra other than this. I ran oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system  before running this script. Error -
Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_istio-ingressgateway-794b94fc7-4mvl8_istio-system_63e9b944-1309-4558-beab-0acef6e97317_0(b22a82bab32658dcb6e7b2eb13c72b20e4a68d2976bc49b45d246acdaf2d99bc): Multus: [istio-system/istio-ingressgateway-794b94fc7-4mvl8]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in getting k8s network for pod: GetNetworkDelegates: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (istio-cni) in namespace (istio-system): network-attachment-definitions.k8s.cni.cncf.io "istio-cni" not found


Is there a timeline for the Dynamic Forward Proxy feature to transition out of Alpha status?

Is there a timeline for the Dynamic Forward Proxy feature to transition out of Alpha status?
Description:
Use case would be for envoy to replace a current solution that requires httpd+mod_security as well as squid to provide both forward and reverse proxy functionality for http and https.
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/http_proxy
https://github.com/envoyproxy/envoy/issues/9916

In the v3 API this filter/cluster has been moved out of alpha.

That is great, thank you Matt!

@mattklein123 Thank you for confirmation. Does it mean it is in Beta? Or it has been tested and is suitable for production?

Hi, I got this HTTP forward proxy working but it fails for HTTPS. There is a problem with the configuration example on https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/dynamic_forward_proxy_filter#config-http-filters-dynamic-forward-proxy  where the file trusted_ca: {filename: /etc/ssl/certs/ca-certificates.crt} doesn't exist on the latest Docker repository. So HTTPS doesn't work


CClient: Modify data type of hash, bundle, address and tag from trit array to character array

In CClient types/types.h, the data type of hash, address and tags are defined as trit array.
And these are used in parameters in request and response.
Since we use trytes in every request, why don't we use character array as the data type?
Otherwise we need to transform trits to trytes in each request.

Good point,
But this is because trit_array will support 3 trits per tryte
And it might also be that Node will be able to get a request that is encoded in trit_array
in the near future,

@jkrvivian
As long as trit array is configured with 3 trits per tryte, you get trytes and no need for conversions
IRI implementation will not use trit array, so there will have to be some translation in case of  4/5 trits per tryte
still needs to be figured

@jkrvivian
Thanks for your willingness to contribute, that is awesome,  but we have still got some legal clarification we need to make until we are able to accept such big code contributions to our own repositories,

@tsvisabo
Could you give more information or references about 4/5 trits per tryte?
I could not find any discussion about the issue :thinking:
Thanks!