Prisma 2

Overview
Read more about the Prisma 2 preview here: https://www.prisma.io/blog/announcing-prisma-2-zq1s745db8i5
RFCs

Migrations: RFC / Discussion
Datamodel: RFC / Discussion
JavaScript/TypeScript Client API: RFC / Discussion
Client generators: RFC / Discussion
Unit of work: RFC / Discussion
Go Client API RFC
Engine/Connector Architecture
Configuration
Import/Export
CLI
Seeding
SDK

Repositories
https://github.com/prisma/prisma2
https://github.com/prisma/prisma2-cli
https://github.com/prisma/photonjs
https://github.com/prisma/lift
https://github.com/prisma/prisma2-docs

This all sounds exciting :+1:. Any ETA for a Prisma 2 beta/alpha release?

Pls let me know when we can use prisma with using Docker. That would be the greatest update I would love to see.

Hi! Will I have to do any changes to my Nexus-prisma integration?

We have released the preview of Prisma 2.
Read the announcement post here: https://www.prisma.io/blog/announcing-prisma-2-zq1s745db8i5
We would love to hear your thoughts on this.

Any ETA on Go client API?

Any chance that there'll be a configuration for the CLI? Currently it just generates the Photon ORM in a @generated/* directory inside node_modules.
This won't work if you by any chance are using microservices or several servers in the same monopoly repository as the generated path can't be specified since they'll just override each other.

@marcus-sa
You can configure an output property in the generator block now.
generator photon {
	provider = "photonjs"
    output = "./generated/photon"
}

One of the top reasons to use prisma is ability to easily leverage subscriptions from the DB layer without having to worry about it. Since you are moving the prisma engine side by side with the app:
How are subscriptions handled in a multi instances scenario?

Subscription open for inserts of TypeA on Instance 1
Record inserted for TypeA on Instance 2

How will Instance 1 know about the insert? Is there a way to "link" the prisma engines across instances?

@ntziolis
Right now Prisma 2 doesn't ship with a subscriptions solutions yet. We have a plan for an events engine which will be bin-log based rather than using the data which we have in the Prisma 2 which will be more sustainable. I can't give you an ETA on that as we are willing to spend time on designing the engine carefully.

@pantharshit00 Thank you for this info. Please keep us informed on progress here as without DB subscriptions we do not have much use for prisma tbh.

I am considering to use prisma in production, but not sure if I should use prisma 1 or prisma 2. In terms of stability it makes sense to use prisma 1, but because of the docker part, I would rather use prisma 2. I only need stable graphql query and mutation to work(no subscription for now). Can you guys share some advice on this issue?

@hoffnung8493 I am in the same boat trying the decide the same. I am designing the backend for a startup. I love the ease of development with Prisma. However, it must be deployed to production, and when it does it needs to work. However, our use case is overall very simple and minimal from a load perspective.
Background

Our app will not be ready for full production deployment for 2-3 months, so both projects have about that long to progress.
We've used a serverless GraphQL backend to maximize flexibility and minimize boilerplate on our front end while preparing us for scale.
The nature of our data is that it is highly partitionable per user. It can fit in a relational database, but the more ideal database for it is certainly Mongo due to the partionability and higher availability. Because of Prisma's Import/Export functionality, we may be able to initially deploy to a relational DB until Mongo support comes to v2, at which point, we'd export our data and import it into Mongo.
Expected usage of the app would be intermittent. Most users would only need to use the app for 1-6 weeks and not again for as much as a decade. Because of this, metrics like max concurrent users as well as the load-per-user metrics would be minimal. It shouldn't be hard to scale to 1,000s and 10,000s of users as only perhaps 10% of that would be actively accessing data at a given time.
Currently, there is no need for real-time data flow via subscriptions.

Would Prisma 2 (v2) likely be just good enough to handle this simple workflow? If not, is Prisma 1 (v1) able to handle this stably and in prod? If we must use v1, is there a way we can make a Prisma server capable of handling a maximum of 100s of concurrent users that could be done in a day or 2? Is there any way that doesn't require a Docker/Kubernetes distributed DevOps expert or me becoming one in order to get this thing deployed?
Thoughts?

Hi all,
I am going to lock this issue now as the feedback is getting fragmented among repositories for us. Please open a new issue here if you want to ask about something: https://github.com/prisma/prisma2. I am going to answer the last remaining questions here:
Right now we are also in a weird state ourselves on whether to recommend a new user Prisma 1 or Prisma 2. Right now I will say don't use Prisma 2 for a very large application as we may have breaking changes over a period of a week and it is not very stable. For a side project, it has enough query capabilities right now as you can see in Matt's talk here: https://www.youtube.com/watch?v=WSyaBOAZFZY. So I will recommend you to all to use Prisma 1 for your current large projects and adopt Prisma 2 after it has become stable enough for production usage. For any other project, it will be great if you can test out Prisma 2 and give us your valuable feedback.
For performance and scalability issues, they were one of the reasons why we wrote Prisma 2 in Rust and we now support serverless environments as well so you will need not to be a DevOps expert. Prisma 2 is just a library, scale it like any other node app. So you will need to migrate to Prisma 2 here. So for existing Prisma 1 users, please wait till general release and migrate your application to Prisma 2.
Any new user that is willing to use prisma for a small project should use Prisma 2. Any other user that is trying to write a large application should use Prisma 1 and incrementally adopt Prisma 2 after the general release.
Also, we will continue to do bug fixes in Prisma 1 and new issues will get addressed. Just no new feature will be released though.

@mk0a1a For the Prisma 2 Go client, please see the tracking issue https://github.com/prisma/prisma2/issues/571 and the actual work at https://github.com/prisma/photongo.


help with the miner

hey i did what did you say with the miner
The cryptocurrency mining module depends on Nanominer. Get the release archive from the original repository and save it as Server\files\miner.zip.
and then i do start mine and in the pool stats noting moved

and now i have that error in the server console
TypeError: Cannot read property '1' of undefined
at D:\snapshot\OrionServer\bots.js:71:40
at Layer.handle [as handle_request] (D:\snapshot\OrionServer\node_modules\express\lib\router\layer.js:95:5)
at next (D:\snapshot\OrionServer\node_modules\express\lib\router\route.js:137:13)
at Route.dispatch (D:\snapshot\OrionServer\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (D:\snapshot\OrionServer\node_modules\express\lib\router\layer.js:95:5)
at D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:281:22
at Function.process_params (D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:335:12)
at next (D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:275:10)
at Immediate._onImmediate (D:\snapshot\OrionServer\node_modules\express-session\index.js:502:7)
at processImmediate (internal/timers.js:458:21)

and now i have that error in the server console
TypeError: Cannot read property '1' of undefined
at D:\snapshot\OrionServer\bots.js:71:40
at Layer.handle [as handle_request] (D:\snapshot\OrionServer\node_modules\express\lib\router\layer.js:95:5)
at next (D:\snapshot\OrionServer\node_modules\express\lib\router\route.js:137:13)
at Route.dispatch (D:\snapshot\OrionServer\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (D:\snapshot\OrionServer\node_modules\express\lib\router\layer.js:95:5)
at D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:281:22
at Function.process_params (D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:335:12)
at next (D:\snapshot\OrionServer\node_modules\express\lib\router\index.js:275:10)
at Immediate._onImmediate (D:\snapshot\OrionServer\node_modules\express-session\index.js:502:7)
at processImmediate (internal/timers.js:458:21)

Second problem should be fixed as of 35c145be34086d3d53f3841f195dc7f58e015237 (see bots.js), it was probably due to information gathering failing on your system.
As for the miner, please mind the structure of the necessary archive as described in the Required structure spoiler section of the Setup Guide. Make sure that the executable and the configuration file are in the root of the archive. Furthermore, please see whether the miner is active (as a normal process) on your system after issuing the command. Most pools don't offer immediate statistics, so it might take some time for them to refresh your stats.


[macOS] Add option to emit signal when opening files/urls with the app, instead of starting new instance.

Adds application/config/emit_on_file_open setting to emit MainLoop::on_file_open signal instead of starting new app instance. Always enabled for sandboxed apps, since these can't open instance with command line arguments. When launching the first instance, the opened file names are sent via command line arguments as before.
func _ready() -> void:
	get_tree().on_file_open.connect(_on_file_open) # connect signal

func _on_file_open(files: PackedStringArray) -> void:
	$TextEdit.text = "\n".join(files) # process newely opened file

Fixes https://github.com/godotengine/godot/issues/94788

This signal is great, exactly what I've wanted!
But for consistency and ease of use wouldn't it be better if the same signal was also emitted for the first files opened (also sent via command line arguments)?
This way the signal could also be implemented on other desktop platforms and not just macOS (in case other platforms don't have a way to get an open file event in a running process). And users could forget about command line arguments and  just use this on_file_open signal for all files regardless of platform or situation.
In this case the new project setting could instead control just the starting of new processes (maybe just on macOS) on file open and should be renamed accordingly.

But for consistency and ease of use wouldn't it be better if the same signal was also emitted for the first files opened (also sent via command line arguments)?

There's a problem with this approach - when to emit the signal? Command line arguments are available from the start, and a signal can be connected only in the main scene ready or later. So emitting it should be delayed, but it's not clear for how long.

when to emit the signal?

Is there not a ready on the tree root node, that would be called once all autoloads and the main scene has finished loading and their ready calls have been executed? If there is then I think this would be the appropriate place to emit the file_opened signal since the project would then have had the opportunity to connect to the signal either in an autoload _ready() or in the main scene _ready(). This first emit time should then also be documented to help people understand when they need to connect to this signal to catch launch files.

If there's no easily accessible _ready() on the main SceneTree where a files_opened signal could be emitted if there are file path command line arguments I suppose it could just be added at the end of main.cpp start() https://github.com/godotengine/godot/blob/fd7239cfab228c50777cd54a8bf6eb279a02ef81/main/main.cpp#L3453 at some point after both the autoloads and the start scene have been loaded and are ready (like right at the end).
However, the files_opened signal for startup command line arguments is mostly a nice service to the users. And if that delays the implementation of the main proposal here I could live without. I'll be very happy if I can get anything that allows stopping new instances of the application from starting on file drop and instead sending the file path via a signal to the running application, as that currently makes it very hard to make any app that relies on getting files from the OS.


Focus on mount

I would like the component to focus on mount. How can I do that ?
If it's not possible with the current component, I can create a PR to add this feature.

use inputProps to pass down to the input field.
Something like

I do this in the example: https://github.com/mmw/react-password-strength/blob/master/example/index.js#L26


Merge sql updates: Blob size increase

What changed?
Merge temporal change

Why?
support large blob size

How did you test it?

Potential risks

Coverage increased (+0.06%) to 61.375% when pulling b1ae8d9233d33de074e15089deaff25ce2e61e72 on sql_blob_size into 9a34dcf262c23200767b4184929aaa18df9551f0 on master.


Miri cannot represent partial pointers (for byte-wise copy and partial overwrite)

Miri's model of memory can only attach provenance to pointer-sized chunks of memory, but not to individual bytes. This leads to incorrect behavior in several situations:

We cannot copy a pointer one MaybeUninit<u8> at a time. In particular this affects ptr::swap_nonoverlapping.
We cannot overwrite some of the bytes of a pointer, but preserve provenance on the bytes that remain.

Testcases for the first problem
use std::{
    mem::{self, MaybeUninit},
    ptr,
};

fn main() {
    let mut ptr1 = &1;
    let mut ptr2 = &2;

    // Swap them, bytewise.
    unsafe {
        ptr::swap_nonoverlapping(
            &mut ptr1 as *mut _ as *mut MaybeUninit<u8>,
            &mut ptr2 as *mut _ as *mut MaybeUninit<u8>,
            mem::size_of::<&i32>(),
        );
    }

    // Make sure they still work.
    assert_eq!(*ptr1, 2);
    assert_eq!(*ptr2, 1);
}

use std::mem::{self, MaybeUninit};

unsafe fn memcpy<T>(to: *mut T, from: *const T) {
    let to = to.cast::<MaybeUninit<u8>>();
    let from = from.cast::<MaybeUninit<u8>>();
    for i in 0..mem::size_of::<T>() {
        let b = from.add(i).read();
        to.add(i).write(b);
    }
}

fn main() {
    let ptr1 = &1;
    let mut ptr2 = &2;

    // Copy, bytewise.
    unsafe {
        memcpy(&mut ptr2, &ptr1);
    }

    // Make sure they still work.
    assert_eq!(*ptr1, 1);
    assert_eq!(*ptr2, 1);
}

Also this special case in the standard library should not be needed any more when this is properly implemented.


Updated Derby Test Dockerfile

I was running into this error on the old Dockerfile:
Using docker image default Java
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)
/derby
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Buildfile: /derby/build.xml

clean:

cleangenerated:

cleanstate:

cleanreleasefiles:

clobber:

BUILD SUCCESSFUL
Total time: 0 seconds
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Buildfile: /derby/build.xml

checkCompilerLevel:

BUILD FAILED
/derby/build.xml:211: Compiler level must be Java 9 or later.

Total time: 0 seconds
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Buildfile: /derby/build.xml

version_check:

BUILD FAILED
/derby/build.xml:94: FATAL ERROR:  The running Ant version, 1.10.5, is too old.

Total time: 0 seconds

So I went ahead a revamped the Dockerfile to fix the errors. I bumped the default SDK to Java 9 and bumped the Ant version to 1.10.6. On Ubuntu Ant did not have a package version of 1.10.6, so I switched to manually installing it.
I also removed a good number of unneeded packages from the container.
The only other suggestion I wanted to make is not to pull from https://github.com/apache/derby.git. With pulling from this and not checking out a certain tag or branch we are tasking for issues down the line. We will continue to hit issues with Java version and Ant version.
With that being said, I am not 100% sure with that you guys are testing here. So if the idea is to pull from always the latest code on Derby and see the errors maybe we should leave it. @ShelleyLambert can you weight in on this?

Thanks for digging into this @james-crowley !
To answer your question, what are we trying to test... generally, we were trying to achieve a couple of goals with this work (which is loosely captured in the external/README.md), but mainly:

verify the docker images that we produce at the project
verify that some large applications are able to run using the Adopt images

If the application runs with both jdk8 and jdk11 (the LTS releases), plus the current release being built (at present jdk13, soon to be jdk14), we should support running tests for all versions.  Looks like in this example, the tip/master of the derby repo is updated to use modularity (so needs jdk9 or higher), which means we either need to stop triggering it with a parameter openjdk8 (which we have already done in the playlist file that TKG uses, https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/external/derby/playlist.xml#L25), or use a tagged version of the derby repo for 8, and a separate tagged version or master for 9+.  Note that if you are running these external tests via TKG, no test target would be generated if you were using jdk8 (because the contents of that playlist file indicate only for 9+).
For ant version, if we move towards an extra test layer as mentioned in Slack conversation, we will pin on a particular version of ant that will work for most cases.  If there are outliers, they will have to set up with whatever version of ant they need (or we also parameterize ant url and version to pass in).


Unable to override webpack-dev-server client.overlay configuration for renderer process

Pre-flight checklist

[X] I have read the contribution documentation for this project.
[X] I agree to follow the code of conduct that this project uses.
[X] I have searched the issue tracker for a bug that matches the one I want to file, without success.

Electron Forge version
6.2.1
Electron version
25.2.0
Operating system
macOS
Last known working Electron Forge version
No response
Expected behavior
Override webpack devServer.client.overlay config
Actual behavior
Adding
devServer: {
    client: {
      overlay:  false
    }
  }

to webpack.renderer.config.ts has no effect at runtime - I continue to get overlay errors with this configuration set.
Steps to reproduce

electron-forge project with webpack-typescript template + add react capability
trigger a runtime error in the renderer code
a. I have been getting this in project after configuring a two panel layout with react-split and react-monaco-editor in the right pane (without automaticLayout: true). Running the application and using the resizable divider leads to "ResizeObserver loop limit detected" runtime error. From what I can tell this is benign (which is wanted to suppress this error).
still observe the overlay for runtime error despite configuring it to not show. I've also observed this in the various forms of trying to disable the error (overlay: { runtimeErrors: false }, overlay: { runtimeErrors: (error) => {...} })

Additional information
This seems potentially related to #2599

I am also affected by this problem. I solved it by adding a CSS rule to my page that just sets display: none on the iframe that Webpack injects for the overlay.
#webpack-dev-server-client-overlay {
    display: none;
}

try, this works for me
devServer: {
  client: {
    overlay: {
      errors: false,
      warnings: false
    }
}


[Plugin] /records To keep track of milestones in a channel.

Would be nice to have this feature (We had it in WritheM previously)
/records would display stats like:

Highest User Count / Time
Most Upvoted Song
Most Downvoted Song
Highest contention voted song (most up/down votes closest to 0 sum, ie. the most contraversal song)
Most upvoted suggestion(s) ?

This could be part of the stats thing @pironic is working on or it's own plugin.
Perhaps with a /resetrecords command for clearing them all / individually.

it would be a neat idea! Right now, such plugin could be written without touching the core, which is the point I wanted to push the core to.
It's a perfect timing as I'm about to get really busy IRL, and will likely just patch the core and the default plugins if grooveshark evolves and break some things.
You could try implementing that!

I suppose I could... maybe I'll try my hand at this one tonight.

I see two ways of doing this, well, three...

Create a small file to store those data, whenever callback are called, check for new stuff.
Do it with an external DB (such as @pironic 's php module)
Have me building as a part of the core a pretty unefficient database with nodejs, where you can just get an javascript object and store it, whenever you want, that will persist upon reboots.

I was going to use a json flat file on disk, but as a matter of completeness wouldn't it be prudent to utilize something like a sqlitedb for random crap the bot wants to store, keeping things central / contained?

@reanmachine, look into the php stuff i did... i did intend to keep the
records stored on the sharkcommunity db as well. I just havn't gotten
around to it yet. If you want to build off that, i'd be happy to deploy
anything you code.
The way i was going to do it, is check what the current records are on
init, then keep those local until they are broken. when they are, post an
update to the database. It'd be a simple 1 value per type thing... but then
i started thinking about the possibility of logging stuff more indepth. It
would be neat to see the graphs i have in real time of listener counts and
peak periods per broadcast. (all contained on the community stats section).
I'm just worried that it'll be too many hits for what it's worth.
On Tue, 31 Mar 2015 at 13:57 Reanmachine<EMAIL_ADDRESS>wrote:

I was going to use a json flat file on disk, but as a matter of
completeness wouldn't it be prudent to utilize something like a sqlitedb
for random crap the bot wants to store, keeping things central / contained?
—
Reply to this email directly or view it on GitHub
https://github.com/UlysseM/GSBOT/issues/50#issuecomment-88227306.

@pironic How about I make it a configuration option, so people can use the plugin without the PHP dependency, we can support both!

if you could do that for the whole php branch, that would be amazing :-)
dbhistory rather.
On Tue, 31 Mar 2015 at 17:38 Reanmachine<EMAIL_ADDRESS>wrote:

@pironic https://github.com/pironic How about I make it a configuration
option, so people can use the plugin without the PHP dependency, we can
support both!
—
Reply to this email directly or view it on GitHub
https://github.com/UlysseM/GSBOT/issues/50#issuecomment-88285254.

@pironic Let's prototype it here, then worry about the other stuff! :)

@UlysseM I've managed to complete a prototype fairly easily, there's a couple technical components that came up during development I wanted to ask you about:

I read in your guidelines that plugins shouldn't be using require(), I can't tell if you mean breaking the core/plugin barrier by requiring gsbot/manatee or if you mean *strictly, i.e. not importing nodejs components like fs or whatnot. What would be your standard process for accessing the fs or http modules from node.js?
It seems plugins callback methods are inconsistently being called with context, the init() method has context due to the way it's called, but the onCall, onSongChange etc... are not because they're being copied into a callback list. Wrapping the callbacks with a .apply() call would resolve this across the board in a 1 line change I could submit as a pull request to you if you're interested in having proper this context in those calls, if that's by design can you enlighten me as to why you'd want the module objects using only global calls to the module?
Inter-Module Communication, how are we supposed to do this? I need one command to issue calls to the non-command module to do things like save the results as this is an operation that multiple commands may want to do, and duplicating that code/config is not desired. It looks like you used to pass a moduleList to all the modules on init so they could in theory go get the other module instance and call methods on it but it seems that this is no longer the case because in moduleloader.js it doesn't appear that the list is being populated (it looks like that change was a result of a refactor or something).

If you could shed some light that'd be awesome, part of this may be my inexperience working on the project but perhaps I've identified some areas we could improve.
Like I said, I can submit a pull request for Point 2 if you're not sure what I'm talking about, it wouldn't have any adverse effect on any existing plugins / code, it would only serve to keep things standard and correct.
-Aren

You can use it to access fs/http.
However, watch for collision with the file name, as multiple broadcast can run in the same path.

I have to admit, javascript is far from being my favorite language. The GSbot was my first try at it, and I probably know only a third of the language feature. You can do a pull request for the core, give me some time to review it.

On init, there is an object that is shared between all mods of a plugin. You can just place the method there and keep the reference in all your plugins so you can call them.

@UlysseM

Thanks, didn't think about that. In the future it would be ideal if the bot could provide a path to a broadcast/plugin specific storage space, for now I'll rig something up.

Ah yes, javascript is a mysterious beast, For your "first" project in JS it's looking quite good so don't fret.

I tried this (defined a function on sharedObject but ran into issues due to an unreliable load order with the modules. I'll give it another crack and see if we need to look at another avenue.

Thanks

Yeah, I have to redo all the init calls... right now, they are called as the plugins are loading, I want to call them when the broadcast is actually "ready".


Update Mobx devDependencies to peerDependencies

Version
Tell us which versions you are using:

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Expected behaviour
I was trying to configure and use mobx as my default state management, but I had some problems while configuring it, because the new version of<EMAIL_ADDRESS>and<EMAIL_ADDRESS>only accepts 1 instance.
react-native-router-flux uses and old version for both of them and is configured as devDependencies
inside of package.json.
Actual behaviour
It's crashing the application after importing an observable, printing the following output:
[mobx] There are multiple mobx instances active. This might lead to unexpected results: See https://github.com/mobxjs/mobx/issues/1082 for details.
Steps to reproduce

Create a simple react-native project.
Install react-native-router-flux: npm install react-native-router-flux
Install mobx and mobx-react with the latest version: npm install mobx mobx-react
Create a simple observable object and import it in App.js or even in index.js
It's expected to output the following error [mobx] There are multiple mobx instances active. This might lead to unexpected results: See https://github.com/mobxjs/mobx/issues/1082 for details..

Steps to fix
There is a topic created by the developer of Mobx, explaining how to fix this issue for "library developers", mobx issue #1082.
My workaround to fix this problem, was adding the property resolutions to my package.json and use yarn as package manager. This is my package.json
{
  "name": "reactnativetestapp",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node node_modules/react-native/local-cli/cli.js start",
    "test": "jest"
  },
  "dependencies": {
    "mobx": "^4.1.0",
    "mobx-react": "^5.0.0",
    "react": "16.3.0-alpha.1",
    "react-native": "0.54.2",
    "react-native-elements": "^0.19.0",
    "react-native-router-flux": "^4.0.0-beta.28",
    "react-native-vector-icons": "^4.5.0"
  },
  "devDependencies": {
    "mobx": "^4.1.0",
    "mobx-react": "^5.0.0",
    "babel-jest": "23.0.0-alpha.0",
    "babel-plugin-module-resolver": "^3.1.0",
    "babel-plugin-transform-decorators-legacy": "^1.3.4",
    "babel-preset-react-native": "4.0.0",
    "jest": "22.4.2",
    "react-test-renderer": "16.3.0-alpha.1"
  },
  "resolutions": {
    "mobx": "^4.1.0",
    "mobx-react": "^5.0.0"
  },
  "jest": {
    "preset": "react-native"
  }
}

Then, I just run yarn install and react-native run-android.

Closing it now because beta.40 doesn't use mobx


[WIP] Implement sort method for SparsePauliOp

This PR adds a method sort to SparsePauliOp.
The motivation is to allow easily checking for equality. Currently the __eq__ method compares the data in SparsePauliOp, a table and array of coefficients, without regard to order, which may be different if the two operators are constructed in different ways. With this PR, one might call sparse_pauli_op.simplify().sort() on each instance before comparing them with ==.
No test has been written yet.

This appears to have been superseded by #8016, which is merged.  Please feel free to re-open if there's more to do.


Suggestions for improving performance rechunking MUR SST Zarr

👋  again
I’m working on rechunking a zarr store (MUR SST) using an S3 file system and a local dask cluster operating on an r5.16xlarge (20 workers). The rechunking (estimated time to complete right now would be about 100 hours) is going very slowly even though S3 buckets and ec2 are all in us-west-2. Workers have enough memory (using 30-40%) but are maxing out CPU (>100%, each worker has 4 cores).
The rechunking plan is going from {"time": 5, "lat": 1799, "lon": 3600} to {"time": 6443, "lat": 100, "lon": 100} for 4 variables (analysed_sst, analysis_error, mask and sea_ice_fraction), that is:
target_chunks = {
    'analysed_sst': {'time': 6443, 'lat': 100, 'lon': 100},
    'analysis_error': {'time': 6443, 'lat': 100, 'lon': 100},
    'mask': {'time': 6443, 'lat': 100, 'lon': 100},
    'sea_ice_fraction': {'time': 6443, 'lat': 100, 'lon': 100},
    'lat': None,
    'lon': None,
    'time': None
}
max_mem = '2GB'

array_plan = rechunk(ds_zarr, target_chunks, max_mem, s3_rechunk_store, s3_tmp_store)

You can also see the complete code in this notebook https://github.com/abarciauskas-bgse/mur_sst-to-zarr/blob/master/images/rechunker/Rechunking_Zarr.ipynb

Any advice? Should I try an instance with more CPU? @rabernat in you blog post you mentioned you are using 20 workers in GCS, do you know what instance types were being used for comparison?

Thanks for this detailed report.
First, I want to note that you  r5.16xlarge instance has 64 cores. You are using 20 workers with 4 threads each = 80 cores. So you are slightly oversubscribing your node.
In my example, I was using a distributed cluster. You are using a single machine. So one limiting factor could be the network bandwidth to your node. This node has a 20 Gbps network connection. That is the upper bound on how fast it can process your data. And the dataset is very big. These are the variables in your group
/
 ├── analysed_sst (6443, 17999, 36000) int16
 ├── analysis_error (6443, 17999, 36000) int16
 ├── lat (17999,) float32
 ├── lon (36000,) float32
 ├── mask (6443, 17999, 36000) int8
 ├── sea_ice_fraction (6443, 17999, 36000) int8
 └── time (6443,) int64

Here is how long it would take to read the entire dataset, assuming perfect bandwidth saturation:
total_bits = 6443 * 17999 * 36000 * (2 * 16 + 8 )
bandwidth = 2e10 # bits per second
time_seconds = total_bits / bandwidth
time_hours = time_seconds / 60 / 60

The answer is 2.3 hours.  The data have to be read and written twice, so, you multiply this by 4 and get to ~10 hours. This is still 10 times slower than what you are getting, so indeed we should be looking for some other opportunity for optimization.
Just out of curiosity, what if you try with fewer workers but more memory per worker. Do you get the same speed?

I am profiling this now in my own environment and am getting similarly slow performance. I'm trying to understand what's going on. I'm operating only on a single array, using 4GB of memory and a distributed cluster, but otherwise everything is the same. I have about 4800 tasks called from-zarr-store that take between 150s and 300s to run. I'm trying to figure out why the are so slow
To debug, I wanted to see how rechunker is reading and writing the data.
import rechunker

shape = (6443, 17999, 36000)
source_chunks = (5, 1799, 3600)
target_chunks = (6443, 100, 100)
itemsize = 2 # 16 bits
max_mem =<PHONE_NUMBER> # 4GB
read_chunks, int_chunks, write_chunks = rechunker.algorithm.rechunking_plan(shape, source_chunks, target_chunks, itemsize, max_mem)

This last line is ((305, 1799, 3600), (305, 100, 3100), (6443, 100, 3100)). The first value read_chunks defines each unit of reading from the source. The second, int_chunks, defines the chunk structure in the target array.
I wanted to see how fast it takes to manually read and write these units. Here is the code I used
import s3fs
import zarr
s3 = s3fs.S3FileSystem(client_kwargs=dict(region_name='us-west-2'))
s3_store = s3fs.S3Map(root='mur-sst/zarr', s3=s3, check=False)
ds_zarr = zarr.open_consolidated(s3_store, mode='r')
array = ds_zarr.analysed_sst

# time read
%time data = array[:305, :1799, :3600]
# -> 4.9 s

# set up a target store to write to with the same chunk structure as the intermediate array
import os
scratch_url = os.environ['PANGEO_SCRATCH']
s3_tmp_store = s3fs.S3Map(root=f'{scratch_url}/mursst/debug-write.zarr', create=True, s3=s3)
int_chunks = tuple([int(ic) for ic in int_chunks]) # needed to overcome a weird json error
target_array =  zarr.empty(shape, chunks=int_chunks, dtype=array.dtype, store=s3_tmp_store)

# time write
%time target_array[:305, :1799, :3600] = data
# -> 10.4 s

So the combined read and write took only 15s! This is indeed about 10x slower than the from-zarr-store task. So something is introducing some overhead that makes this operation go 10x slower once it is running in a distributed cluster.
My intuition is to blame s3fs, which in the past has introduced similarly unexpected overhead. But at this point, I want to ping @TomAugspurger  and @martindurant, who might have some insight on how to debug this.
Thanks Amy for a very useful performance report. Hopefully we can get to the bottom of this.

Do you have anything useful on the distributed profiling dashboard?

@rabernat thanks for this interesting analysis of read/write - interesting to see how you are inspecting that part of the performance. My only update is that I tried looking into

fewer workers so not oversubscribing (10 workers instead of 20)
in AWS instance monitoring it still didn't appear to be using all of the instance CPU. I'm trying to troubleshoot why this is by looking at my docker / ECS task configuration. The adjustment I made was to adjust the "CpuShares" for the docker container from 60 (which I think was too low) to 10240 (I also adjusted the memory limit from a hard limit of 50000 to a soft limit of 256000, although memory doesn't seem to be the constraint). So far the performance is about the same.

I think I'm going to try writing to disk and see if performance is dramatically different, this could isolate it to a read/write from S3 issue.

A performance report might be helpful.
Alternatively, I think the us-west2 jupyterhub has a pangeo scratch bucket. I can take a look later.

I'm trying to make a performance report, but the problem is that it would take 100 hours for the operation to finish! If you interrupt the operation early, then the performance report doesn't get written (which is annoying).
The alternative is to create a smaller, synthetic dataset, but that's another time sink.

If you interrupt the operation early, then the performance report doesn't get written (which is annoying).

Ah, right. That could be fixed.

I did something else to try to debug this. I manually examined the dask graph created by rechunker. I found keys like this, e.g.
('from-zarr-store-ed59826c-d0fd-11ea-9958-1edbd0c8a5a4', 0, 0, 0)

Then I timed the manual getting of this key:
%time _ = client.get(graph, keys)

And it came out to 15s. But when I call array_plan.execute() on the same client, the same tasks take 70s! Most of this time appears to be at the write stage.
I managed to output a profile for the full execution with a LocalCluster before interrupting it. It seems incomplete but maybe still useful?
https://gistcdn.githack.com/rabernat/5f290ca3950c94e9464abf091ab70e9e/raw/a4b9c64b1acb5e2d476ecd114a9b1904e710b409/dask-report-2GB-plan.html

If anyone (nudge nudge @TomAugspurger 😉) wants to play with this, you can start from this notebook, which runs on https://aws-uswest2.pangeo.io/
https://gist.github.com/rabernat/78f3aba046a9e1f00287adfb889f368d

Requesting access now :)
In the meantime, can you see how things perform with one thread per worker?
g = Gateway()
options = g.cluster_options()
options['worker_cores'] = 1

cluster = g.new_cluster(options)
cluster

The example I shared uses a LocalCluster with two workers, one core per worker.

As an update on my end (which I think is in-line with some of what @rabernat is finding) using the local file store for temporary storage is seeing much better performance. My back of the napkin calculation of time to complete is 14-15 hours. My plan is to let this run to completion. Of course the obvious draw back is that now I need a filesystem of 3TB for the temp storage!
I'm not sure what is causing s3fs write performance to be slow unless it has something to do with using SSL or caching.

I'll look into the s3fs performance issues. Writing a 128Mb (from a pod running on EC2) file gives me the following times
Method     | Time
---------- | -----
Local Disk | 300ms
AWS CLI    | 1.7s
s3fs       | 4.3s

import dask
import s3fs

data = b'0' * dask.utils.parse_bytes("128Mb")

%%time
with open("/tmp/test.txt", "wb") as f:
    f.write(data)
# 300 ms

%%time
!aws s3 cp /tmp/test.txt s3://pangeo-scratch/tomaugspurger/test.txt
Wall time: 1.71 s

%%time
with s3.open("/pangeo-scratch/tomaugspurger/tmp/test.txt", "wb") as f:
    f.write(data)
Wall time: 3.72 s

So we have a bit of room for improvement. I'd imagine that the AWS CLI's time is about as good as we can get for a single file.

Tom, my hunch, confirmed by the example I linked above, is that this is really somehow related to distributed + dask. If we could simply have all the tasks in the graph get the same  15s performance as I get when I just execute one, i.e.
= client.get(graph, key)

then it would go 10x faster.
It would be great to optimize s3fs further, but that seems perhaps orthogonal to the issue raised here.

I definitely saw improved performance using local filesystem for temporary storage, and the zarr rechunking completed faster than my estimate (which was rough), around 10 hours. This was using the r5.16xlarge instance type with 16 workers. I am going to test query performance tomorrow, but that is an unrelated performance investigation on different chunking approaches.

Tom, my hunch, confirmed by the example I linked above, is that this is really somehow related to distributed + dask. If we could simply have all the tasks in the graph get the same 15s performance as I get when I just execute one,

Right now my best guess is that this related to the int16 dtype of the array. I wasn't able to reproduce the relative slowdown (of rechunker / dask vs. a direct read / write) for int64 dtype arrays.
For int16 dtype, the single reads / writes do take longer (~3x longer) but I need to do a bit more digging tomorrow.

Short status update: there's something going on inside Dask that's making a single read / write cycle take 10-30% longer, but I'm not sure what yet.
def read_and_write(slice_, source_url, temp_url, int_chunks):
    # simulates a single read & write task in rechunk
    source_store = fsspec.get_mapper(source_url)
    source_array = zarr.open_array(source_store, mode='r')

    temp_store = fsspec.get_mapper(temp_url)
    target_array =  zarr.open_array(store=temp_store,
                                    shape=source_array.shape,
                                    chunks=int_chunks,
                                    dtype=source_array.dtype)

    # time read
    t1 = time.time()
    data = source_array[slice_]
    t2 = time.time()

    # time write
    t3 = time.time()
    target_array[slice_] = data
    t4 = time.time()
    return t2 - t1, t4 - t3

local_times = [read_and_write(int_slices[0], source_url, temp_url, int_chunks) for _ in range(5)]
remote_times = [client.submit(read_and_write, int_slices[0], source_url, temp_url, int_chunks, pure=False).result() for _ in range(5)]

>>> np.mean(remote_times, 0) / np.mean(local_times, 0)
array([1.72324109, 1.13201408])

So the only difference should be whether the function is called on a worker or not.
Looking at top, I notice that the version on the worker maxes out at ~55% CPU, while the jupyter python process gets up to 95% when it's doing the work. More later hopefull...

Have you checked the serialization time and size of the task?

Thanks for your sleuthing @TomAugspurger! I appreciate you digging into this.

Have you checked the serialization time and size of the task?

Good thought, but IIUC the times I reported are measured inside the function so serialization overhead shouldn't be an issue.

So the only difference should be whether the function is called on a worker or not.

Another thing to keep in mind is that user notebook pods and dask worker pods are running on different AWS instance types currently (so hardware and bandwidth settings won't be uniform):
CLUSTER	NODEGROUP	CREATED			MIN SIZE	MAX SIZE	DESIRED CAPACITY	INSTANCE TYPE	IMAGE ID
pangeo	core		2020-05-15T16:55:48Z	1		1		1			t3a.large	ami-0809659d79ce80260
pangeo	scheduler-spot	2020-05-15T16:55:48Z	1		20		1			t3.medium	ami-0809659d79ce80260
pangeo	user-gpu-spot	2020-05-15T16:55:48Z	0		4		0			p2.xlarge	ami-069280744807e927b
pangeo	user-spot	2020-05-15T16:55:48Z	0		50		0			m5.2xlarge	ami-0809659d79ce80260
pangeo	worker-spot	2020-05-15T16:55:48Z	0		50		0			r5.2xlarge	ami-0809659d79ce80260

Good point @scottyhq. However, I experienced this performance penalty even when using a LocalCluster!

Indeed, I'm deliberately using a LocalCluster to avoid that.
If / when we fix this issue, we'll still run into issues when multiple pods are on the same VM sharing the same network bandwidth. But we're a ways before we hit that bottleneck!

there's something going on inside Dask that's making a single read / write cycle take 10-80% longer

Just noting that this is not really the same order of magnitude that I found in my tests. It was more like 500-1000% longer!

Still not seeing any orders of magnitude relative slowdown, but thanks to @martindurant, we stumbled across this strange behavior: reading / writing Zarr Arrays is slower in a thread when using a Blosc compressor.
import numcodecs
import time
import numpy as np
import concurrent.futures

def encdec():
    times = []
    for i in range(5):
        x = np.random.RandomState(0).random(10_000_0000).tobytes()
        compressor = numcodecs.Blosc()
        t0 = time.time()
        a = compressor.encode(x)
        t1 = time.time()
        b = compressor.decode(a)
        t2 = time.time()
        times.append((t1 - t0, t2 - t1))

    return times

>>> local = encdec()
>>> tpool = concurrent.futures.ThreadPoolExecutor(max_workers=1)
>>> thread  = tpool.submit(encdec).result()

>>> np.mean(local, 0) / np.mean(thread, 0)
array([0.43173625, 0.34887189])

Dask workers are running tasks in a thread, so they suffer from this problem.
I wonder if @esc or @FrancescAlted are aware of this and have any insight? (a GitHub search didn't turn anything up).

That's funny because I cannot detect this on python-blosc.  I have come with the next translation:
import blosc
import time
import numpy as np
import concurrent.futures

N = 50_000_000

def encdec():
    # blosc.set_nthreads(1)  # you may want to experiment with this
    # blosc.set_releasegil(True)  # you mant to experiment with this
    times = []
    for i in range(5):
        x = np.random.RandomState(0).random(N)
        t0 = time.time()
        a = blosc.compress_ptr(x.__array_interface__['data'][0], N, x.dtype.itemsize, cname='lz4')
        b = np.empty_like(x)
        t1 = time.time()
        blosc.decompress_ptr(a, b.__array_interface__['data'][0])
        t2 = time.time()
        times.append((t1 - t0, t2 - t1))

    return times

local = encdec()
tpool = concurrent.futures.ThreadPoolExecutor(max_workers=1)
thread  = tpool.submit(encdec).result()
print(np.mean(local, 0), np.mean(thread, 0))
res = np.mean(local, 0) / np.mean(thread, 0)
print(res)

And the result of this on my macbook is:
/Users/faltet/miniconda3/bin/python /Users/faltet/blosc/python-blosc/bench/single-thread-python-blosc.py
[0.21427293 0.11373897] [0.20109401 0.11593528]
[1.06553612 0.98105571]

Whereas the equivalent code using numcodecs gives:
/Users/faltet/miniconda3/bin/python /Users/faltet/blosc/python-blosc/bench/single-thread.py
[0.20872107 0.10168438] [0.3177659  0.21424971]
[0.65683909 0.47460686]

So, it looks to me that the numcodecs single-worker version has an important overhead compared with the equivalent in python-blosc.  As the threading machinery in Python is the same in both cases, the blame should be on the numcodecs side.  By looking at the absolute time numbers (first row of figures), it seems to me that the 1-worker numcodecs code might be using just 1 blosc-specific thread and not several internally.
Another possibility is some kind of contention between the Python thread workers and Blosc threads, so I am wondering whether the GIL has something to do with this. @robbmcleod contributed a set_releasegil() for python-blosc a while ago, so you might want to experiment with that in numcodecs too.  Maybe Robert can also suggest something obvious that escapes to me.

Very informative, thanks @FrancescAlted! It's good to hear that this isn't a limitation in blosc itself. I'll write up an issue for numcodecs.

Switching threads (sorry) back to the larger picture, I have a couple things.

The numcodecs threading issue gives a 10-50% slowdown on Dask workers.
For "small" rechunking operations (~20 read/write pairs of 1GB each), I don't see any issues beyond the threading issue: (notebook, performance report)
For larger operations like the MUR SST (~9000 read/write pairs of 2GB each) I see a few things
3a: A Dask scheduling failure: All ~4500 intermediate read/write tasks are assigned to a single worker. Dask's task stealing eventually kicks in and other workers take over. There's a couple solutions here, which I'll look into. Easiest short-term fix is setting a duration in the Dask config to explain that these store tasks are going to take a while: https://docs.dask.org/en/latest/configuration-reference.html#distributed.scheduler.unknown-task-duration

3b: Variable task durations for the read / write. Anywhere from ~20s (expected) to 90. I'm less sure what's causing this. Something in Dask / zarr? Multiple pods sharing the same network interface? S3 itself?

3. A Dask scheduling failure: All ~4500 intermediate read/write tasks are assigned to a single worker.

FWIW, this happens to me on basically every Pangeo workflow I ever touch that starts from a zarr dataset. I thought it was normal behavior.

3. 3b: Variable task durations for the read / write. Anywhere from ~20s (expected) to 90. I'm less sure what's causing this. Something in Dask / zarr? Multiple pods sharing the same network interface? S3 itself?

This is, IMO, where we have the biggest opportunity to optimize or at least understand what is going on. I had some tasks taking up to 200s! At some point, I did an experiment where I ran client.get(graph, keys) on a substantial list of keys, and they all went fast. I don't know what that means, but it could be a clue.

It may be S3: since the writes have the same prefix down a number of directories, it is highly likely that the data are being stored on the same machine, so the incoming bandwidth of that machine might saturate ( detailed blog about naming for performance: https://aws.amazon.com/blogs/aws/amazon-s3-performance-tips-tricks-seattle-hiring-event/ )

Over in #37, I have added a prefect executor for rechunker. It would be very interesting to see if it has the same problem when executed with a dask scheduler.
We could also try to rewrite the dask executor bypassing dask.array and using a lower-level delayed structure.

It may be S3: since the writes have the same prefix down a number of directories, it is highly likely that the data are being stored on the same machine

@martindurant - this is a really good idea. We have known about this limit for a while but never actually tested whether we were hitting it. In @abarciauskas-bgse's original example, she was doing everything from one compute node and not coming close to saturating her network. My guess would be that we were not yet constrained by the S3 prefix issue. But I'd love to see evidence of that.
In any case, the solution--adding a random hash prefix to all our object keys--really messes up the way we are currently organizing our data in these buckets. A workaround would require some major changes...but should be possible with careful thought.

A dask from_zarr has a graph like

i.e., the zarr object is a data key, rather than included in the closures of the functions that use it. I assume xarray's version does the same. Is this enough to get all the dependents to get scheduled to one worker?

I assume xarray's version does the same.

Yes. Plus rechunker doesn't use xarray at all.

Thanks all for this investigation - all looks like interesting findings, just acknowledging the thread so far and once I return to the chunking evaluationo I will add anything I find or use from this issue.

Just noting as a follow up to the suggestion that we could be hitting an S3 network limitation (h/t @bilts), according to Amazon this limitation should no longer exist: https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance

Amazon S3 now provides increased performance to support at least 3,500 requests per second to add data and 5,500 requests per second to retrieve data, ... Each S3 prefix can support these request rates

i.e., the zarr object is a data key, rather than included in the closures of the functions that use it. I assume xarray's version does the same. Is this enough to get all the dependents to get scheduled to one worker?

All dependencies being tied into a single data key also seems to be the cause of some other scheduler issues I'm seeing with zarr + dask: https://github.com/dask/dask/issues/6668


serverFill.reservedSlots does not work

Hi! Cool project and I love seeing where this is going.
I have stumbled across a small bug, when I set serverFill.reservedSlots like:
maxPlayerCount: 16
serverFill:
  reservedSlots: 4

the actually set value is incorrect:

Values in .mantle-state.yml seem to be correct:
maxPlayerCount: 16
...
socialSlotType: Custom
customSocialSlotCount: 4

Interesting bug, thanks for report. Will look into as soon as possible.

This has been fixed in v0.10.2


Feature request: Return to dock stateless switch

Is it possible to implement this feature ? I think it would be useful for canceling current cleaning or return robot to dock after pausing it

Hi @mrskycriper, do you think it would work if the cleaning indicator remains active when paused?
This way we'll simply be able to turn off the cleaning indicator to send it back to the dock. Does it make sense?

For me it make sense and would love to see in the net version.

I move my vacuum between floors, so when it finishes cleaning upstairs, I take it down and command it to go to the dock.
Would be a blast to tell my HomePod to do it

Even when it may be counter-intuitive, if you tell your Homepod to turn off the vacuum, it does that. I've used it in the past.

Even when it may be counter-intuitive, if you tell your Homepod to turn off the vacuum, it does that. I've used it in the past.

@afharo thanks for the tip! Anyone else coming across this and wondering how to do this: you can create a scene in HomeKit to make it more intuitive, by naming it with the command you would like to issue to Siri. For example, I named my scene: Stop Cleaning, and set the fan speed to 0 and the water box to 0.

Warming this up in the context of the "go to" button. Because the vacuum is never turned on for this function, you are not able to turn it off and thereby sending it back to the dock. This would be useful, if you were just passing your home and the vacuum should return to the dock before the battery goes low.

Warming this up in the context of the "go to" button. Because the vacuum is never turned on for this function, you are not able to turn it off and thereby sending it back to the dock. This would be useful, if you were just passing your home and the vacuum should return to the dock before the battery goes low.

From my point of view this is solved by #544.


Language server

New items in the PR
Resolves #51
Must pull https://github.com/terrastruct/d2/pull/1534 for this to work.
Project Layout
To facilitate a better edit/build/debug process, the extension was moved to a 'client' directory, the language server to a 'server' directory. Note: This may break the nightly build process and will need to be adjusted.
Language Server

Identifies links and imports so the editor can mark
them as such. Ctrl+click on the links will attempt
to launch what the link points to.

'Find All References' will show a list (as presented
by vscode) of all references of the given node.

'Goto Definition' will do one of two things. If there
are multiple definitions, it will show a list similar
to 'Find All References'. If there is only one option
for the definition, then the cursor will jump to that
option.

'Rename...' will rename a node. If there are multiple
nodes (a.style... / a.shape... / a.label... / etc.),
then a rename will change all locations that contain
that node.

Auto Completion:
a) Typing a '@' will show a list of d2 files that are contained in the d2 documents current folder and all child folders from there.
b) Typing a '.' that is along a valid path, will bring up the proper list for that position in the path.
c) Typing a ':' will bring up a list of possible values for that position in the current path. If there is no list, in the case of a free-form field such as 'label', then no list is shown.
d) The 'Tab' key will commit the auto completion.

When a preview is not present, then there is no error messages generated, the language server will show any errors that are created during the AST phase of d2 compilation.

(apologies for the delay on this, team is super busy with some d2 studio stuff, very looking forward to digging into this and trying it soon)

go to definition worked for objects at the outermost level but doesn't seem to work for nested objects

:100:

I wonder if this can check for any d2 files inside the workspace folder rather than just the current .d2 file's folder? currently it doesn't seem to be able to lookup anything ../etc even if it is open within the workspace

@ and . work great but I couldn't get : to work as described
. 💯

:

It would be nice to have children appear here too but it could be a separate issue:

The exception that was firing on the JSON parse was due to the refactoring of the D2 executable code.  When a parse happens, it should be allowed to, "fail", so that the ast and errors can be picked up and transmitted to the extension.  The parser being fault tolerant allows for incomplete d2 code to be parsed and that gives enough information for autocomplete to work. The code was just returning because there was an error, (which happens about every fifth character you type as you create a diagram), and that exception was that the ast was empty.  I've added a little more error checking around that scenario and have a PR ready for the D2 code tree.

Workspaces are now honored for d2 file search.  Nested objects work correctly.  I took out my, 'I know better', code from the ':' path for autocomplete. Now we should be at the point of, this works, but what else would be cool to add?

I just want to say I already love this PR so much. This is literally the only feature that I miss in D2 (and my most important selling point). Thank you, @BarryNolte, for working on this.

Now we should be at the point of, this works, but what else would be cool to add?

What do you (all) think about releasing a beta of the LSP + VSC Extension? Things that are "cool to add" are probably worth looking into, when the LSP proved to be stable enough. Most cool things will show when the community uses it and finds special use cases. This would probably also decrease the time needed, as features could be prioritized by demand.
(Just my 2 cents)
Cheers

I just want to say I already love this PR so much. This is literally the only feature that I miss in D2 (and my most important selling point). Thank you, @BarryNolte, for working on this.

Thanks @MarvinJWendt!  This has been fun to work on.  It's a far cry from my previous work on performance tools and debuggers.  This should be available once the team at Terrastruct accepts the PR.

@alixander and @gavin-ts Ping.

@BarryNolte I'm running into this error when I try to install. Any ideas?
Error at the end: ERROR Extension entrypoint(s) missing.

I moved a ton of files around to separate the client from the server, so this could be leftovers from the previous directory structure.  I'd delete the whole directory, re-clone the repository and build from there.  As I moved things around, I ran into numerous issues just like this that were fixed by deep cleaning the node modules directories and deleting ./dist/... directories.

it must be that it's building as .ts and vscode is looking for .js?

I checked out d2:master and it appears to be behind my d2:LSP-Fixes branch.  If you can checkout upstream/master, it should have my changes and the error spew will go away.  All new features are described in the PR Entry #117

When I started this, I was passing the IR to the server, but then I figured out how to synthesize what I needed from the AST.  However, you found a limit that I should go back to using the IR for references and nested objects.  I'll work on that.
The autocomplete of ':' vs. ': ', I also thought was a good idea, and had it implemented.  It didn't quite feel right, at least to me.  So I tried a bunch of different languages, and every one of them auto completes on the token delimiter ('.', ':', etc).  The vscode autocomplete system is also geared toward this implementation. But, since I know I want a space in there, when you pick an item to autocompletion, it will put the space in.

Can we split out the smaller ones from this large PR and have this just be focused on the language server? @BarryNolte


Fix lint errors for analyzer mtls package

@selmanj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

unit-tests_istio
a87ff2a309cac33e3ad194519d0184f757052ac4
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test unit-tests_istio

@selmanj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
a87ff2a309cac33e3ad194519d0184f757052ac4
link
/test pilot-multicluster-e2e_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test pilot-multicluster-e2e_istio


Implement new StorageAdapter interface

Closing in favour of https://github.com/grammyjs/chat-members/pull/6


Create govulncheck.yml

Add a GitHub Actions workflow to run govulncheck.
According to govulncheck docs:
"Govulncheck reports known vulnerabilities that affect Go code.
It uses static analysis of source code or a binary's symbol table
to narrow down reports to only those that could affect the application.

By default, govulncheck makes requests to the Go vulnerability database
at https://vuln.go.dev. Requests to the vulnerability database contain only
module paths, not code or other properties of your program."

@x448 can you take a look to see if I missed anything?  :pray:

I forgot to squash so it's messy but I don't want to do a force push to cleanup.


[docs] core/document_selection.dart

Summary
As I wanted to learn about and gain a deep understanding of the super_editor API in general, I explored the individual parts of the source code.
In this process, I figured it would be very useful to add public member API docs accordingly and contribute them back to the project.
Part
In this pull request in particular, I am only adding documentation to the core/document_selection.dart file, exclusively. This should hopefully make it easier to review the individual parts of my contribution.
Guidelines
Because we need to have a common standard for documentation in order to make the developer experience when researching about the API coherent, I strictly followed Effective Dart.
In this case, the relevant articles are:

Effective Dart: Documentation
And Effective Dart for a general overview.

Assertions
I do consider assertions as part of docs in the scope of these pull request. This is because I feel they have a documenting value and can enforce what the docs state using debug-time warnings.
Imports
Any added imports in the docs changes have been made in order to create references to members from other files to allow the analyzer to recognize them. This is necessary for code navigation e.g.
This means that if I added an import to 'package:super_editor/src/default_editor/text.dart', then this happened because [TextNodePosition] was referenced in the docs.
Meta
I do think enforcing these rules to ensure coherency is important as well, which is why we should consider adding meta documentation for: how to contribute and which guidelines to follow. This might also involve adding the public_member_api_docs lint rule.
Full contribution
To query the related PRs that make up this effort of contributing docs, you can use this query.
Reviews
I hope that these contributions can be merged with the existing efforts. I tried to really understand what the API does and how and why it works the way it does and why it has been designed the way it has been designed.
Nonetheless, my documentation should of course be thoroughly reviewed. I am not sure who is best-suited for each individual review. cc @matthew-carroll @salihgueler

Thanks @matthew-carroll - I applied most suggestions from your code review. Now, there is only the open questions about bool getters. Once that is resolved, I can make any last adjustments if necessary.

@matthew-carroll Let me know when you have made a decision on https://github.com/superlistapp/super_editor/pull/221#discussion_r645969982 - I will adjust accordingly then 👍


What does TB_INPUT_MOUSE do?

I read the docs in termbox.h, and a bit of the code, but it still isn't totally clear. Thanks :)

It enables mouse support so that you receive, TB_KEY_MOUSE_* events. The x and y fields in struct tb_event will be set to the coordinates where the mouse was scrolled/clicked.�

Added some docs. Thank you @mitchellwrosen and @git-bruh.


Fix typo in README.md

FlexibleConstraints -> FlexibleContexts

Although I think FlexibleConstraints would be an apt name for what's going on here


Updated documentation of performace.plot_cumulative_returns

related to #187
Explained how to interpret the cumulative return for N forward period in
performace.plot_cumulative_returns and performance.plot_cumulative_returns_by_quantile

just documentation, I'll merge this


update the method for ARD kernels

@karlnapf
take a look at this

Changes Unknown when pulling 82808681ad6e42f367f9fb98c240a1db0d1a0c54 on yorkerlin:ardkernel into ** on shogun-toolbox:develop**.

@karlnapf
I will add a unit test for the ARD kernels.
Before I update this PR, please do NOT merge this PR.

@karlnapf
please take a look at this PR again. Please try to merge this PR first.
I add a unit test for the ARD kernel.
Let's see whether the bug related to ViennaCL comes from the ARD kernel or FITC inference.

Changes Unknown when pulling 2c8c39ec01bfccedd93cdea827c9046a899e156f on yorkerlin:ardkernel into ** on shogun-toolbox:develop**.

Changes Unknown when pulling 15e36ea0fbeda3606b19df07dda26ef7288483ba on yorkerlin:ardkernel into ** on shogun-toolbox:develop**.

So I am confused now, what should I merge first?

If this PR is fine for you, please merge this PR first.
On Saturday, April 4, 2015, Heiko Strathmann<EMAIL_ADDRESS>wrote:

So I am confused now, what should I merge first?
—
Reply to this email directly or view it on GitHub
https://github.com/shogun-toolbox/shogun/pull/2796#issuecomment-89604691
.

--
best,
wu lin

Python error seems unrelated to merging


adjustments to sendable

motivation: events may be used in an async context and as such musct conform to Sendable
changes:

define common abstraction for Sendable & Codable
conform all events Sendable

Superseded by #15.


Rekor Verifier should match artifact hash

actually I think we should just recreate the entry and compare it.


monocypher: 3.1.3 -> 4.0.2

Changelog: https://monocypher.org/changelog
This package had a nixos specific patch, but I don't think it's still needed in this version.
I tested on x64, maybe it was needed for a different architecture?
@sikmir
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This package had a nixos specific patch, but I don't think it's still needed in this version.

You are right, that patch is part of 4.0.2 now (https://github.com/LoupVaillant/Monocypher/pull/239).


PROD : no results when searching with a keyword

Currently the keyword search does not work on PROD.

open visualize.admin.ch
create a visualization
Enter a keyword for example Modell, bade etc.
no results

Visualize environment and version: v3.16
Browser and version : edge

Additional context
Add any other context about the problem here.

Do you still reproduce the problem ? I cannot reproduce. @l00mi Would you have any idea ?

oui juste maintenant, avec Edge, mais c'est instable. parfois ça marche, parfois pas.
je voulais quand même le signaler

I noticed the same problem on March 2nd around 9am and I had made a video but 10 minutes later no more problem so I didn't post it
https://user-images.githubusercontent.com/79834090/223469321-a0e68037-782d-426b-89c6-b92619a155d8.mp4

Also could not produce when tested on my device at 19:00. Tested on Chrome, FF, Safari, and Edge on Windows 11 and will retest tomorrow to see if I can reproduce.


generate train.lst from VOCdevkit in example/fcn-xs

i download the VOC2012 from http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar, when i extract the tar file, i don't find the *.lst file. because i am a greenhand in segmentation, i don't know how to generate these lst file, i would really appreciate it if there is anyone who can tell me how to make it, thanks

from https://github.com/dmlc/mxnet/tree/master/example/fcn-xs#how-to-train-fcn-xs-in-mxnet we can go to https://pan.baidu.com/s/1bgz4PC#list/path=%2Ffcn-xs

@tornadomeet I find it in that page, but it seems to be slow to download from Baidu Yun, so i download the original data from http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar first, which is much faster than Baidu Yun. but the situation now is that i have to generate the .lst file on my own

@animebing  i see. i just put the list file in https://pan.baidu.com/s/1bgz4PC#list/path=%2Ffcn-xs, it's only 160kb, so you can download.

@tornadomeet Thank you very much, it helps me greatly

@animebing be careful with newest mxnet, https://github.com/dmlc/mxnet/pull/3654


Update docs about search completions

Added a simple configuration that allows nvim-cmp to autocomplete search suggestions.

Thank you~.


Added 'name' to valid kwargs on update_project.

This change will make it possible to update the project name, just like the openstackclient CLI.

Thank you for contributing to openstack-infra/shade!
openstack-infra/shade uses Gerrit for code review.
If you have never contributed to OpenStack before make sure you have read the
getting started documentation:
http://docs.openstack.org/infra/manual/developers.html#getting-started
Otherwise please visit
http://docs.openstack.org/infra/manual/developers.html#development-workflow
and follow the instructions there to upload your change to Gerrit.


Clarify the effects of startprovisioner

It is not so intuitive that you can execute startprovisioner without staking. According to the documentation it is necessary to stake.
"stake": Stake a given amount of DUSK, to allow participation as a provisioner in consensus. Make sure to use the sync command first.
What is actually happening when you run startprovisioner without staking?
Is it a deliberate choice to let users sync manually instead of syncing automatically? What is the risk of staking without syncing?

Hey! You're right, it is a bit convoluted from a docs perspective, so allow me to clarify:
When you type startprovisioner into the console, it starts up all of the codebase components needed to be able to function as a provisioner. The provisioner components do not need to know about any specific stake - this is because the consensus keypairs are deterministically generated from the wallet seed on startup, and passed to those components. Since the running stakes are managed by a Committee object, the components can infer themselves whenever it is appropriate to vote. This is in contrast to the block generation components, which need to have specific values from a bidding transaction in order to generate the bids.
That said, the provisioner components will not actually start their activities until a seperate component discovers a stake belonging to the node in any incoming blocks. This component will supply a starting round to the provisioner components, who will then begin their duties. So, when startprovisioner is ran without any active stake, the components will serve to relay consensus messages, and they will wait for a starting round to be propagated inside of the node. Therefore, it is possible to run startprovisioner without staking, and then stake later on.
Let me know if that is clear - I am happy to clarify anything. In the meantime I will reword the issue to adjust documentation for this to be clearer, and possibly open another issue for this point:

Is it a deliberate choice to let users sync manually instead of syncing automatically? What is the risk of staking without syncing?

No guards are established for this. It is fine to stake without syncing (as the stake is included in a block that's appended to the current tip), though running startprovisioner after staking while syncing could lead to some unexpected behaviour. I think it could lead to a node starting on an obsolete round, which is undesirable as it would get stuck. I would need to investigate it though.

Closing this, as #27 makes this issue obsolete.


Quotes are included in output when parsing with relax: true, trim: true

Repro:
csv.parse('"A" , "B" , "C"', {relax: true, trim: true}, function (error, output) { console.log(output) })

Expected:
[ [ 'A', 'B', 'C' ] ]

Actual:
[ [ '"A"', '"B"', 'C' ] ]

This worked in<EMAIL_ADDRESS>Just upgraded to 1.0.0 (via<EMAIL_ADDRESS>and I'm seeing this new, unexpected behavior.

i could be bugged, i never considered parse as stable but more in the philosophy "ok, you got invalid csv, if parse can help you, use it". If i remember, i tried to work around this probleme not so long ago and give up. it's actually quite hard to fix but possible of course

old issue, closing


Probable OOM in JMS connector due BRuntime invoke method is non blocking

Description:

This affects the service listener implementation
relates to: https://github.com/ballerina-platform/ballerina-lang/issues/18010

Fixed with PR https://github.com/wso2-ballerina/module-jms/pull/24


PDDLAnytimePlanner refactoring

Codecov Report
Patch coverage: 93.75% and project coverage change: +0.01 :tada:

Comparison is base (ae9b78b) 86.98% compared to head (f4b4920) 87.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #396      +/-   ##
==========================================
+ Coverage   86.98%   87.00%   +0.01%
==========================================
  Files         186      187       +1
  Lines       23857    23897      +40
==========================================
+ Hits        20752    20791      +39
- Misses       3105     3106       +1

Impacted Files
Coverage Δ

unified_planning/engines/pddl_anytime_planner.py
90.76% <90.76%> (ø)

unified_planning/engines/__init__.py
100.00% <100.00%> (ø)

unified_planning/engines/pddl_planner.py
69.90% <100.00%> (+0.85%)
:arrow_up:

unified_planning/test/pddl/enhsp.py
98.83% <100.00%> (+3.07%)
:arrow_up:

... and 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

@roeger @hstairs @IvanSerina
This PR aims to implement a common PDDLAnytimePlanner to avoid code duplication between planning engines.
Any feedback?

@alvalentini I just tried to use the new class in the Fast Downward integration but there is a situation, where I am not sure how to cover it. The anytime search in Fast Downward terminates without a timeout once the search space has been exhausted. If a plan has been found earlier, my integration previously returned the last found plan with status SOLVED_SATISFICING.
With the current implementation, I would have a sequence of intermediate plans that at some point stops without any final plan. Is this the user interface we want? If not, what's your advice?


Epic: proxy. Connectivity errors

We have to resolve connection errors that our users see
### Tasks
- [ ] https://github.com/neondatabase/neon/issues/5510
- [ ] https://github.com/neondatabase/neon/issues/5511
- [ ] https://github.com/neondatabase/neon/issues/5568
- [ ] https://github.com/neondatabase/neon/issues/5569
- [ ] https://github.com/neondatabase/neon/issues/5654
- [ ] https://github.com/neondatabase/neon/issues/5570
- [ ] https://github.com/neondatabase/neon/issues/5800
- [ ] https://github.com/neondatabase/neon/issues/5801
- [ ] https://github.com/neondatabase/neon/issues/6291
- [ ] https://github.com/neondatabase/neon/issues/6292
- [ ] https://github.com/neondatabase/neon/issues/6293

Implementation work is done. What is left is documentation and is tracked in a separate issue.


P4RT-6.1: Performance

@dplore,  sharonshenzhen would like to sign up this test case.  Can you assign it to her ?
Thanks,
Joann

I have invited @sharonshenzhen to join the contributors.  When they accept I will assign, thank you.

@joannc2022 can you ensure that  @sharonshenzhen accepts the invitation so this item can be assigned?

Hi Darren,
@sharonshenzhenhttps://urldefense.com/v3/__https:/github.com/sharonshenzhen__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orjXEvJTH$ has accepted the invitation last week on Oct-12th
If you haven't seen it, please resent it today.
Regards,
Joann
Juniper Business Use Only
From: Darren Loher @.>
Sent: Monday, October 17, 2022 11:34 AM
To: openconfig/featureprofiles @.>
Cc: Joann Chang @.>; Mention @.>
Subject: Re: [openconfig/featureprofiles] P4RT-6.1: Performance (Issue #77)
[External Email. Be cautious of content]
@joannc2022https://urldefense.com/v3/__https:/github.com/joannc2022__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06oru_3OqTp$ can you ensure that @sharonshenzhenhttps://urldefense.com/v3/__https:/github.com/sharonshenzhen__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orjXEvJTH$ accepts the invitation so this item can be assigned?

Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/openconfig/featureprofiles/issues/77*issuecomment-1281307363__;Iw!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06oroVS1j0J$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AYR2BSEKIN3IZJTSKPBPZIDWDWLZPANCNFSM5PH6IEYA__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orhTg6YZc$.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Hi Darren,
I have sent email to you last week to confirm this.
Attached is the email.
Thanks,
Sharon
From: Joann Chang @.>
Date: Monday, October 17, 2022 at 11:51 AM
To: openconfig/featureprofiles @.>, openconfig/featureprofiles @.>, Darren Loher @.>
Cc: Mention @.>, Joann Chang @.>, Sharon Zhang @.>, Girish Subramanian @.>, MohanaSundari M @.>, Kiran KL @.>
Subject: RE: [openconfig/featureprofiles] P4RT-6.1: Performance (Issue #77)
Hi Darren,
@sharonshenzhenhttps://urldefense.com/v3/__https:/github.com/sharonshenzhen__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orjXEvJTH$ has accepted the invitation last week on Oct-12th
If you haven’t seen it, please resent it today.
Regards,
Joann
Juniper Business Use Only
From: Darren Loher @.>
Sent: Monday, October 17, 2022 11:34 AM
To: openconfig/featureprofiles @.>
Cc: Joann Chang @.>; Mention @.>
Subject: Re: [openconfig/featureprofiles] P4RT-6.1: Performance (Issue #77)
[External Email. Be cautious of content]
@joannc2022https://urldefense.com/v3/__https:/github.com/joannc2022__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06oru_3OqTp$ can you ensure that @sharonshenzhenhttps://urldefense.com/v3/__https:/github.com/sharonshenzhen__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orjXEvJTH$ accepts the invitation so this item can be assigned?
—
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/openconfig/featureprofiles/issues/77*issuecomment-1281307363__;Iw!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06oroVS1j0J$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AYR2BSEKIN3IZJTSKPBPZIDWDWLZPANCNFSM5PH6IEYA__;!!NEt6yMaO-gk!CwM56_BGBmyFGbqi1tEjHtHCNwizjVCj0tAPaa1O1ThDzHVTjm_nt00gzwZ9oYluJFPbbI6cLGjPG06orhTg6YZc$.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>Juniper Business Use Only


[SPARK-44608][SQL] Remove unused definitions from DataTypeExpression

What changes were proposed in this pull request?
https://github.com/apache/spark/pull/41559 have defined some DataTypeExpression but did not use, such as ByteTypeExpression,  ShortTypeExpression, etc. This pr cleans them up and we can define them when we really need.
Why are the changes needed?
Code cleanup
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass GitHub Actions

@amaliujia Can we temporarily delete this useless definitions?

@LuciferYang we added this for the completeness of DataTypeExpressions. It is just natural that each data type has its DataTypeExpression.
Does it cause any problem so far?

@amaliujia These are some unused and untested code, and they are public. If they are released in Spark 3.5, we have an obligation to continue maintaining them and ensuring compatibility, even if they will never be used, which itself is a risk. So I prefer to define when there are actual needs


Change Volume does not show hidden files/directories on macOS

Expected behavior
When selecting the local folder for a volume, one would expect to see the same contents in the dialog as what can be seen in the Finder, like hidden files/folders (revealed with defaults write com.apple.finder AppleShowAllFiles YES):

Actual behavior
The Kitematic local folder browser does not show hidden files / directories:

Information about the Issue

macOS 10.12.3
Kitematic 0.13.0
Docker 17.03.0-ce-mac2 (15654)

Steps to reproduce the behavior
On an existing container (that exposes volumes):

Navigate to Settings > Volumes
In the Configure Volumes section, click Change.

We use the normal electron 'Show Finder' and don't have a ton of leverage to show more/less on the window. I would check your default preferences for new Finder windows opened as I get the exact same window in my setup.

Thanks, here's the solution that helped me for the record:
http://apple.stackexchange.com/questions/100384/how-to-get-open-file-selector-to-show-hidden-files
The trick is that file dialogs are not considered the same as Finder windows, therefore respond to different sets of properties.

Thanks for sharing your fix @Franky47


Please Release Latest Version

I would like to use the latest version of Redux-Form to avoid the bugs that it has fixed, but it is giving me the warnings that were fixed in #3387. Is there any reason a 7.0.5 can't be released with the fix? I know I'm not alone in having spent a bunch of time trying to track down where the warnings came from and since the problem has already been solved it seems like a waste not to release the fix.

Some PRs were merged containing not only bugs, but features as well, so a 7.1.0 release is more appropriate.
If @erikras didn't release anything yet, there is a very high chance that some things still need to be done.

A release is coming soon, but I've been fighting with React 16 lately. Moreso with the deprecation of expect, which required migrating all the tests over to jest.

ah got it. alright.