Use EvalInteraction instead of EvalScript for Interaction Directives

@dsyme Continuation from https://github.com/Microsoft/visualfsharp/issues/1392
Repro steps
test.fsx
failwith "test"

let scriptContents = System.IO.File.ReadAllText("test.fsx")
//session.EvalInteraction("#line 1 \"test.fsx\"")
session.EvalInteraction(scriptContents)

Expected behavior
System.Exception: runtime error
   at <StartupCode$FSI_0006>.$FSI_0006.main@()
Stopped due to error

Actual behavior
 failwith "test"
 ^^^^^^^^^^^^^^^

stdin(5,1): error FS0030: Value restriction. The value 'it' has been inferred to have generic type
    val it : '_a
Either define 'it' as a simple data term, make it a function with explicit arguments or, if you do not intend for it to be generic, add a type annotation.

Related information
This should be a workaround to properly load a script with interaction directive support, but its not working.
Should I add the string "; ();;" to the script contents?

Appending \n() seems to work. I'm not sure if this handles all cases.
The only way I could make it work was:
let scriptContents =
  sprintf "#line 1 @\"%s\"\n" scriptFile +
  System.IO.File.ReadAllText scriptFile +
  "\n()"
x.EvalInteraction scriptContents

Not sure if this is the recommended way of doing things.

@matthid If I start fsi.exe and enter the input
failwith "test";;

then we get the same behaviour as "Actual" above - so EvalInteraction is just doing what fsi.exe has always done.  So it looks like the correct behaviour for EvalInteraction given that input.
Note that fsc.exe and #load will accept a file containing the above, i..e the value restriction is not applied.  This is because of the following:

For fsi.exe, the result of the last expression in the interaction (the failwith) is implicitly assigned to a value called "it".  The value restriction is applied to this value.

For fsc.exe, the result of expression is discarded, so the value restriction is not applied.

For fsi.exe, the result of the last expression in the interaction (the failwith) is implicitly assigned to a value called "it". The value restriction is applied to this value.

Question is: How can I do the same? EvalInteration was already the suggested way for implementing a "real" #load, but as shown here it's not equivalent.

Question is: How can I do the same?

From the behaviour you report above in the "Actual" section it looks to me like EvalInteraction is doing exactly this.

EvalInteration was already the suggested way for implementing a "real" #load, but as shown here it's not equivalent.

Equivalent to what?  Perhaps I don't understand what you mean by a "real" #load.  EvalScript is the equivalent of a #load.  EvalInteraction is the equivalent of using the input text as an interaction (incuding #cd, binding-to-it etc.).  It looks like you're doing the right thing if you want the former, and that the behavior above is the correct behavior when processing text as interactions.

This is related to https://github.com/Microsoft/visualfsharp/issues/1392. I want to have a #load with behavior (3) from https://github.com/Microsoft/visualfsharp/issues/1392#issuecomment-237859375, but I want it lazy in an evaluation session (after I setup some state via EvalInteraction).

@matthid OK, so this is a feature request for FCS right?  You're after a behavior that can't be done with either F# Interactive or the F# compiler today?
Specifically, you want an EvalInteraction that doesn't actually run the initialization code of the interaction but instead just checks/compiles it and then uses on-demand initialization?
If so, could you explain the scenario a bit more to help me understand why this is needed

Specifically, you want an EvalInteraction that doesn't actually run the initialization code of the interaction but instead just checks/compiles it and then uses on-demand initialization?

I'm not sure.

OK, so this is a feature request for FCS right?

Ideally I'd like to see this implemented in the compiler (see Use case 1 below).

You're after a behavior that can't be done with either F# Interactive or the F# compiler today?

No. It can already be done. It's just fsi script.fsx. Ideally I'd like to see the exact same behavior when using "#evaluate 'script.fsx'" (ie evaluating line by line instead of compiling the whole thing). I think people actually expect "#load" to do the same thing already (at least for scripts, but that might be only my personal feeling)
Maybe it makes more sense after the use cases...
Use case 1:
See https://fsharp.slack.com/files/cloudroutine/F1X9JV4V7/bootstrap.gif
Consider having two scripts which use Paket or nuget to bootstrap (note: this is untested dummy code):
test1.fsx
// ...
webClient.Download("nuget.client.dll")
#r "nuget.client.dll"
Nuget.Client.Restore("deps.config")
// .. more restore action

#r "package1.dll"
#r "package2.dll"
#r "package3.dll"

// Regular script logic of test1.fsx
// Use other interactive directions like #cd and stuff

test2.fsx
// ...
webClient.Download("nuget.client.dll")
#r "nuget.client.dll"
Nuget.Client.Restore("deps.config")
// .. more restore action

#r "package1.dll"
#r "package2.dll"
#r "package3.dll"

// Regular script logic of test2.fsx
// Use other interactive directions like #cd and stuff

Now you want to "extract" the common bootstrapping logic to a shared script -> Doesn't work with #load.
Use case 2
Consider test1.fsx from above. Now consider that you want to provide a scripting environment via FCS with predefined variables (ie executing EvalInteraction before loading the script). But still allow people to use the same bootstrapping code (ie at the same time allow test1.fsx to be executed as is).

@matthid Thanks
Ah OK, thanks
The primary FCS issue seems to be that EvalInteraction doesn't do fragment-by-fragment evaluation. Let's change this issue to cover that .
The #load discussion is I think orthogonal.  The #load mechanism is restricted to be what it is on purpose (i.e. it's not a #use, nor did we support a seaprate '#use).  The primary rationale for this is to make sure we can give sane typechecking/errors/intellisense/autocomplete/.... , and to give proper resolution of diamond dependencies (same file #load'd from multiple paths only gets referenced once).  It also allows type checking to be efficient, since we only check dependent files once (and recheck when they change).
Yes this means you can't factor out common fragments of scripts that create DLLs etc. I think the right thing is to propose on fslang.uservoice.com to add a #use directive, and provide a PR with its implementation.  The Paket scenario is compelling and it would be reasonable to support it.  The implementation would not be hard I think.
A key question is how the type checker in Visual Studio, Xamarin etc. would treat the #use fragments.  My assumption is that it would effectively be textual inclusion, but wouldn't actually execute the fragments.
Cheers
don

Thanks for the detailed answer! Now we managed to get to the root problem :)

The #load mechanism is restricted to be what it is on purpose

I assumed something like that, but couldn't figure out why...
Honestly I expected one of the reasons to be to force people to write libraries instead of scripts, but that doesn't seem to be the case?

The #load discussion is I think orthogonal.

That discussion is now over with #use...
One note after reading your answer: If we can't handle it in the tooling I'm actually against introducing #use. However from an architectural standpoint it still think would make sense to implement it in "visualfsharp" and expose it via FSharp.Compiler.Service (that would cover Use case 2, which is my primary interest actually). This has a lower impact, we could test the functionality and later expose #use if it still makes sense. Does that sound reasonable?

Honestly I expected one of the reasons to be to force people to write libraries instead of scripts, but that doesn't seem to be the case?

Yes, that's part of the thinking too. #load helps enable a transition from script to library, #use doesn't since diamond dependencies and textual inclusion allow for all sorts of coding patterns.

However from an architectural standpoint it still think would make sense to implement it in "visualfsharp" and expose it via FSharp.Compiler.Service (that would cover Use case 2, which is my primary interest actually).

We can just add it to FCS - It's reasonable for EvalInteraction to effectively be a multi-fragment #use.  If we decidde to light up #use in fsi.exe proper then we can backport to visuafsharp.

@dsyme Thanks. I'm fine with that. Will try to hack something together :) (Can take some days/weeks though)


[Request] LEGO MINDSTORMS EV3

##Package name:
LEGO MINDSTORMS EV3
##If not available in a package repository please provide a download URL for the package:
Currently missing

Package added


Creating circuit breakers in advance

Hi,
All of the demo code and documentation examples favour creating circuit breakers every time the protected method is called. That feels like it might not be efficient and earlier this year there seemed to be an admission that it is not the best approach.
Creating circuit breakers in advance seems to work fine but I've found that trying to configure them using Customizer beans has no effect. That appears to be because Customizer beans update the circuit breaker factory in auto-configuration code that runs after the circuit breaker factory has been used to create circuit breakers in the constructors of other components. The only way I've found to configure circuit breakers created in advance of any method calls is to provide a custom CircuitBreakerFactory bean that includes the desired customisations. Here's an example where the circuit breaker factory's defaults are configured along with some custom set-up for a particular circuit breaker named "myCircuitBreaker" that is created elsewhere (in the constructor of the bean that uses it):

    @Bean
    public CircuitBreakerFactory circuitBreakerFactory() {
        Resilience4JCircuitBreakerFactory factory = new Resilience4JCircuitBreakerFactory();

        factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
                .circuitBreakerConfig(
                        CircuitBreakerConfig.custom()
                                .failureRateThreshold(50)
                                .waitDurationInOpenState(ofMillis(10000))
                                .slidingWindow(20, 20, COUNT_BASED)
                                .build())
                .timeLimiterConfig(
                        TimeLimiterConfig.custom()
                                .timeoutDuration(ofMillis(5000))
                                .build())
                .build());

        factory.configure(configBuilder -> configBuilder
                        .circuitBreakerConfig(
                                CircuitBreakerConfig.custom()
                                        .failureRateThreshold(33)
                                        .waitDurationInOpenState(ofMillis(5000))
                                        .slidingWindow(10, 10, COUNT_BASED)
                                        .build())
                        .timeLimiterConfig(
                                TimeLimiterConfig.custom()
                                        .timeoutDuration(ofMillis(2000))
                                        .build())
                        .build(),
                "myCircuitBreaker");

        return factory;
    }

Are there any guidelines or recommendations about whether it is better to create circuit breakers inside every method call or in advance? If creation in advance is supported then is it a bug that Customizer beans appears to have no effect on circuit breakers unless they are created in every method call - or is it just an omission in the documentation and examples?

The simplest way I can think of doing this is basically delay creation until the first time it is used.  For example.
@RestController
public class DemoController {

	Logger LOG = LoggerFactory.getLogger(DemoController.class);

	private HttpBinService httpBin;
	private CircuitBreakerFactory circuitBreakerFactory;
	private CircuitBreaker cb;

	public DemoController(CircuitBreakerFactory circuitBreakerFactory, HttpBinService httpBinService) {
		this.circuitBreakerFactory = circuitBreakerFactory;
		this.httpBin = httpBinService;
	}

	@GetMapping("/get")
	public Map get() {
		return httpBin.get();
	}

	@GetMapping("/delay/{seconds}")
	public Map delay(@PathVariable int seconds) {
		if(cb == null) {
			cb = circuitBreakerFactory.create("delay");
		}
		return cb.run(httpBin.delaySuppplier(seconds), t -> {
			LOG.warn("delay call failed error", t);
			Map<String, String> fallback = new HashMap<>();
			fallback.put("hello", "world");
			return fallback;
		});
	}
}

Let me know what you think of that.  I will also see if I can come up with a more elegant solution.

I made a pull request to fix this (#56)

@ebussieres thanks!  We also need to make similar changes for the other implementations as well

@ryanjbaxter Just made a commit to the PR #56 to apply the fix to spring-retry

@ebussieres thanks if you want to take a stab at reworking the Hystrix implementation in Spring Cloud Netflix, that would be appreciated as well

Sure I'll take a look

If you do create a PR in spring-cloud-netflix be sure to link it here

@ryanjbaxter Just made the PR to spring-cloud-netflix (https://github.com/spring-cloud/spring-cloud-netflix/pull/3722)

Great thanks I will take a look at this once I get back from vacation on January 6th


Extension issue

Issue Type: Bug
Extension Name: LiveServer
Extension Version: 5.6.1
OS Version: Windows_NT x64 10.0.19042
VS Code version: 1.58.2

:warning: We have written the needed data into your clipboard. Please paste! :warning:

Issue Type: Bug
Extension Name: LiveServer
Extension Version: 5.6.1
OS Version: Windows_NT x64 10.0.19042
VS Code version: 1.58.2
⚠ We have written the needed data into your clipboard. Please paste! ⚠


Deleted All Modded Items and Reciepe

I'm trying to update my server I run for me and my friends and the only real mod I have is this one. I was able to update my server and it runs np, but everything from the mod is deleted. I can't craft them either. The only thing I'm capable of is /give the items but the items do not drop upon breaking. They do seem to have their functionality, but I am seeing barrier blocks appear in my systems can't grab them though.
Upon joining the server for the first time, the console spits out
Invalid statistic in .\world\stats\3d0d4fdf-570d-4abf-8b2d-04c53610859e.json: Don't know what toms_storage:ts.inventory_cable_connector is
And
Tried to load unrecognized recipe: toms_storage:trim removed now.
And it did think for every recipe and item deleting them from my world and don't have access to them without cheating

Yeah it also doesn't even let you pair the advanced wireless terminal to your system. I updated my pre-existing world but can't re-pair it. Now i have to go to my storage system to use it :( First World Problems, huh?

The 1.21 version is a major rewrite, so the block ids, and their functions changed. (There won't be a filtered inventory cable connector, the regular connector will have filtering, all of the painted blocks work completly differently on fabric, removing many duplicate blocks).
I released a patch for the recipe and terminal issue.
Duplicate of #380 and #382


Cutting from the command line

So this is following up on a Reddit post Reddit post.
Just as a summary:
I'm currently using a K40 chinese laser cutter (M2nano board) connected through USB to a Raspberry Pi 3 running Raspbian w/ GUI.
My goal is to get the k40 to cut using just the Terminal.
I have downloaded the pre-compiled MeerK40t pi executable, and also the 0.6.5 source folder.
When I open the executable, this takes a long time to display the MeerK40t GUI frontend which is probably not an issue, as it's a pretty big file, so that's expected.
When I issue the commands:
python3 MeerK40t.py -zav
or
sudo python3 MeerK40t.py -zav
or
python3 MeerK40t.py -zavS 10
or
sudo python3 MeerK40t.py -zavS 10
it moves the head on the XX and YY axis but only slightly, however it doesn't follow the path of the outline in the file, nor does it activate the laser.
I've attached the output of the commands above as well as the .svg file I'm trying to cut, basically it's a 20x20mm square outline.
logs.zip

If it finishes too soon it might be a bug we're currently working like at this point.
We're pumping out a 0.6.5 patch, so we figured we'd make sure to get your stuff correctly done.
We're pretty sure the problem is/was that the shutdown killed the controller before it sent the data it had. So it now, joins the thread and adds a quit command to the controller buffer. So it will flush the entire buffer before moving on to the next item.
At least that's the current hope.

We're testing if we have it fixed now. A tiny square that size might all get eaten by the premature shutdown. The very latest version on the patch branch might have it. We're testing it now, to make sure we have something that'll work for your stuff before publishing 0.6.5

Testing seemed to go well. Especially with the operation in the SVG you won't even need an -S command for that.

I wouldn't say that it's finishing too soon as it's not starting.
Like I said before, if I open the file on the GUI it cuts everything fine.
What happens is that on the CLI is that it moves the head to specific point in space (this point is relative to the size of the drawing but it's not in the path of any line)
Note: attached a svg file output from CorelDraw
3colour_test.zip

It's corrected in the main branch. And yeah, it was quitting too soon. Since your file was so fast, it finished nearly before it started. But, if you ran a longer one it would cut off a nice clip of commands at the end. It's why it twitched a little and then died. At the end it shutdown but the shutdown could beat the remaining buffer in the controller send. Now it only ends after the controller send buffer is finished.

Thought that the issue might be related to CorelDraw so I used Inkscape to create a similar file.
100x100mm page with two square outlines, one black and one red.
Issued the command:
sudo python3 MeerK40t.py -zavS 10 //test_inkscape.svg
behaves exactly like with the Corel file
Note: I noticed that the master branch was updated a few minutes ago, so that's the version I'm using.
test_inkscape.zip

Downloaded the latest version (apparently I somehow didn't download the latest version earlier) and it's working!!!
Thank you so much guys!
Awesome job!!!

This was corrected with 0.6.5.
Homing is being dealt with in 0.6.6


Different runtimes for pmap vs pjit

Hi,
I have a simple script below that compares runtimes for pmap vs pjit. I expected that the runtime for pjit with full data parallelism would be the same for pmap, as they are functionally the same (shard across data, replicate the model), but it's around 1.5x-2x slower in my experiments.
# script.py
import jax
from jax.experimental.pjit import pjit
from jax.experimental.maps import Mesh
from jax.experimental import PartitionSpec as P
import numpy as np

parser = argparse.ArgumentParser()
parser.add_argument('-m', '--mode', type=str, choices=['pmap', 'pjit'], default='pmap')
args = parser.parse_args()

# Init data
x = np.random.randn(32, 1024).astype(np.float32)
W = np.random.randn(1024, 8).astype(np.float32)

def step(x, W):
    return jax.lax.dot(x, W)

# Compute pmap or pjit functions
# Preload batch data and model parameters onto the devices as ShardedDeviceArrays
if args.mode == 'pmap':
    p_step = jax.pmap(step, axis_name='batch')
    x = np.reshape(x, (jax.local_device_count(), -1, x.shape[1]))

    # Gets correct device order that matches pmap
    devices = jax.lib.xla_bridge.get_backend().get_default_device_assignment(jax.device_count())
    x = jax.device_put_sharded(list(x), devices)
    W = jax.device_put_replicated(W, devices)
else:
    mesh = Mesh(np.asarray(jax.devices(), dtype=object).reshape(jax.local_device_count(), 1), ['dp', 'mp'])
    jax.experimental.maps.thread_resources.env = (
        jax.experimental.maps.ResourceEnv(physical_mesh=mesh, loops=())
    )
    p_step = pjit(step, in_axis_resources=(P('dp'), P('mp', None)), out_axis_resources=P('dp'))

    # Map batch and weights to devices
    p_init = pjit(lambda x, W: (x, W), in_axis_resources=(P('dp'), P('mp', None)), out_axis_resources=(P('dp'), P('mp', None)))
    x, W = p_init(x, W)

# Warmup for initial compilation
p_step(x, W).block_until_ready()

# Time
iterations = 1000
avg = timeit.timeit(lambda: p_step(x, W).block_until_ready(), number=iterations) / iterations
print('Estimated Time:', avg, 'per itr')

I see the following outputs:
wilson@t1v-n-588e2d9f-w-0:~$ python3 script.py -m pmap
Estimated Time: 0.0006339213080937043 per itr

wilson@t1v-n-588e2d9f-w-0:~$ python3 script.py -m pjit
Estimated Time: 0.0009550822210730985 per itr

Note that there is some noise in the output, but even when turning up the number of samples, there's a pretty substantial difference - even more so if I run larger models (i.e. an MLP, or transformer vs just a linear layer).
All of these commands are done on a v3-8 TPU instance, with jax==0.3.14, jaxlib==0.3.14, libtpu-nightly==0.1.dev20220627.
Any help or insight would be appreciated on this! (Or, if there's an issue with any of my code)

If I remove the model parallelism component in the pjit portion of the code, i.e. only ['dp']
mesh = Mesh(np.asarray(jax.devices(), dtype=object).reshape(jax.local_device_count(),), ['dp'])
    jax.experimental.maps.thread_resources.env = (
        jax.experimental.maps.ResourceEnv(physical_mesh=mesh, loops=())
    )
    p_step = pjit(step, in_axis_resources=(P('dp'), None), out_axis_resources=P('dp'))

    # Map batch and weights to devices
    p_init = pjit(lambda x, W: (x, W), in_axis_resources=(P('dp'), None), out_axis_resources=(P('dp'), None))
    x, W = p_init(x, W)

It's slightly faster, but still slower than pmap:
wilson@t1v-n-588e2d9f-w-0:~$ python3 test/script.py -m pjit
Estimated Time: 0.000860201039002277 per itr

Is pjit do some extra unnecessary computations that pmap isn't?

Would be interested in knowing the answer to this as well, as I am also observing it in my code.

This computation is actually quite fast, so I imagine the difference might be due to the pmap dispatch path being optimized (as it's implemented in C++) compared to pjit's dispatch (written entirely in Python).

I tried running similar code on some larger models, and get similar effects.  This is also done with only data parallelism (no model axis in mesh)
Code is run on a v3-8 TPU-VM with

jax==0.3.15
jaxlib==0.3.15
libtpu-nightly==0.1.dev20220722
flax==0.5.3

Transformer 1: 1.6M params, Transformer(hidden_dim=128, num_heads=4, num_layers=8) (1.4x slower)
pmap: Estimated Time: 0.00693524660900016 per itr
pjit: Estimated Time: 0.010101222762999896 per itr
Transformer 2:  25M params, Transformer(hidden_dim=512, num_heads=8, num_layers=8) (1.2x slower)
pmap: Estimated Time: 0.015546864855000194 per itr
pjit: Estimated Time: 0.01896321659 per itr
Transformer 3:  100M params, Transformer(hidden_dim=1024, num_heads=16, num_layers=8) (1.08x slower)
pmap: Estimated Time: 0.037007930983999814 per itr
pjit: Estimated Time: 0.04010667563000016 per itr
There seems to still be a different, with the different decreasing as I increase model size (though still statistically significant).
I tried a different model that similar to one I'm using (on videos, frame-wise encoding / decoder with a transformer over space-time). For some reason, under an identical setup, this model is significantly slower between pmap and pjit (7x)
pmap: Estimated Time: 0.042936902346999886 per itr
pjit: Estimated Time: 0.3086913418910008 per itr
Is there any reason why model architecture could cause this large difference in performance?
The code for both models is below (script.py corresponds to the Transformer, and script2.py corresponds to the other model):
# script.py
import timeit
import argparse
import jax
from jax.experimental.pjit import pjit
from jax.experimental.maps import Mesh
from jax.experimental import PartitionSpec as P
import numpy as np
import flax.linen as nn

class Transformer(nn.Module):
    hidden_dim: int
    num_heads: int
    num_layers: int

    @nn.compact
    def __call__(self, x):
        x = nn.Dense(self.hidden_dim)(x)
        x = nn.LayerNorm()(x)
        for _ in range(self.num_layers):
            x = TransformerBlock(self.hidden_dim, self.num_heads)(x)
        return x

class TransformerBlock(nn.Module):
    hidden_dim: int
    num_heads: int

    @nn.compact
    def __call__(self, x):
        h = nn.LayerNorm()(x)
        h = nn.SelfAttention(num_heads=self.num_heads)(x)
        x = x + h

        h = nn.LayerNorm()(x)
        h = nn.Sequential([
            nn.Dense(4 * self.hidden_dim),
            nn.gelu,
            nn.Dense(self.hidden_dim)
        ])(h)
        x = x + h
        return x

def print_model_size(params, name=''):
    model_params_size = jax.tree_util.tree_map(lambda x: x.size, params)
    total_params_size = sum(jax.tree_util.tree_flatten(model_params_size)[0])
    print('model parameter count:', total_params_size)

parser = argparse.ArgumentParser()
parser.add_argument('-m', '--mode', type=str, choices=['pmap', 'pjit'], default='pmap')
args = parser.parse_args()

# Init data
x = np.random.randn(32, 1024, 64).astype(np.float32)
model = Transformer(hidden_dim=128, num_heads=4, num_layers=8)
variables = model.init(rngs=jax.random.PRNGKey(0), x=x)
print_model_size(variables)

def step(x, variables):
    return model.apply(variables, x)

# Compute pmap or pjit functions
# Preload batch data and model parameters onto the devices as ShardedDeviceArrays
if args.mode == 'pmap':
    p_step = jax.pmap(step, axis_name='batch')
    x = np.reshape(x, (jax.local_device_count(), -1, *x.shape[1:]))

    # Gets correct device order that matches pmap
    devices = jax.lib.xla_bridge.get_backend().get_default_device_assignment(jax.device_count())
    x = jax.device_put_sharded(list(x), devices)
    variables = jax.device_put_replicated(variables, devices)
else:
    mesh = Mesh(np.asarray(jax.devices(), dtype=object).reshape(jax.local_device_count(),), ['dp'])
    jax.experimental.maps.thread_resources.env = (
        jax.experimental.maps.ResourceEnv(physical_mesh=mesh, loops=())
    )
    p_step = pjit(step, in_axis_resources=(P('dp'), None), out_axis_resources=P('dp'))

    # Map batch and weights to devices
    p_init = pjit(lambda x, variables: (x, variables), in_axis_resources=(P('dp'), None), out_axis_resources=(P('dp'), None))
    x, variables = p_init(x, variables)

# Warmup for initial compilation
p_step(x, variables).block_until_ready()

# Time
iterations = 1000
avg = timeit.timeit(lambda: p_step(x, variables).block_until_ready(), number=iterations) / iterations
print('Estimated Time:', avg, 'per itr')

# script2.py
from typing import Tuple, Any
import timeit
import argparse
import jax
from jax.experimental.pjit import pjit
from jax.experimental.maps import Mesh
from jax.experimental import PartitionSpec as P
import numpy as np
import flax.linen as nn

class Model(nn.Module):
    enc_args: Any
    tfm_args: Any
    dec_args: Any

    @nn.compact
    def __call__(self, x):
        x = jax.vmap(Encoder(**self.enc_args), 1, 1)(x)

        old_shape = x.shape[1:-1]
        x = x.reshape(x.shape[0], -1, x.shape[-1])
        x = Transformer(**self.tfm_args)(x)
        x = x.reshape(x.shape[0], *old_shape, x.shape[-1])

        x = jax.vmap(Decoder(**self.dec_args), 1, 1)(x)
        return x

def block(x, depth):
    skip = x
    if skip.shape[-1] != depth:
        skip = nn.Conv(depth, [1, 1], use_bias=False)(skip)
    x = nn.Sequential([
        nn.GroupNorm(),
        nn.elu,
        nn.Conv(depth, [3, 3]),
        nn.GroupNorm(),
        nn.elu,
        nn.Conv(depth, [3, 3])
    ])(x)
    return skip + 0.1 * x

class Encoder(nn.Module):
    depths: Tuple
    blocks: int

    @nn.compact
    def __call__(self, x):
        x = nn.Conv(self.depths[0], [3, 3])(x)
        for i in range(1, len(self.depths)):
            x = nn.avg_pool(x, (2, 2), strides=(2, 2))
            for _ in range(self.blocks):
                x = block(x, self.depths[i])
        return x

class Decoder(nn.Module):
    depths: Tuple
    blocks: int

    @nn.compact
    def __call__(self, x):
        for i in range(len(self.depths) - 1):
            for _ in range(self.blocks):
                x = block(x, self.depths[i])
            x = jax.image.resize(x, (x.shape[0], 2 * x.shape[1], 2 * x.shape[2], x.shape[3]),
                                 method='nearest')
        x = nn.Conv(3, [3, 3])(x)
        return x

class Transformer(nn.Module):
    hidden_dim: int
    num_heads: int
    num_layers: int

    @nn.compact
    def __call__(self, x):
        x = nn.Dense(self.hidden_dim)(x)
        x = nn.LayerNorm()(x)
        for _ in range(self.num_layers):
            x = TransformerBlock(self.hidden_dim, self.num_heads)(x)
        return x

class TransformerBlock(nn.Module):
    hidden_dim: int
    num_heads: int

    @nn.compact
    def __call__(self, x):
        h = nn.LayerNorm()(x)
        h = nn.SelfAttention(num_heads=self.num_heads)(x)
        x = x + h

        h = nn.LayerNorm()(x)
        h = nn.Sequential([
            nn.Dense(4 * self.hidden_dim),
            nn.gelu,
            nn.Dense(self.hidden_dim)
        ])(h)
        x = x + h
        return x

def print_model_size(params, name=''):
    model_params_size = jax.tree_util.tree_map(lambda x: x.size, params)
    total_params_size = sum(jax.tree_util.tree_flatten(model_params_size)[0])
    print('model parameter count:', total_params_size)

parser = argparse.ArgumentParser()
parser.add_argument('-m', '--mode', type=str, choices=['pmap', 'pjit'], default='pmap')
args = parser.parse_args()

# Init data
x = np.random.randn(32, 100, 16, 16, 3).astype(np.float32)
model = Model(enc_args=dict(depths=[64, 128, 256], blocks=2),
              tfm_args=dict(hidden_dim=512, num_heads=8, num_layers=8),
              dec_args=dict(depths=[256, 128, 64], blocks=2))
variables = model.init(rngs=jax.random.PRNGKey(0), x=x)
print_model_size(variables)

def step(x, variables):
    return model.apply(variables, x)

# Compute pmap or pjit functions
# Preload batch data and model parameters onto the devices as ShardedDeviceArrays
if args.mode == 'pmap':
    p_step = jax.pmap(step, axis_name='batch')
    x = np.reshape(x, (jax.local_device_count(), -1, *x.shape[1:]))

    # Gets correct device order that matches pmap
    devices = jax.lib.xla_bridge.get_backend().get_default_device_assignment(jax.device_count())
    x = jax.device_put_sharded(list(x), devices)
    variables = jax.device_put_replicated(variables, devices)
else:
    mesh = Mesh(np.asarray(jax.devices(), dtype=object).reshape(jax.local_device_count(),), ['dp'])
    jax.experimental.maps.thread_resources.env = (
        jax.experimental.maps.ResourceEnv(physical_mesh=mesh, loops=())
    )
    p_step = pjit(step, in_axis_resources=(P('dp'), None), out_axis_resources=P('dp'))

    # Map batch and weights to devices
    p_init = pjit(lambda x, variables: (x, variables), in_axis_resources=(P('dp'), None), out_axis_resources=(P('dp'), None))
    x, variables = p_init(x, variables)

# Warmup for initial compilation
p_step(x, variables).block_until_ready()

# Time
iterations = 1000
avg = timeit.timeit(lambda: p_step(x, variables).block_until_ready(), number=iterations) / iterations
print('Estimated Time:', avg, 'per itr')

I looked into it and I think it's basically that the SPMD partitioner ends up failing to propagate the data-parallel sharding through the full HLO. The HLO module does contain some while loops that might be confusing for it. SPMD partitioner is heuristic based, so it might not get everything right. This is why sometimes manual partitioning with pmap or xmap can lead to better results, since they can provide you with explicit guarantees.

Thanks for looking into it! Is there a good way to prevent this issue from happening code-wise? i.e. differently coding the architecture or enforcing certain constraints to help the partitioner partition how we'd like it to?

You could try sprinkling some with_sharding_constraints over your code. It might help the SPMD partitioner do the right thing. But if you know what sharding you want to get, and you actually prefer explicit control, then xmap sounds like a better way to go.
In the meantime, I've brought this up with the XLA team. I've opened an internal bug with them, and since there's nothing we can do on the JAX side to fix it, I'm going to close it. Feel free to reopen if you think there's still something more we could do. Thanks!

And also thanks so much for a detailed repro!


How to pull data updates?

This is brilliant but how could I introduce the ability to download data for just one period? For example, in case I want to download only an yearly update or split my request in smaller pieces?

Hi, what do you mean by yearly update or by splitting the request in smaller pieces?
Can you describe a specific use case?
Also note that wrds provides an official  API to connect through matlab and build SQL queries, but is obviously much slower than this package.
Sent from my iPhone
On 11 Oct 2016, at 23:18, "afonsovj" <notifications<EMAIL_ADDRESS>wrote:
This is brilliant but how could I introduce the ability to download data for just one period? For example, in case I want to download only an yearly update or split my request in smaller pieces?

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/okomarov/wrds/issues/9, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADju5dDfLVT48YU7yhjBcWgbPwJkxKo4ks5qzAsXgaJpZM4KUKpk.

Hi Oleg, I mean that it would be great to have the ability to request a download of a subset of a database.dataset. For example, OptionMetrics distributes large files. It would be great if I could get data for the latest year of the Implied Vol Dataset (lets say June 2015 to 2016) and append it to my existing data, of just download it by pieces and save monthly files? Maybe this is bad data management on my part.
I tried Matlab's API but to be honest yours is more robust. Can you explain me why your's is faster?
Also, I had to change something in your code for it to work on my mac. I think I had to remove that data\ bit. Error below.
Thanks!
Error using ssh2_main (line 169)
Error Transferring File
SEE JAVA ERROR BELOW
Java exception occurred:
java.io.IOException: Error during SCP transfer.
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:703)
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:596)
Caused by: java.io.FileNotFoundException: /Users/[MyUserName and folder path]/data/dataset.zip (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:221)
at java.io.FileOutputStream.(FileOutputStream.java:171)
at ch.ethz.ssh2.SCPClient.receiveFiles(SCPClient.java:344)
at ch.ethz.ssh2.SCPClient.get(SCPClient.java:699)
... 1 more
Error in ssh2 (line 84)
ssh2_struct = ssh2_main(ssh2_struct);
Error in scp (line 19)
ssh2_struct = ssh2(ssh2_struct);
Error in scp_get (line 50)
ssh2_struct = scp(ssh2_struct);
Error in wrds/getFile (line 114)
obj.SSH2conn = scp_get(obj.SSH2conn, [rfname, rext], lpath, rpath);
Error in sas2csv (line 43)
wrds.getFile('~/tmp/dataset.zip', outfile);
Error in wrds/getDataset (line 14)
sas2csv(wrds, libdataname, outfile);

I thought about implementing some sort of file splitter to download chunk of datasets, however, that would require some considerable work and my subscription to wrds expires in december.
The suite is faster because it converts their raw sas datasets into compressed csvs and sends over that stream. No java encapsulation, no other interfaces or additional overhead.
I will look into the data\ error though

@afonsovj I should have fixed the data\ problem.
For the partial dataset download, do you feel the problem is that Matlab hungs for several hours until the download is complete, right? The issue with this, is potential crashes and unexpected interruptions to this operation.
The solutions are:

do nothing and just keep one Matlab session open, potentially overnight, and work in another Matlab instance
implement an asynchronous download request which launches a matlab session on the background

Both those cases however would benefit from a file splitting procedure that could recover from the last downloaded part, if things went wrong.
If you are interested in working on this task, you can issue a pull request to this submission. I can help with guiding you through the code and suggesting which direction to take. The task would require getting acquainted with a bit of Unix shell commands, a bit of SAS (I sought help on their forums) and the Matlab codebase.


Fetching entire emoji library on initialisation

I think this was introduced somewhere between v3.0.4 and v3.1.4 test case here: https://jsfiddle.net/mjwxefqd/
If you check the network requests there's 1600+ request for pngs which seems to be delaying page loading process somewhat, Does the option for a sprite sheet still exist or is there a way to delay this load until it's needed?

Hi @MerlinBB
I think this thread will answer your query: https://github.com/Ranks/emojione/issues/243
I was wondering the same, hopefully a workable fix can be made for this issue.

Unfortunately no, in Emojione v2.1.2 was removed support for scalable png sprites,
so you currently can't use sprite option

Thanks for the info guys, will keep an eye on that issue in the emojione repo.

fixed


Routes with extension

A very common endpoint is to do some sort of a match on an extension:
/foo/bar.jpg

Usually, you want to get bar as a variable value. I propose a new route segment type: ext that would allow make it super simple to setup these types of routes.
There are three general scenarios:

Match any extension
Match a single extension
Match one of a specific set of extensions

I think something like the following should work:
/foo/<file:ext>
/foo/<file:ext:jpg>
/foo/<file:ext:jpg|png|gif|svg>

This would result in two passed variables: whatever is defined, and the extension as ext.
For example:
@app.get("/invoice/<invoice_id:ext:pdf|txt>")
async def invoice_handler(request, invoice_id, ext):
    ...

How would one specify the type of the file stem?
e.g. To ensure that invoice_id is a UUID and the extension is pdf or text.

How would one specify the type of the file stem?
e.g. To ensure that invoice_id is a UUID and the extension is pdf or text.

:thinking: Hmm.... that is not something I considered. This implementation would accept anything for the filename.

Perhaps something like this?
@app.get("/invoice/<invoice_id:uuid.ext:pdf|txt>")
async def invoice_handler(request, invoice_id, ext):
    ...

Using . as the separator could support all of these patterns:
/foo/<file.ext> # type is assumed to be 'file:str'
/foo/<file.ext:jpg>
/foo/<file.ext:jpg|png|gif|svg>
/foo/<file:uuid.ext>
/foo/<file:path.ext:jpg>
/foo/<file:int.ext:jpg|png|gif|svg>

Alternatively, if the file in <file:ext:png> also matched paths (/ character) then that would be sufficient for my use case.

I have been looking at this one some more so we can finally wrap it up. The real challenge for me has been to determine what is the appropriate syntax here. While I can appreciate what @jacebrowning offered with file.ext, it really breaks a lot of other patterns and the entire concept that ext is itself the parameter "type".
Therefore, I offer this as the syntax:
/foo/<file:ext> # type is assumed to be 'file:str'
/foo/<file:ext=jpg>
/foo/<file:ext=jpg|png|gif|svg>
/foo/<file=uuid:ext>
/foo/<file=path:ext=jpg>
/foo/<file=int:ext=jpg|png|gif|svg>


containerlab: 0.43.0 -> 0.44.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/srl-labs/containerlab/releases.
meta.description for containerlab is: Container-based networking lab
meta.homepage for containerlab is: https://containerlab.dev/
meta.changelog for containerlab is: https://github.com/srl-labs/containerlab/releases/tag/v0.44.0
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.44.0 with grep in /nix/store/2p80scrapy3pfs613wds10qzl0hm2gwv-containerlab-0.44.0
found 0.44.0 in filename of file in /nix/store/2p80scrapy3pfs613wds10qzl0hm2gwv-containerlab-0.44.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
containerlab

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/2p80scrapy3pfs613wds10qzl0hm2gwv-containerlab-0.44.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A containerlab https://github.com/r-ryantm/nixpkgs/archive/852db6a45b11f451fd40c1d4bb188a455dff9b4b.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/2p80scrapy3pfs613wds10qzl0hm2gwv-containerlab-0.44.0
ls -la /nix/store/2p80scrapy3pfs613wds10qzl0hm2gwv-containerlab-0.44.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

containerlab

Maintainer pings
cc @Janik-Haag for testing.

Result of nixpkgs-review pr 250410 run on x86_64-linux 1

1 package built:

containerlab


Implement infix: and family

Eight operators in total: ff ff^ ^ff ^ff^ fff fff^ ^fff ^fff^.
I dunno why we haven't done these long ago. They are the perfect poster child for macros. As far as I can see, we can do them already. Also, I'd say they're a perfect example to have in the new examples/ directory. (See #194.)
my values = ["A", "B", "A", "B", "A"];
for values -> v {
    if v == "B" ff v == "B" {
        say(v);
    }
    else {
        say("x");
    }
}
# Result for ff: xBxBx
# Result for fff: xBABx

The reason they're perfect is that they are actually operators with internal state. If we implement this right, the operator state should be per sub clone. See these tests from the spectest suite:
# See thread "till (the flipflop operator, formerly ..)" on p6l started by Ingo
# Blechschmidt, especially Larry's reply:
# http://www.nntp.perl.org/group/perl.perl6.language/24098
# make sure calls to external sub uses the same ff each time
{
    sub check_ff($i) {
        $_ = $i;
        return (/B/ ff /D/) ?? $i !! 'x';
    }

    my $ret = "";
    $ret ~= check_ff('A');
    $ret ~= check_ff('B');
    $ret ~= check_ff('C');
    $ret ~= check_ff('D');
    $ret ~= check_ff('E');
    is $ret, 'xBCDx', 'calls from different locations use the same ff';
}

# From the same thread, making sure that clones get different states
{
    my $ret = "";
    for 0,1 {
        sub check_ff($_) { (/B/ ff /D/) ?? $_ !! 'x' }
        $ret ~= check_ff('A');
        $ret ~= check_ff('B');
        $ret ~= check_ff('C');
    }
    is $ret, 'xBCxBC', 'different clones of the sub get different ff'
}

I think this is a correct implementation of ff:
macro infix:<ff>(lhs, rhs) {
    my active = False;
    return quasi {
        if {{{lhs}}} {
            active = True;
        }
        my result = active;
        if {{{rhs}}} {
            active = False;
        }
        result;
    };
}

Right now it fails with this error on master: No such method 'eval' for invocant of type 'Q::Block'. I think this is related to #212 and us never really supporting several statements in a quasi.

Removed the "low-hanging-fruit" label, since this issue is blocked on another.

Branch.

The reason they're perfect is that they are actually operators with internal state. If we implement this right, the operator state should be per sub clone.

This won't happen on its own. Why? Because the macro will be called exactly once, at parse time, and so there will be only one "instance" of the variable active. There needs to be one per sub clone.
(See the test at the end of OP.)
I'm not clever enough today to nail down how this ought to work. But it needs to be something that re-enters the scope with the active variable whenever the sub surrounding the ff operator is cloned.

I'm not clever enough today to nail down how this ought to work. But it needs to be something that re-enters the scope with the active variable whenever the sub surrounding the ff operator is cloned.

Thinking about this a bit more, I'm struck by the fact that the semantics we want is as if active were declared in the scope calling ff. (In the OP example, that would be the sub check_ff.)
This is exactly the mechanism proposed by S06 as my $COMPILING::new_variable;. Leaving all concerns about un-hygiene aside for the moment, I believe that would be a neater solution than forcing scopes to re-enter manually.

Leaving all concerns about un-hygiene aside for the moment

And for when we feel up to solving the concerns about un-hygiene satisfactorily, there's a well-tested solution out there: EcmaScript 6 Symbols:
let s1 = Symbol("active");
let s2 = Symbol("active");
s1 === s2;        # false
let scope = { [s1]: false, [s2]: false };
scope[s1] = true;
scope[s2];        # false

This has everything we need:

Keys/variables declared using Symbol instead of strings aren't visible to the normal user and won't collide with normal userland variables.
Two symbols with the same name won't collide. (Meaning that two macros that accidentally grab the same name won't interact.)

I don't know the exact relation/history between (EcmaScript) symbols and (Lisp) gensyms, but I feel pretty comfortable I understand symbols. I think we want something like that when we install state-managing variables in scopes not owned by the macro itself.

Just slapped a "currently blocked" label on this issue. It's one of our most desirable issues to have in place (since it's at the top of the #194 list) and yet we can't move forward with it until we solve #212.

I was toying around with having a class-based API for stateful macros. The infix:<ff> macro would come out something like this:
return class {
    property lhs;
    property rhs;

    private property active = False;

    constructor(lhs, rhs) {
        self.lhs = lhs;
        self.rhs = rhs;
    }

    eval(runtime) {
        if lhs.eval(runtime) {
            active = True;
        }
        my result = active;
        if rhs.eval(runtime) {
            active = False;
        }
        result;
    }
}

I don't know if thinking about macros as classes is fruitful in any way. To be honest I expected the class to be a better fit than the macro+quasi, but the latter is shorter and no less clear. Maybe the only interesting thing about it is that it brings us quite close to how we currently implement 007 operators with the macro nature in Perl 6. (But even that might change with #185.)

<masak> triumphant progress report: I have macro infix:<ff> working in the `ff-macro` branch
<masak> need to do some (hopefully simple) cleaning-up, and then I can merge it to master
<masak> this has been a long time coming ;)

I'll have to take my November me's word for it that the ff-branch used to work... it doesn't now.
$ bin/007 examples/ff.007
Variable 'v' is not declared
[...]

I haven't looked in detail, but I'm fairly certain why this happens. The two occurrences of v on this line:
if v == "B" ff v == "B" {

are going to be "dragged" into the infix:<ff> macro and processed there. The macro will spit out the generated injectile code, cocooned in a Q::Expr::BlockAdapter so that it can get the right environment. So far so hygienic.
But the two macro operands v == "B" and v == "B" don't get a similar treatment. (They should.) So currently lookup happens from the injectile's environment, which indeed does not have a v. (Nor should it.)
In other words, we should look into wrapping the unquoted things in a Q::Expr::BlockAdapter in such a way that they retain their mainline environment.
Go team hygienic macro!

Just dropping in here to point out a connection — previously not pointed out, I believe — between the statefulness of the ff family of macros, and the statefulness of gather from #241.


changed "sex" to playergender

client main 278-282
if playerGender == 0 then
playerGender = 'male'
else
playerGender = 'female'
end
change all ref from "sex" to playerGender
fixes the issue of calling wrong sex for police alert

uhhh as far as I can tell IsPedMale() is the only way to tell the sex or gender of a ped, if it's being inconsistent I don't know why, I've never noticed any issues with this but I also haven't really looked that close playerGender isn't a native and if you try to use that it won't work


ADBDEV-6344: Fix gpperfmon recording plpgsql queries with SPI cursors

Fix gpperfmon recording plpgsql queries with SPI cursors
Before this patch gpperfmon recorded plpgsql queries that used cursors (for
example FOR loops) even with log_min_messages < DEBUG 5. Logging for all the
other queries in plpgsql has been already disabled before by
cc2446b9333c52932c7a4ceeb522966d9cce31c3, this patch fixes the same issue for
SPI cursors.

Allure report https://allure.adsw.io/launch/81108

Allure report https://allure.adsw.io/launch/81161

Allure report https://allure.adsw.io/launch/81174

Failed job Behave tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/2004009

Allure report https://allure.adsw.io/launch/81285

Allure report https://allure.adsw.io/launch/81555

What is the original purpose of this patch and the previous patch? After all, SPI can be used without PL-languages, of which there are also several, and not just one plpgsql.


Code action support in LSIF

As mentioned in #825, I see a lot of potential in LSIF for tracking code smells in codebases and am working on an ESLint LSIF exporter. I'd also like to include in the LSIF dump how certain problems can be fixed automatically. LSP uses WorkspaceEdit for these, which can be serialized, and for complicated cases a Command reference can be included as a hint that the language server needs to be fired up to fix the problem. The code action can link to diagnostic vertexes.
Anything that speaks against adding this?
Spec link: https://microsoft.github.io/language-server-protocol/specification#textDocument_codeAction

I can't think of any argument right now why we would not allow to store these.

I created an experimental ESLint LSIF exporter which can emit code actions from ESLint autofixes: https://github.com/sourcegraph/eslint-formatter-lsif


Fix issue with cancelled touch pointers not resetting state correctly

Description

When a touch pointer gets cancelled, we expect that there still would be a pointer exited event, however that does not occur. Because of that, when we get pointer cancelled for touch pointers, we should treat this also as a pointer-exited event.
Motivation and Context

Fixes #6863
How Has This Been Tested?

Tested manually.
Screenshots (if appropriate):

/azp run


Remove language about preventing deletes.

The API no longer prevents deletes for groups sent to in the last 60 days. It seems the functionality was removed in this ticket: https://jira.sendgrid.net/browse/MAKO-1469

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hello @alexabreu,Thanks again for the PR! We want to show our appreciation by sending                                              you some swag. Could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX

Thanks Elmer! I’m glad I was able to help.
On Thu, Jul 19, 2018 at 9:21 AM, Elmer Thomas<EMAIL_ADDRESS>wrote:

Hello @alexabreu https://github.com/alexabreu,
Thanks again for the PR!
We want to show our appreciation by sending you some swag. Could you
please fill out this form https://dx.sendgrid.com/swag so we can send
it to you? Thanks!
Team SendGrid DX
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sendgrid/docs/pull/3730#issuecomment-406315087, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFkBPYQclhVz-cf-rodiJed-wIwyqb9ks5uIKQLgaJpZM4VVviL
.

--
Alex Abreu
Front End Software Engineer | Denver


SA1625 should accept localized "The parameter is not used."

When implementing the localization feature in #2198 I forgot to also localize "The parameter is not used." which is the only parameter documentation that's allowed to appear multiple times in the same method when SA1625 is enabled.

I'll probably implement it tomorrow.


Add additional options to disable any non-wanted plugins

We had this in the past, but we decided to have these features enabled by default. CC @hansgschossmann @cristipogacean

Close for now as we agreed before that we want these features to be always on


Adding members

Synax, CloudQuery, Prisma
Pre-submission checklist:
Please check each of these after submitting your pull request:

[x] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[x] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[x] Have you picked the single best (existing) category for your project?
[x] Does it follow the other guidelines from the new entries section?
[x] Have you added your SVG to hosted_logos and referenced it there?
[x] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[x] Does your project/product name match the text on the logo?
[x] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~15 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Build failed because of:
no headquarter addresses for cloudquery at https://www.crunchbase.com/organization/cloudquery
No cached entry, and Synax (member) has issues with logo: synax-gmbh.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/synax-gmbh.svg'
No cached entry, and Prisma (member) has issues with logo: prisma.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/prisma.svg'
No cached entry, and CloudQuery (member) has issues with logo: cloudquery.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/cloudquery.svg'
No cached entry, and Valve Software (member) has issues with twitter: https://twitter.com/valveoficial, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
No cached entry, and Sosivio (member) has issues with twitter: https://twitter.com/SosivioLtd, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
Empty twitter for Rancher Federal (member): https://twitter.com/rancherfederal
No cached entry, and Kingsoft (member) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Kingsoft Cloud (KCSP) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Kingsoft Cloud has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
Empty twitter for Standard Library: https://twitter.com/AutocodeHQ
Empty twitter for StackStorm: https://twitter.com/Stack_Storm
CloudQuery (member) either has no crunchbase entry or it is invalid

Build failed because of:
no headquarter addresses for cloudquery at https://www.crunchbase.com/organization/cloudquery
No cached entry, and Synax (member) has issues with logo: synax-gmbh.svg, SVG file embeds a png. Please use a pure svg file
No cached entry, and Valve Software (member) has issues with twitter: https://twitter.com/valveoficial, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
Empty twitter for Rancher Federal (member): https://twitter.com/rancherfederal
No cached entry, and Kingsoft (member) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Sosivio (member) has issues with twitter: https://twitter.com/SosivioLtd, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
No cached entry, and Kingsoft Cloud (KCSP) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Kingsoft Cloud has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
Empty twitter for StackStorm: https://twitter.com/Stack_Storm
Empty twitter for Standard Library: https://twitter.com/AutocodeHQ
CloudQuery (member) either has no crunchbase entry or it is invalid

Build failed because of:
no headquarter addresses for cloudquery at https://www.crunchbase.com/organization/cloudquery
Empty twitter for Rancher Federal (member): https://twitter.com/rancherfederal
No cached entry, and Valve Software (member) has issues with twitter: https://twitter.com/valveoficial, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
No cached entry, and Kingsoft (member) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Kingsoft Cloud (KCSP) has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
No cached entry, and Sosivio (member) has issues with twitter: https://twitter.com/SosivioLtd, 404 - {"errors":[{"code":34,"message":"Sorry, that page does not exist."}]}
No cached entry, and Kingsoft Cloud has issues with twitter: https://twitter.com/KingsoftC, 401 - {"request":"/1.1/statuses/user_timeline","error":"Not authorized."}
Empty twitter for Standard Library: https://twitter.com/AutocodeHQ
Empty twitter for StackStorm: https://twitter.com/Stack_Storm
CloudQuery (member) either has no crunchbase entry or it is invalid


Update to curve25519-dalek-ng (1.x)

Updates the curve25519-dalek dependency in the 1.x series to the -ng version at zkcrypto https://github.com/zkcrypto/curve25519-dalek-ng

Thanks for letting us know about the changes to these crates.
It looks like the -ng crates differ from curve25519-dalek by upgrading rand_core to 0.6 and exposing it in the public API. We currently do not need to support rand_core 0.6.
If -dalek does not upgrade rand as we need it, or -ng introduces other features we need, we will revisit switching to -ng. But for now we'll stick with curve25519-dalek.


Create widgets with real data

For each of the charts in here: NCA_Figures_ClimateProjections.docx please attempt to recreate the map or graph as a widget.
This should help us identify:

Is current hosting sufficient for visualizations?
What level of data processing should we expect?
What sort of metadata should we surface on the front end to maximize usability?

Hey @apfister,
Have you had a chance to look at this doc and see if you can extract the data? Let me know if you want me to have someone do additional data prep or extract specific summary info.
Thanks!
F

can we close/combine this with #10 ?

Yep. Thanks


Widget rendering in Jupyter but not in Bokeh App

I wrote a small application that renders beautifully in Juypter and shows the dropdown menu with the additional but unused dimension "date".
But when I deploy it to a bokeh app, the widget is not shown.
# Create the main plot
def create_map_figure(data):
    import geoviews as gv
    import holoviews as hv
    import geoviews.tile_sources as gts
    from holoviews import dim, opts
    from colorcet import rainbow
    gv.extension("bokeh")

    plot_width = 800
    plot_height = 400
    tile_opts = dict(height=plot_height, xaxis=None, yaxis=None, show_grid=False)
    map_tiles = gts.CartoLight.opts(style=dict(alpha=1), plot=tile_opts)

    points_ds = gv.Dataset(data[["lon", "lat", "val", "date"]])

    points = (points_ds.to(gv.Points, kdims=["lon", "lat"], vdims=["val"])
              .options(active_tools=['wheel_zoom'], tools=["hover"])
              .opts(colorbar=True, cmap=rainbow)).opts(opts.Points(color=dim('val'), size=5))

    return map_tiles * points.redim.range(change=(0, 10))

def modify_doc(doc):
    import holoviews as hv
    from bokeh.layouts import column, row
    from bokeh.models import Div

    conn = get_db_connection()
    df = get_data(conn)
    plot_map = create_map_figure(df)
    renderer = hv.renderer('bokeh').instance(mode='server')
    plot_map_rendered = renderer.app(plot_map, doc)

    doc.add_root(plot_map_rendered.state)

bokeh_app = Application(FunctionHandler(modify_doc))

server = Server(
        {'/': bokeh_app},
        io_loop=io_loop,
        allow_websocket_origin=ALLOW_WEBSOCKET_ORIGIN,
        debug=True,
        **{'port': PORT, 'address': HOST}
        )
server.start()

if __name__ == '__main__':
    io_loop.add_callback(server.show, "/")
    io_loop.start()

I'm sorry, after reading http://holoviews.org/user_guide/Plots_and_Renderers.html I realised I have to extract the widget myself...

At this point I'd strongly recommend using Panel to deploy the HoloViews app. It also has a more robust implementation of the HoloViews widgets.

No other chance to get this working as this is a simple app? Is it easy to implement panel in this "bokeh server started using the library in tornado" setup?

Yes, pn.panel(hv_obj)._get_server(...). I think I'll make that method public.

Do you have a minimal example in that bokeh server setup?

Could you please provide a short code snipped how to deploy the above app with your suggested function? I just can't make it work.

How can I embed a panel app using params API in that surrounding above?

Here you go:
import geoviews as gv
import holoviews as hv
import numpy as np
import geoviews.tile_sources as gts
from holoviews import dim, opts
from colorcet import rainbow
gv.extension("bokeh")

plot_width = 800
plot_height = 400
tile_opts = dict(height=plot_height, xaxis=None, yaxis=None, show_grid=False)
map_tiles = gts.CartoLight.opts(style=dict(alpha=1), plot=tile_opts)

point_ds = gv.Dataset(
    (np.random.rand(100), np.random.rand(100), np.random.randint(0, 3, 100), np.random.randn(100)),
    ['lon', 'lat', 'group'], 'val')

points = point_ds.to(gv.Points, ['lon', 'lat']).opts(
    active_tools=['wheel_zoom'], tools=["hover"],
    colorbar=True, cmap=rainbow, color=dim('val'), size=5)

plot = map_tiles * points.redim.range(change=(0, 10))

server = pn.panel(plot)._get_server(
    show=True, port=PORT, websocket_origin=ALLOW_WEBSOCKET_ORIGIN,
    address=HOST, debug=True)

server.start()

How can I embed a panel app using params API in that surrounding above?

Not sure I follow, could you expand on this?

Many thanks! I definately have to check this. In the meantime I found that I can serve a panel (took the MPGExplorer() as it is) row using:
def modify_doc(doc):
    row = pn.Row(explorer.param, explorer.plot)
    doc.add_root(plot_map_rendered)

complementary to the rest of my app above.

I did this to make your example work, but using python app.py it loads endlessly in the browser and shows nothing.
import os
from tornado.ioloop import IOLoop

# if running locally, listen on port 5000
PORT = int(os.getenv('PORT', '5000'))
HOST = "<IP_ADDRESS>"

print("HOST", HOST)
print("PORT", PORT)

# this is set in the manifest
try:
    ALLOW_WEBSOCKET_ORIGIN = os.getenv("ALLOW_WEBSOCKET_ORIGIN").split(',')
except:
    ALLOW_WEBSOCKET_ORIGIN = ['localhost:{0}'.format(PORT)]

print("ALLOW_WEBSOCKET_ORIGIN:", ALLOW_WEBSOCKET_ORIGIN)

io_loop = IOLoop.current()

import geoviews as gv
import numpy as np
import geoviews.tile_sources as gts
from holoviews import dim, opts
from colorcet import rainbow
import panel as pn
gv.extension("bokeh")

plot_width = 800
plot_height = 400
tile_opts = dict(height=plot_height, xaxis=None, yaxis=None, show_grid=False)
map_tiles = gts.CartoLight.opts(style=dict(alpha=1), plot=tile_opts)

point_ds = gv.Dataset(
    (np.random.rand(100), np.random.rand(100), np.random.randint(0, 3, 100), np.random.randn(100)),
    ['lon', 'lat', 'group'], 'val')

points = point_ds.to(gv.Points, ['lon', 'lat']).opts(
    active_tools=['wheel_zoom'], tools=["hover"],
    colorbar=True, cmap=rainbow, color=dim('val'), size=5)

plot = map_tiles * points.redim.range(change=(0, 10))

server = pn.panel(plot)._get_server(
    show=True, port=PORT, websocket_origin=ALLOW_WEBSOCKET_ORIGIN,
    address=HOST, debug=True)

server.start()

if __name__ == '__main__':
    io_loop.add_callback(server.show, "/")
    io_loop.start()

Ah, yes I tested in a notebook where the IOLoop is already running. I'd probably just change it to:
if __name__ == '__main__':
    pn.panel(plot)._get_server(
        show=True, port=PORT, websocket_origin=ALLOW_WEBSOCKET_ORIGIN,
        address=HOST, debug=True, start=True)

Changing the main right here?
Name error: name "plot" is not defined

You made someone happy, works like a charm.
import os
from tornado.ioloop import IOLoop

# if running locally, listen on port 5000
PORT = int(os.getenv('PORT', '5000'))
HOST = "<IP_ADDRESS>"

print("HOST", HOST)
print("PORT", PORT)

# this is set in the manifest
try:
    ALLOW_WEBSOCKET_ORIGIN = os.getenv("ALLOW_WEBSOCKET_ORIGIN").split(',')
except:
    ALLOW_WEBSOCKET_ORIGIN = ['localhost:{0}'.format(PORT)]

print("ALLOW_WEBSOCKET_ORIGIN:", ALLOW_WEBSOCKET_ORIGIN)

io_loop = IOLoop.current()

import geoviews as gv
import numpy as np
import geoviews.tile_sources as gts
from holoviews import dim, opts
from colorcet import rainbow
import panel as pn
gv.extension("bokeh")

plot_width = 800
plot_height = 400
tile_opts = dict(height=plot_height, xaxis=None, yaxis=None, show_grid=False)
map_tiles = gts.CartoLight.opts(style=dict(alpha=1), plot=tile_opts)

point_ds = gv.Dataset(
    (np.random.rand(100), np.random.rand(100), np.random.randint(0, 3, 100), np.random.randn(100)),
    ['lon', 'lat', 'group'], 'val')

points = point_ds.to(gv.Points, ['lon', 'lat']).opts(
    active_tools=['wheel_zoom'], tools=["hover"],
    colorbar=True, cmap=rainbow, color=dim('val'), size=5)

plot = map_tiles * points.redim.range(change=(0, 10))

if __name__ == '__main__':
    pn.panel(plot)._get_server(
        show=True, port=PORT, websocket_origin=ALLOW_WEBSOCKET_ORIGIN,
        address=HOST, debug=True, start=True)

The way you showed me to start the application starts only one server. So if another user connects to the server, they see the same applications and the user 2 sees the changes user 1 makes in the widgets.

Wasn't that true for your previous approach as well? If that's not the case we should open a panel issue.
The way servers are usually launched is by using:
pn.panel(plot).servable()

and then launching the server with panel serve script.py.

Oh, I realize what the issue is. Let me think about it.

Do you have an idea here or did you find a solution?


Plagiarise Cucumber-JS's formatter summary

Summary
Cucumber-JS recently added a very nice summary output that lists all the issues found during a test run. Most other testing tools (mocha, rspec) do something similar.
I'd like to converge on what Cucumber-JS is doing, and print this summary at the bottom of all of our console formatters.
Expected Behavior
@charlierudolph who wrote this feature into Cucumber-JS tells me that it's not all that well documented in their tests, but there are clues:

https://github.com/cucumber/cucumber-js/blob/61842114a3e1e6af7310c4b90adb0964b1732ac5/spec/cucumber/listener/summary_formatter_spec.js#L187
https://github.com/cucumber/cucumber-js/blob/61842114a3e1e6af7310c4b90adb0964b1732ac5/features/target_specific_scenarios_by_tag.feature#L70
https://github.com/cucumber/cucumber-js/blob/61842114a3e1e6af7310c4b90adb0964b1732ac5/features/ambiguous_step.feature
https://github.com/cucumber/cucumber-js/blob/61842114a3e1e6af7310c4b90adb0964b1732ac5/features/cli.feature#L126

Current Behaviour
The formatters are inconsistent at the moment. This is a chance to bring all the console formatters into line.
Possible Solution
Historically, the console formatters used inheritance, using the Cucumber::Formatter::Console module to bring in this kind of functionality. That module got big and unwieldy, and I've recently done some refactoring to use composition instead, extracting out the Cucumber::Formatter::ConsoleCounts and Cucumber::Formatter::ConsoleIssues classes.
A suggested solution would be to rename ConsoleIssues to ConsoleRerunSuggestions or similar (because that's actually what it does) and build a new ConsoleIssues that prints the issues list in the same way as Cucumber-JS does.
Context & Motivation
Although the long-term plan is to build a central library of cross-platform formatters, that's a while off. This should be a relatively easy / quick implementation, and should improve the user experience.

@nodo I think this would be more useful than #401 (and probably easier too!)

@mattwynne Thanks! sounds good I will pick this up

👍

A question here is whether if it really is the job of the formatter to print the summary, snippets etc., not the least when considering it from the perspective of the future cross-platform formatters.
In Cucumber-JVM the summary, snippets etc. is not printed by formatters, but by a separate summary plugin. Currently there are two built in summary plugins, the default summary and the null summary - the latter not printing anything at all (when using the JUnit-integration of Cucumber-JVM you might want to have nothing at all printed to the console - that is accomplished by using the null formatter as console formatter and the null summary as summary plugin).
The differentiating question here is, if someone were to (with the --out option) write the pretty formatter output to a file, what should end up in the file? Only the output related to the "event bus events" the the pretty formatter outputs, or that and the summary, snippets etc.?
The printing of the summary in Cucumber-JVM was actually one of the things a did in the Cucumber-JVM code base, with the strict instruction that it "should not be part of the formatter, but rather be implemented close to where snippets are printed".

I've started extracting a couple of things a bit like you describe here @brasmusson, see https://github.com/cucumber/cucumber-ruby/blob/master/lib/cucumber/formatter/console_counts.rb for example.
I'm ambivalent about whether they're part of the formatters or not in the end, but I do think we should break these things up, and use composition rather than inheritance. The Formatter::Console mixin gives me the willies.

It's a good point @brasmusson. I have just started working on this and I think that separating the logic which prints the summary and the formatter will bring more flexibility and clean code.
@mattwynne if you agree I would try to refactor this as part of this PR. What do you think?

@nodo I think that refactoring the console formatter code to make it more modular would be an excellent plan. You might want to do it separately as a "pre-factoring" and then do this work on top of that.


Remove some tags

I think there is some ways to reduce the number of tags generated in intermediate procedure:

when semi-normalize, (x ~ y) or (x | y), we lookahead if y is a ParserRef(z), if so, we do not create a new tag for y, but leave it as z.
in the same situation described above, if y is Fixpoint(m, n), we can pass tag m (and use it for y), instead of introducing a new one.

@QuarticCat close this?

Fixed in #21


Support animation future?

Will sollumz support animations? I'm curious to know how you can add mocap (motion Matching), animation in gta 5, since there are a lot of free mocap animation on the Internet, I hope you will do about this wiki someday, should we expect support this year?
Check rokoko Library / actorcore Library.

https://github.com/Skylumz/Sollumz/wiki/YCD-|-Animations---Introduction
https://github.com/Skylumz/Sollumz/wiki/YCD-|-Import-&-Edit-&-Export
🤔


Mouse wheel scrolling has gotten slower

I see a noticable slowdown in scrolling speed when comparing latest builds with previous builds. I am quite sensitive to this and could not find a relevant setting. I am reporting this because I know we did make changes in this area.
I use an external Logitech mouse and here is the result of pushing the wheel down by 2 ticks (my wheel operates in a mode where I can scroll in discrete steps):

Compared to before:

My settings:
// Editor
"editor.fontFamily": "Fira Code",
"editor.fontSize": 13,
"editor.fontLigatures": true,
"editor.wordWrap": "off",
"editor.dragAndDrop": false,
"editor.folding": true,
"editor.formatOnSave": true,
"editor.codeLens": true,
"editor.renderWhitespace": "none",
"editor.snippetSuggestions": "none",
"editor.minimap.enabled": false,
"editor.find.globalFindClipboard": false,
"editor.inlineSuggest.enabled": true,
"editor.smoothScrolling": false,
"editor.suggestSelection": "first",
"editor.suggest.localityBonus": true,
"editor.hover.delay": 800,
"editor.guides.indentation": false,
"editor.renderControlCharacters": true,
"editor.gotoLocation.multipleDefinitions": "goto",
"diffEditor.renderIndicators": false,
"diffEditor.experimental.showMoves": true,
"diffEditor.useInlineViewWhenSpaceIsLimited": false,

Oh, this is very likely https://github.com/microsoft/vscode/commit/6ab0c535f92306ace1aca3998ac30f9d06a43b57
I am using window.zoomLevel: 1.

6ab0c53 is about fixing smooth scrolling post https://github.com/microsoft/vscode/pull/200776. The root cause of the slowdown you're experiencing is https://github.com/microsoft/vscode/pull/200776 which is a bug fix to https://github.com/microsoft/vscode/issues/146403 .
TL;DR with 1.66 scrolling became twice as fast when on high dpi monitors. With 1.86, the intent is to return scrolling to the speeds of 1.65.

"editor.mouseWheelScrollSensitivity": 1.5 saves me!


Allow specifying uid:gid:mode for empty mount directories created by rkt

Currently if container processes are running as a non-root user, there is no simple way to populate designated mountpoints with default content if they are not mounted by the user.

Content pre-staged in the mountpoint is masked by the empty directory created if nothing is mounted
Content pre-staged elsewhere in the ACI cannot be copied into the now-empty mountpoint because the directory is created with root-writable permissions only

Example: a process running as a user, that wants to read its config file from the /etc directory, so /etc is listed as a mountpoint in the manifest to make it easy to provide a user-written config file.
For this (and other reasons like supporting the ability to run processes with different uid/gid in the same ACI), it should be possible to specify default uid, gid and permissions for an empty mountpoint.  (This can currently be done by rkt at runtime, but the user running the container has to specify them on the command-line, rather than defaults being provided to make that not always necessary.)

I don't think that works -- when I include /bin/chown in a container and try to chown the empty mount directories from a pre-start script, I get "Operation not permitted" -- copying the files also fails of course.  The pre-start handler runs under the uid:gid specified with acbuild set-{user,group} (verified by having it echo the values of $UID and $GROUPS to stdout).
Even if that can be made to work, it would still be nice to have this be a first-class operation, as it is in many package managers.  File an issue against AppC?

Filed an issue against AppC for you. I'll leave this open for when/if AppC adds the necessary functionality.


[Doc] Documentation for distributed inference

In this PR:

Add anyscale's blogpost to README as a reference for continuous batching.
Documentation for distributed inference.

Fix #206

@WoosukKwon Just fixed all comments. Please take a look.


flutter build -v windows Error: Unable to generate build files

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
(win 10)
i had installed Visual Studio and Microsoft Visual C++(many times), and create a new flutter project in Visual Studio Code. I started it and  it worked at the first time, but just disappear quickly, and then i tried to run it, it didn't work.
i also changed the flutter channel to master, it didn't work.
Actual results
it just doesnt work, i found some same issues in 'issues' ,but they cant resolve my problem.
Logs

Logs

[   +8 ms] Skipping pub get: version match.
[ +131 ms] Initializing file store
[   +6 ms] Skipping target: gen_localizations
[   +3 ms] gen_dart_plugin_registrant: Starting due to {InvalidatedReasonKind.inputChanged: The following inputs have updated
contents: H:\Saves\Flutter\test0807\.dart_tool\package_config_subset}
[  +21 ms] gen_dart_plugin_registrant: Complete
[   +2 ms] Skipping target: _composite
[   +1 ms] complete
[   +3 ms] Launching lib\main.dart on Windows in debug mode...
[   +2 ms] H:\Tools\DevelopEnviroment\flutter\bin\cache\dart-sdk\bin\dart.exe --disable-dart-dev
H:\Tools\DevelopEnviroment\flutter\bin\cache\dart-sdk\bin\snapshots\frontend_server.dart.snapshot --sdk-root
H:\Tools\DevelopEnviroment\flutter\bin\cache\artifacts\engine\common\flutter_patched_sdk/ --incremental --target=flutter
--experimental-emit-debug-metadata -DFLUTTER_WEB_AUTO_DETECT=true
-DFLUTTER_WEB_CANVASKIT_URL=https://www.gstatic.com/flutter-canvaskit/cdbeda788a293fa29665dc3fa3d6e63bd221cb0d/ --output-dill
C:\Users\Connect\AppData\Local\Temp\flutter_tools.ad03dbd4\flutter_tool.47705a22\app.dill --packages
H:\Saves\Flutter\test0807\.dart_tool\package_config.json -Ddart.vm.profile=false -Ddart.vm.product=false --enable-asserts
--track-widget-creation --filesystem-scheme org-dartlang-root --initialize-from-dill
build\ade00fc64fa695e0daed02c1105f6d67.cache.dill.track.dill --verbosity=error --enable-experiment=alternative-invalidation-strategy
[  +17 ms] executing: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe -format json -products * -utf8 -latest
-version 16 -requires Microsoft.VisualStudio.Workload.NativeDesktop Microsoft.VisualStudio.Component.VC.Tools.x86.x64
Microsoft.VisualStudio.Component.VC.CMake.Project
[  +59 ms] Exit code 0 from: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe -format json -products * -utf8
-latest -version 16 -requires Microsoft.VisualStudio.Workload.NativeDesktop Microsoft.VisualStudio.Component.VC.Tools.x86.x64
Microsoft.VisualStudio.Component.VC.CMake.Project
[   +1 ms] [
             {
               "instanceId": "f66358ae",
               "installDate": "2023-08-07T04:41:22Z",
               "installationName": "VisualStudio/17.6.5+33829.357",
               "installationPath": "H:\\Tools\\VisualStudio\\VisualStudio",
               "installationVersion": "17.6.33829.357",
               "productId": "Microsoft.VisualStudio.Product.Community",
               "productPath": "H:\\Tools\\VisualStudio\\VisualStudio\\Common7\\IDE\\devenv.exe",
               "state":<PHONE_NUMBER>,
               "isComplete": true,
               "isLaunchable": true,
               "isPrerelease": false,
               "isRebootRequired": false,
               "displayName": "Visual Studio Community 2022",
               "description": "功能强大的 IDE，供学生、开放源代码参与者和个人免费使用",
               "channelId": "VisualStudio.17.Release",
               "channelUri": "https://aka.ms/vs/17/release/channel",
               "enginePath": "C:\\Program Files (x86)\\Microsoft Visual
               Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
               "installedChannelId": "VisualStudio.17.Release",
               "installedChannelUri": "https://aka.ms/vs/17/release/channel",
               "releaseNotes": "https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes-v17.6#17.6.5",
               "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=661288",
               "updateDate": "2023-08-07T04:41:22.6021785Z",
               "catalog": {
                 "buildBranch": "d17.6",
                 "buildVersion": "17.6.33829.357",
                 "id": "VisualStudio/17.6.5+33829.357",
                 "localBuild": "build-lab",
                 "manifestName": "VisualStudio",
                 "manifestType": "installer",
                 "productDisplayVersion": "17.6.5",
                 "productLine": "Dev17",
                 "productLineVersion": "2022",
                 "productMilestone": "RTW",
                 "productMilestoneIsPreRelease": "False",
                 "productName": "Visual Studio",
                 "productPatchVersion": "5",
                 "productPreReleaseMilestoneSuffix": "1.0",
                 "productSemanticVersion": "17.6.5+33829.357",
                 "requiredEngineVersion": "3.6.2115.31769"
               },
               "properties": {
                 "campaignId": "2030:1156185748f3461abaf2afc25493b6b6",
                 "channelManifestId": "VisualStudio.17.Release/17.6.5+33829.357",
                 "nickname": "",
                 "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\setup.exe"
               }
             }
           ]
[   +5 ms] Building Windows application...
[  +24 ms] <- compile package:test0807/main.dart
[   +1 ms] executing: H:\Tools\VisualStudio\VisualStudio\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe -S
H:\Saves\Flutter\test0807\windows -B build\windows -G Visual Studio 17 2022
[  +90 ms] Building Windows application... (completed in 109ms)
[+6870 ms] Exception: Unable to generate build files
[   +1 ms] "flutter run" took 26,056ms.
[   +2 ms]
           #0      throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
           #1      RunCommand.runCommand (package:flutter_tools/src/commands/run.dart:757:9)
           <asynchronous suspension>
           #2      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1336:27)
           <asynchronous suspension>
           #3      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #4      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
           <asynchronous suspension>
           #5      FlutterCommandRunner.runCommand.<anonymous closure>
(package:flutter_tools/src/runner/flutter_command_runner.dart:296:9)
           <asynchronous suspension>
           #6      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #7      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:244:5)
           <asynchronous suspension>
           #8      run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:91:9)
           <asynchronous suspension>
           #9      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #10     main (package:flutter_tools/executable.dart:91:3)
           <asynchronous suspension>

[  +62 ms] ensureAnalyticsSent: 57ms
[        ] Running 1 shutdown hook
[   +3 ms] Shutdown hooks complete
[        ] exiting with code 1

Flutter Doctor output

Doctor output

[√] Flutter (Channel stable, 3.10.6, on Microsoft Windows [版本 10.0.19045.3208], locale zh-CN)
    • Flutter version 3.10.6 on channel stable at H:\Tools\DevelopEnviroment\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f468f3366c (4 weeks ago), 2023-07-12 15:19:05 -0700
    • Engine revision cdbeda788a
    • Dart version 3.0.6
    • DevTools version 2.23.1
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[√] Windows Version (Installed version of Windows is version 10 or higher)

[!] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
    • Android SDK at G:\WorkAbout\AndroidSDK\android-sdk-windows
    X cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    X Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2022 17.6.5)
    • Visual Studio at H:\Tools\VisualStudio\VisualStudio
    • Visual Studio Community 2022 version 17.6.33829.357
    • Windows 10 SDK version 10.0.22000.0

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/windows#android-setup for detailed instructions).

[√] Connected device (3 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [版本 10.0.19045.3208]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 117.0.5927.0
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 115.0.1901.188

[√] Network resources
    • All expected network resources are available.

i resolved this by open vscode as administrator......................................


Draft

Codecov Report

Merging #17 (64134cd) into main (270fba7) will decrease coverage by 6.86%.
The diff coverage is 56.72%.

:exclamation: Current head 64134cd differs from pull request most recent head 2637c03. Consider uploading reports for the commit 2637c03 to get more accurate results

@@            Coverage Diff             @@
##             main      #17      +/-   ##
==========================================
- Coverage   72.67%   65.81%   -6.87%
==========================================
  Files           8       13       +5
  Lines         644     1012     +368
==========================================
+ Hits          468      666     +198
- Misses        176      346     +170

Impacted Files
Coverage Δ

src/structure_factor/plotting.py
13.09% <13.09%> (ø)

src/structure_factor/data/load_data.py
59.45% <16.66%> (-9.30%)
:arrow_down:

src/structure_factor/pair_correlation_function.py
42.64% <42.64%> (ø)

src/structure_factor/transforms.py
93.75% <44.44%> (-0.31%)
:arrow_down:

src/structure_factor/point_processes.py
48.22% <48.22%> (ø)

src/structure_factor/structure_factor.py
59.57% <53.33%> (-10.43%)
:arrow_down:

...c/structure_factor/tapered_estimators_isotropic.py
55.31% <55.31%> (ø)

src/structure_factor/point_pattern.py
50.00% <57.14%> (-17.35%)
:arrow_down:

src/structure_factor/spatial_windows.py
79.19% <58.00%> (-12.98%)
:arrow_down:

src/structure_factor/hyperuniformity.py
87.23% <63.63%> (+4.62%)
:arrow_up:

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4251abf...2637c03. Read the comment docs.


Equation 9.4

Ver: June 5, 2021
Equation 9.4 is missing a constant which is to represent the normalization constant for the denominator p(x;theta). More of a mathematical correctness for using "=" instead of a "proportionality" as the constant is independet of class C

fixed


[Bug] Gradio not launching

🐛 Bug
While in the conda env with python 3.11, using the latest nightly for M1, and Llama-2-13b-chat-hf-q4f16_1, the python script works fine as does executing the model via the python prompt. However, initiating Gradio crashes.
(mlc_llm) ~/src/mlc_llm%python -m mlc_chat.gradio --device metal:0
/Users/rj/miniforge3/envs/mlc_llm/lib/python3.11/site-packages/gradio/layouts/column.py:55: UserWarning: 'scale' value should be an integer. Using 0.3 will cause issues.
warnings.warn(
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/Users/rj/miniforge3/envs/mlc_llm/lib/python3.11/site-packages/mlc_chat/gradio.py", line 247, in
launch_gradio(ARGS.artifact_path, ARGS.device, ARGS.port, ARGS.share, ARGS.host)
File "/Users/rj/miniforge3/envs/mlc_llm/lib/python3.11/site-packages/mlc_chat/gradio.py", line 242, in launch_gradio
demo.launch(share=share, enable_queue=True, server_port=port, server_name=host)
TypeError: Blocks.launch() got an unexpected keyword argument 'enable_queue'`

It's easy fix yourself. Edit the file /Users/rj/miniforge3/envs/mlc_llm/lib/python3.11/site-packages/mlc_chat/gradio.py, or if you're installing mlc_chat from local source ./python/mlc_chat/gradio.py
At the end of the file, on line 242, change the following:
    demo.launch(share=share, enable_queue=True, server_port=port, server_name=host)

to:
    demo.queue(max_size=10)
    demo.launch(share=share, server_port=port, server_name=host)

and you should be able to run the gradio frontend


Fix issues with scrambled M5Stack screen

Not sure what the old code was trying to do, but this fixed issue #120

The old code handles when the user doesn't choose a predefined configuration and set up a rotation for the display, I guess you were using the M5Stack configuring every option for yourself.
Right now the only "validated" options for the M5Stack (when choosing the predefined configuration) are listed below:
https://github.com/littlevgl/lv_port_esp32/blob/32c4ab0f734b0d6afb1b444633aaba6b8e3c0320/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c#L119-L131
If you set everything for yourself the values to configure the MADCTL register are the following:
https://github.com/littlevgl/lv_port_esp32/blob/32c4ab0f734b0d6afb1b444633aaba6b8e3c0320/components/lvgl_esp32_drivers/lvgl_tft/ili9341.c#L147-L166
You can tell the values are different so your M5Stack behaves wrong.
Can you confirm or not if you are not using the predefined configuration?

@rangeli Sorry, i just pushed a PR, can you test #124?


Update enforcement export script

Closes https://github.local/CFPB/super-regular/issues/327

OH, also, when I ran ./frontend.sh, I saw a bunch of diffs of deleted files in the npm-packages-offline-cache folder. Make sure there's no git weirdness there before you merge.


Documentation: Options typo for esversion 10

In the page https://jshint.com/docs/options/, there's a typo in the link in the description for esversion bullet point for 10.  It's causing the URL to include the trailing ')'.  I think it's missing the '[' in front of "ECMAScript 10]".

I noticed that too—@jugglinmike @caitp. Also FYI, there are many pages on JSHint.com that aren't synched with the GitHub repo. Probably needs a fresh sync.


remove type aliases

fix #743

This is so ugly, I have no words for it:

set them in red ("role=red" or something like that ,look at Beast's reference).
Or put a bullet next to each one and at the top of the page put "bulleted items are deprecated and will be removed in a future release."

How do you feel about this change?


Update requirements (install fail on dateutil)

Got an error during installation
(seffnet) sichom@sichom-Precision-T7600:~/projects/seffnet$ seffnet web
Traceback (most recent call last):
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (python-dateutil 2.8.1 (/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages), Requirement.parse('python-dateutil<2.8.1,>=2.1; python_version >= "2.7"'), {'botocore'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sichom/software/miniconda/envs/seffnet/bin/seffnet", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    @_call_aside
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (python-dateutil 2.8.1 (/home/sichom/software/miniconda/envs/seffnet/lib/python3.7/site-packages), Requirement.parse('python-dateutil<2.8.1,>=2.1; python_version >= "2.7"'), {'botocore'})

Hi @martin-sicho sorry for the delay. Which version of python are you using? Seffnet is conflicting with botocore in your python installation. Why dont you try to install it and run it on a virtual environment?

Hi, no problem. It is python 3.7.5 and the version of botocore is 1.13.9. I was installing this with @cthoyt to a vanilla conda environment (just python in it) with pip to try something. We eventually solved the problem by manually installing python-dateutil through pip. It's just it did not work out of the box and we ran into this error. Hence, this issue.

Sorry guys I have the solution but just haven’t sent the PR. Don’t worry
On Fri, Nov 8, 2019 at 16:37 Martin Šícho<EMAIL_ADDRESS>wrote:

Hi, no problem. It is python 3.7.5 and the version of botocore is
1.13.9. I was installing this with @cthoyt https://github.com/cthoyt
to a vanilla conda environment (just python in it) with pip to try
something. We eventually solved the problem by manually installing
python-dateutil through pip. It's just it did not work out of the box and
we ran into this error. Hence, this issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/seffnet/seffnet/issues/26?email_source=notifications&email_token=ABGVXKATTC46X5HLWMEX62LQSWBUNA5CNFSM4JJDFOP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDSPFXY#issuecomment-551875295,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABGVXKBSZNP4SQX3655AQGDQSWBUNANCNFSM4JJDFOPQ
.


[BUG] Color Refinements not bucketed or with bucket type none throw error

Summary
If a color refinement (i.e. c_refinementColor which uses the ColorRefinements component) is configured to use bucket type: None or has unbucketed values that are shown (both these situations are common, especially the latter) the ColorRefinements component will throw an error:
TypeError: Cannot read property 'toLowerCase' of undefined
    at eval (webpack-internal:///./app/pages/product-list/partials/color-refinements.jsx:63:100)
    at Array.map (<anonymous>)
    at ColorRefinements (webpack-internal:///./app/pages/product-list/partials/color-refinements.jsx:39:76)

This is because the response will NOT include a presentationId for any unbucketed values (or at all if the refinement is configured to not bucket).
Recommend falling back to the value of the refinement if presentationId is not available.
The latter case (unbucketed values) is common with merchants as often new values can arrive in the incoming catalog feeds before they are bucketed by the merchant (in fact there's a system job that alerts merchants of this but they may not get to it before it lands on production)
Steps To Reproduce

Configure refinementColor in BM: either a) delete one of the existing buckets (i.e. blue) or b) change the entire refinement bucket type to None (NOTE: this will remove all buckets making it harder to get them back to the original configuration; recommend exporting catalog first) or c) (not tested yet) remove the presentationId from a bucket

Expected result
The page does not error
Actual result
Exception (internal server error)
System Information (as applicable)
Browser:
Node version:
pwa-kit version: 2.5.0
Desktop OS:
Mobile Device Info:
Additional information

Thanks @clavery for reporting this bug.