fix: attachment filename error in sqllab with i18n

SUMMARY
Because sqllab tab name supports i18n. The sqllab .CSV button will use that name for the attachment file name. But the sqllab query tab name can be something unsafe in HTTP header. Without URL encoding this will cause an error like this:
Error on request:
Traceback (most recent call last):
  File "/Users/auxten/.pyenv/versions/3.7.6/envs/superset-0.35.2/lib/python3.7/site-packages/werkzeug/serving.py", line 323, in run_wsgi
    execute(self.server.app)
  File "/Users/auxten/.pyenv/versions/3.7.6/envs/superset-0.35.2/lib/python3.7/site-packages/werkzeug/serving.py", line 315, in execute
    write(data)
  File "/Users/auxten/.pyenv/versions/3.7.6/envs/superset-0.35.2/lib/python3.7/site-packages/werkzeug/serving.py", line 276, in write
    self.send_header(key, value)
  File "/Users/auxten/.pyenv/versions/3.7.6/lib/python3.7/http/server.py", line 516, in send_header
    ("%s: %s\r\n" % (keyword, value)).encode('latin-1', 'strict'))
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 49-54: ordinal not in range(256)

From the discussion here: https://stackoverflow.com/questions/93551/how-to-encode-the-filename-parameter-of-content-disposition-header-in-http#comment65669446_6745788
And blog here: https://fastmail.blog/2011/06/24/download-non-english-filenames/
Here comes the patch.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

Change language to anything other than English.
Sqllab run some query with a new tab.
Hit the .CSV button, succeed.

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #11762 (089f680) into master (1ef3a02) will decrease coverage by 0.78%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #11762      +/-   ##
==========================================
- Coverage   63.14%   62.36%   -0.79%
==========================================
  Files         897      443     -454
  Lines       43461    27262   -16199
  Branches     4188        0    -4188
==========================================
- Hits        27444    17002   -10442
+ Misses      15837    10260    -5577
+ Partials      180        0     -180

Flag
Coverage Δ

javascript
?

python
62.36% <100.00%> (-0.97%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/views/core.py
73.74% <100.00%> (-0.46%)
:arrow_down:

superset/db_engines/hive.py
0.00% <0.00%> (-85.72%)
:arrow_down:

superset/databases/commands/create.py
32.65% <0.00%> (-59.19%)
:arrow_down:

superset/db_engine_specs/hive.py
54.08% <0.00%> (-29.97%)
:arrow_down:

superset/views/database/mixins.py
59.64% <0.00%> (-22.81%)
:arrow_down:

superset/db_engine_specs/presto.py
69.76% <0.00%> (-12.53%)
:arrow_down:

superset/db_engine_specs/mysql.py
79.59% <0.00%> (-12.25%)
:arrow_down:

superset/databases/commands/update.py
85.71% <0.00%> (-8.17%)
:arrow_down:

superset/databases/dao.py
94.11% <0.00%> (-5.89%)
:arrow_down:

superset/databases/api.py
81.93% <0.00%> (-5.47%)
:arrow_down:

... and 472 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ef3a02...089f680. Read the comment docs.

@auxten you need to run black on the PR. Check the contributing guide for instructions on setting up pre-commit hooks to do it automatically.

@auxten you need to run black on the PR. Check the contributing guide for instructions on setting up pre-commit hooks to do it automatically.

Sorry for that, all checks pass now. @villebro

Thanks @auxten!


[TOPIC-GPIO] drivers: gpio_mchp_xec: update to use new GPIO API

Update driver code and board files to use new GPIO configuration flags
such as GPIO_ACTIVE_LOW. Also add implementation of new port_* driver
API as well as gpio_pin_interrupt_configure function.

Yes, the test passed. I also tried manually with the sample samples/drivers/gpio/.

Rebased to latest topic-gpio and tested with both gpio_basic_api and gpio_api_1pin.

The code looks good. Since the driver supports simultaneous input/output configuration could you also run the tests/drivers/gpio/gpio_api_1pin test? All the test_gpio_port_* test cases are unfortunately going to fail on your board (I'm working on the fix), please ignore. The remaining should pass.

I ran it too and it passed.


[Bug] Head Node service is not getting exposed on Internal Load Balancer despite proper annotations

Search before asking

[X] I searched the issues and found no similar issues.

KubeRay Component
ray-operator
What happened + What you expected to happen
I am trying to set up the head node service exposed as LoadBalancer but on an internal one, but KubeRay is still using public LoadBalancer despite giving proper annotations.
I have all the necessary modules installed in my EKS cluster, such as AWS Load Balancer, and I also have a sample service working to expose internal LoadBalancer. That way, I know my annotations are correct.
For the ray cluster setup, I have given annotations both at the top level and within the template - hoping it will be picked from one of them.
Pls see the config snippet below.
Reproduction script
apiVersion: ray.io/v1alpha1
kind: RayCluster
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: ${RAY_CLUSTER}
  namespace: ${RAY_CLUSTER_NS}
  annotations:
   service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
   service.beta.kubernetes.io/aws-load-balancer-scheme: internal
   service.beta.kubernetes.io/aws-load-balancer-internal: "<IP_ADDRESS>/0"
   service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip
spec:
  rayVersion: '2.0.0'
  enableInTreeAutoscaling: true

  autoscalerOptions:
    upscalingMode: Default
resources.
    idleTimeoutSeconds: 60
    imagePullPolicy: Always
required.
    resources:
      limits:
        cpu: "1"
        memory: "2Gi"
      requests:
        cpu: "1"
        memory: "2Gi"
  ######################headGroupSpec#################################
  headGroupSpec:
    serviceType: LoadBalancer
    enableIngress: true

    rayStartParams:
      # Flag "no-monitor" will be automatically set when autoscaling is enabled.
      dashboard-host: '<IP_ADDRESS>'
      block: 'true'
      num-cpus: '0' # can be auto-completed from the limits
    template:
      metadata:
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
          service.beta.kubernetes.io/aws-load-balancer-scheme: internal
          service.beta.kubernetes.io/aws-load-balancer-internal: "<IP_ADDRESS>/0"
          service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip
...

Anything else
No response
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

any updates on this, pls ?

any updates on this, pls ?

Thank you for your followup! I started to take a look at the ingress support in KubeRay and found some questions. Currently, I am working on https://github.com/ray-project/kuberay/pull/646. The PR is a blocker of #637. Thank you for your patience.

Hi @snow01,
I started to work on this issue by getting some context of aws-load-balancer-controller, ALB, and NLB. I will try to reproduce this issue on my EKS.
I have several questions on this issue.

I am trying to set up the head node service exposed as LoadBalancer but on an internal one, but KubeRay is still using public LoadBalancer despite giving proper annotations.

Q1: What is the definition of "internal one"? Do you mean aws-load-balancer-scheme: internal?
Q2: What is the definition of "public LoadBalancer"? Do you mean aws-load-balancer-scheme: internet-facing?

service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internal
service.beta.kubernetes.io/aws-load-balancer-internal: "<IP_ADDRESS>/0"
service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip

Q3: Why the value of aws-load-balancer-internal is "<IP_ADDRESS>/0"? This doc indicates that the value should be boolean. In addition, it was deprecated in favor of aws-load-balancer-scheme. The following is the deprecation note in doc. Maybe this annotation is unnecessary?

This annotation is deprecated starting v2.2.0 release in favor of the new aws-load-balancer-scheme annotation. It will be supported, but in case of ties, the aws-load-balancer-scheme gets precedence.

Q4: The doc indicate that nlb-ip is deprecated. In addition, the backwards compatibility note for nlb-ip type in the doc said that nlb-ip is equivalent to aws-load-balancer-type: external and aws-load-balancer-nlb-target-type: ip. Although there is no conflict, maybe it is better to use external instead.

For backwards compatibility, both in-tree and AWS Load Balancer controller supports nlb-ip as value of service.beta.kubernetes.io/aws-load-balancer-type annotation. The controllers treats it as if you specified both annotation below:
service.beta.kubernetes.io/aws-load-balancer-type: external
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip

Q5: Why do we need enableIngress: true?
Q6: What's the result of kubectl describe $YOUR_HEAD_SERVICE?
Thank you!

I took a look at the source code. The annotations will not be passed to the head service. We can double-check this with the answer to Q6 mentioned above. If so, I will update KubeRay to enable the annotation propagations.
https://github.com/ray-project/kuberay/blob/3f7b34c731bd2ee413fb622a05124b52f1588f20/ray-operator/controllers/ray/common/service.go#L13-L52

Q1: What is the definition of "internal one"? Do you mean aws-load-balancer-scheme: internal?

Yes

Q2: What is the definition of "public LoadBalancer"? Do you mean aws-load-balancer-scheme: internet-facing?
Yes

Q3: Why the value of aws-load-balancer-internal is "<IP_ADDRESS>/0"? This doc indicates that the value should be boolean. In addition, it was deprecated in favor of aws-load-balancer-scheme. The following is the deprecation note in doc. Maybe this annotation is unnecessary?

You are right, this should not matter.

Q4: The doc indicate that nlb-ip is deprecated. In addition, the backwards compatibility note for nlb-ip type in the doc said that nlb-ip is equivalent to aws-load-balancer-type: external and aws-load-balancer-nlb-target-type: ip. Although there is no conflict, maybe it is better to use external instead.

This is also deprecated. So doesn't matter.
Following configurations are the one that should matter -
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
service.beta.kubernetes.io/aws-load-balancer-scheme: internal

Thank @snow01 for your reply!
I have added this issue to today's KubeRay community sync agenda. If you are interested in the discussion, welcome to join the community sync!
(1) Agenda
(2) Google Meet

Let me try this as per doc and confirm back. Thanks

KubeRay has already exposed the entire head service for users https://github.com/ray-project/kuberay/pull/1040. Everything should be configurable now.


Results from benchmarking pairing libraries

Hi all, I would like to share with you guys some benchmarks I ran on zexe, gnark/goff and zkcrypto's BL12-381. I believe this is a good place to start to identify what the key avenues for optimisation are.
From these results, it seems that major areas of optimisation are in

Fq operations, including: add, sub, mul, inverse, negate, to and from montgomery form. Analogous results hold for field extensions, but we should check for improvement here after optimising the base field. From pure optimisations in Fq ops, one can aspect an overall improvement in other parts of the library by approximately 120%.
G1 ops: this can be partly explained by the Fq performance. However, scalar mul seems significantly slower, should check what differences exist between gnark and zexe.

More benchmarking should be done once Fq has been sufficiently optimised.
Yet to benchmark:
zk-swap-libff/libff
barretenberg
Results:
BLS12-381
Fq
goff

Zexe

G1
gnark

Zexe

zkcrypto

Pairing
Zexe

zkcrypto

gnark

Thanks for the comprehensive comparison! I agree that there seem to be some nice avenues for optimization here! Let's collect here the things that goff is doing to optimize arithmetic.

Nice! Interesting. It seems that G1 additions gnark < zexe < zkcrypto, while gnark is slowest in full pairing.

Dear all, after some more exploration, here are my findings:

First of all, I tried to keep extraneous apps to zero this time, which explains my overall lower timings.
The way that Goff and other libraries benchmark is different from how Zexe benchmarks. Zexe makes one function call per b.iter loop. As I suspect, due to the function call overhead, this actually severely affects functions that run very quickly i.e. on the order of ns. Hence, my suggestion is that for such functions, one should iterate over a for loop inside the b.iter call for 1000 iterations instead.
Applying this to fields of size 384 and 256, I got Fq mult timings of 46-51ns and 25-26ns respectively, and Fq squarings of 44-47ns and 23-24ns. This is on par or faster than goff (I got new timings for goff: 384: 50-53 ns mult/48-49 ns square and 256:  25-26 ns mult/23-24 ns square)

I applied Goff's suggested optimisation of no carry, and I got a speedup of 3-5ns (down from 52-54ns), less than advertised, but still significant. The full pairing was about 6% faster with this optimisation.
I rewrote the multiplication as a compact loop and used the crate unroll to make sure it was unrolled. Doing this did not seem to affect performance at all vis a vis writing the unrolled loops out explicitly. I suggest to keep this style as it is much more readable and compact.
With regards to G1 timings, the gap was narrowed by about half, but I should check back once squaring is optimised too.
My suggestions:

[ ] Benchmark all functions running at < 500 ns by iterating thousand-fold
[ ] Macro-unroll all field muls, squarings and montgomery reductions for more compact and readable form
[ ] Implement no-carry optimisation for squaring

Thanks for the investigation @jon-chuang ! If you could make a PR with the Goff changes and with the unroll! macro that would be awesome!
Re: benchmarking it’s a slightly more involved process, as there’s a lot of code that would have to be changed :/

@Pratyush Actually, I don't think so. I will write some macros for benches for the algebra-benches crate. I will iterate by 1000 everything accept G1/G2 muls, pairing-related ops, and sqrt.
Not all of the curves have the same extension fields, I will figure out how I want to tackle this. I will split up the macros by type.
By the way, is it normal that one needs +nightly for #[feature(test)]?

If you'd like to update the benchmarks anyway, you might want to use something like criterion.

Criterion is a lot slower and verbose however, we may not want it for minor benchmarks. It takes ages to run the full benchmark as it is.
Although, I could try to find a way to tweak the sampling time. Further, I should look into whether there is similar functionality to cargo benchcmp.

I noticed a troubling phenomenon. G1 doubling seems to be slower. This seems to affect G1 scalar mult as well. @Pratyush , can you replicate this?

G1 Benchmark
name                                          old-ec ns/iter  no_carry_ec ns/iter  diff ns/iter  diff %  speedup
 bls12_377::ec::g1::bench_g1_add_assign        1,178           1,158                         -20  -1.70%   x 1.02
 bls12_377::ec::g1::bench_g1_add_assign_mixed  820             819                            -1  -0.12%   x 1.00
 bls12_377::ec::g1::bench_g1_double            591             602                            11   1.86%   x 0.98
 bls12_377::ec::g1::bench_g1_mul_assign        295,849         297,099                     1,250   0.42%   x 1.00
 bls12_377::ec::g1::bench_g1_rand              294,117         294,953                       836   0.28%   x 1.00
 bls12_377::ec::g2::bench_g2_add_assign        5,121           5,018                        -103  -2.01%   x 1.02
 bls12_377::ec::g2::bench_g2_add_assign_mixed  3,546           3,837                         291   8.21%   x 0.92
 bls12_377::ec::g2::bench_g2_double            2,332           2,279                         -53  -2.27%   x 1.02
 bls12_377::ec::g2::bench_g2_mul_assign        1,237,827       1,196,713                 -41,114  -3.32%   x 1.03
 bls12_377::ec::g2::bench_g2_rand              1,193,239       1,180,192                 -13,047  -1.09%   x 1.01
 bls12_381::ec::g1::bench_g1_add_assign        1,172           1,170                          -2  -0.17%   x 1.00
 bls12_381::ec::g1::bench_g1_add_assign_mixed  817             859                            42   5.14%   x 0.95
 bls12_381::ec::g1::bench_g1_double            588             614                            26   4.42%   x 0.96
 bls12_381::ec::g1::bench_g1_mul_assign        297,426         305,844                     8,418   2.83%   x 0.97
 bls12_381::ec::g1::bench_g1_rand              299,963         303,203                     3,240   1.08%   x 0.99
 bls12_381::ec::g2::bench_g2_add_assign        4,229           4,122                        -107  -2.53%   x 1.03
 bls12_381::ec::g2::bench_g2_add_assign_mixed  2,902           3,158                         256   8.82%   x 0.92
 bls12_381::ec::g2::bench_g2_double            1,815           1,788                         -27  -1.49%   x 1.02
 bls12_381::ec::g2::bench_g2_mul_assign        980,179         963,952                   -16,227  -1.66%   x 1.02
 bls12_381::ec::g2::bench_g2_rand              1,003,275       951,161                   -52,114  -5.19%   x 1.05
 sw6::ec::g1::bench_g1_add_assign              4,416           4,403                         -13  -0.29%   x 1.00
 sw6::ec::g1::bench_g1_add_assign_mixed        3,228           3,094                        -134  -4.15%   x 1.04
 sw6::ec::g1::bench_g1_double                  2,765           2,927                         162   5.86%   x 0.94
 sw6::ec::g1::bench_g1_mul_assign              1,898,836       1,928,669                  29,833   1.57%   x 0.98
 sw6::ec::g1::bench_g1_rand                    1,916,670       1,931,870                  15,200   0.79%   x 0.99
 sw6::ec::g2::bench_g2_add_assign              35,544          33,616                     -1,928  -5.42%   x 1.06
 sw6::ec::g2::bench_g2_add_assign_mixed        23,615          22,878                       -737  -3.12%   x 1.03
 sw6::ec::g2::bench_g2_double                  19,324          19,530                        206   1.07%   x 0.99
 sw6::ec::g2::bench_g2_mul_assign              13,692,588      13,392,381               -300,207  -2.19%   x 1.02
 sw6::ec::g2::bench_g2_rand                    13,614,042      13,254,145               -359,897  -2.64%   x 1.03

I believe this would help us in our diagnosis of why G1 ops are much slower than in gnark. Could it be something to do with inlining/memory usage (e.g. cache-friendliness)? It seems odd that even though our Fp ops are faster or on par with goff, and even though we both use the exact same formulas, the go implementation would be faster than rust. It's a bit of an enigma. I don't know if careful profiling will help.

I did a final careful benchmarking of the recent PR #163. My first reccomendation is to try adding RUSTFLAGS="--emit=asm" as this can improve speed significantly.
I also find that Fq mul_assign is about 7-15% faster, square_in_place is about 5% faster, into_repr is about 5-12% faster, and pairing computations are about 1.5-6% faster.
The timings I got vis a vis goff/gnark are essentially the same, but now faster. G1 is the only area where we are still lagging behind.
New avenues of optimisation I would like to look into are G1 add_assigned_mixed, which I believe is part of the bottleneck for G1 mul_assign. However, the majority of the bottleneck seems to come from pippenger multiexponentiation. So this is something I will work on for now, which is in parallel with my intention to add WebGPU support for the parallel form of multiexponentiation.
By the way, there are no benchmarks for MSM, but this is a very critical number for provers.

I think the issue with scalar multiplication should be gone now? From local benchmarks mixed_add, add, and double are all faster
Btw the scalar multiplication benchmarks are noisy because the scalars are random and have varying hamming weight, so that’s something to keep in mind

@Pratyush nonetheless, the scalar_mul timings are significantly slower (1.22x) than gnark. Further, the mixed_add timings are 1.06x slower. This is a finding I repeatedly arrive at.

I also find that inverse is significantly slower than goff (2.5x). How performance critical is this @pratyush?

Hmm inversion isn't strictly performance critical because the elliptic curve code does try to minimize it. For the sqrt code, it would be helpful to optimize it is critical for point compression.

(That being said, having faster algorithms is always welcomed =D)

Also, thanks again for the thorough benchmarks!

I see. Is there any scenario in which batch inversion actually helps performance? Perhaps for something like pippenger-scalar mul/MSM?

Currently our batch inversion code takes in affine coordinates and uses mixed addition.

How about batch inversion in particular?

Hmm since batch_inversion performs O(N) multiplications but only one inversion, the cost of a single inversion is usually drowned out for large N. However, using batch inversion with a small N  might make the additional cost of inversion significant again.

I see. But where is batch inversion used? You mentioned it is used for mixed addition? Or is that standard addition?

I also find that inverse is significantly slower than goff (2.2x 13us vs goff’s 6us). Further, sqrt is also very slow (1.5x slower, 75/54us vs goff's ~25/54us). How performance critical are these @Pratyush?

Goff inversion is not constant-time. If that is desired, you cannot compare it with the usual Little Fermat's based inversion.
Also as mentionned by @Pratyush for pairing in a Fp12 tower, the tower setup passes the inversion down the tower and so the ratio compared to a Fp operation stays acceptable and we can use Jacobian/Projective coordinates to avoid the inversion tax (at the price of more addition/multiplication).
More discussion on fast constant-time modular inversion here: https://github.com/randombit/botan/issues/1479#issuecomment-602178718
Also, AFAIK the fastest open-source pairing library is MCL if your CPU is at least Broadwell (from) and supports MULX/ADOX/ADCX instructions. The gain is about 15% on my machine. See benchmarks at https://github.com/mratsim/constantine/pull/18 (Note that in those benchmark I was still using inversion via Little Fermat Exponentiation, my new constant time inversion algorithm is about 30% faster though 10x slower than Goff non-constant-time version or GMP).

Since we are on par with your library, that would make us about 22-30 clock cycles behind MCL. That's roughly 25% extra bloat. That's remarkable.
@pratyush, given this amazing result, I may find some time to read the MCL code and implement the ADX instructions in a maintainable way, since x86 intrinsics do not support them due to lack of support from LLVM.

As some updates, I managed to integrate Fp256 assembly leading to about 25% speedup in Fp256 mults. But it is proving harder to write generic assembly code than expected, and the inline assembly module doesn't really play nice with catching errors and doing what it's supposed to. I am going to try to simplify my code by writing smaller chunks of inline assembly that are easier to verify correct in a piecemeal fashion and leaves less for me to reason about
Also, I should use the following tool for benchmarking clock cycles: https://git.zx2c4.com/kbench9000/about/. But it may only work with C?


Idea

Its not a issue but its a feedback , Include one line code to deal with streaming http response

Thanks for the feedback. Could you provide some links or more information about this? We'd be happy to tackle it, if we can.

https://github.com/Priyansh2001here/StreamingHttpResponse/blob/master/StreamingHttpResponse/apps/stream/templates/stream/index.html
i got this , I was searching for projects based on StreaminHttpResponse in dango and i found this hope it will help you

Hi people

i'll work on it

Closing due to inactivity.


any way to get this to work with ipython?

Is there anyway to get this to fix ipython as well?

Going through the issues in this repo. @dalupus I know this is old but are you still experiencing this problem?

I have the problem that scripts installed using setuptools`s entry_points are not fixed by this.
IPython use this to install the ipython executable so this is the actual culprit.


[SPARK-15153] [ML] [SparkR] Fix SparkR spark.naiveBayes error when label is numeric type

What changes were proposed in this pull request?
Fix SparkR spark.naiveBayes error when label is numeric type.
See details about how to reproduce this bug at SPARK-15153.
How was this patch tested?
Add unit test.

Test build #57877 has started for PR 12930 at commit bc2c0a3.

Test build #57877 has finished for PR 12930 at commit bc2c0a3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57877/
Test PASSed.

cc @mengxr @jkbradley

Is it okay to always cast the target label column to string?

It's feasible to always cast the target label column to string, but we should refactor RFormula. RFormula handles label of numeric type and string type differently, we should unified them. But I'm not sure whether we can finish that before 2.0.

Do we still need this?

It's still a problem.  (I just tried the unit test on master.)  Sorry for the delay @yanboliang !

Should/Would this be better if this is handled in the RFormula level? I could see other models could benefit from this too, in addition to just R.

Oh!  Now I see the JIRA's linked JIRA.  I agree that updating RFormula is a better solution and that we can close this PR.  I'll check out the other PR.

Move the new fix to #15431 which leverages RFormula forceIndexLabel, more succinct.


What is the syntax for converting to pdf?

I have tried naively with output = pypandoc.convert('paper.md', 'pdf') which won't work.

@gregist Can you provide the native pandoc command that works for the conversion? That will give me a clearer idea of why it doesn't work.

@bebraw This line works for me.
os.system("pandoc " + "paper.md -o paper.pdf --latex-engine=xelatex)

@gregist So in order to work it requires that latex-engine thinger? Is that the only way to get pdf output out of pandoc? If so, we can treat this as a special case or make something more generic and give access to those custom parameters of pandoc.

@bebraw Yes, you need LaTex preinstalled in your system.
official document here.

@gregist Ok, got it. So it defaults to pdflatex unless engine is specified. What if I provided you latexEngine parameter? Then you could do something like this:

output = pypandoc.convert('paper.md', 'pdf', latexEngine='xelatex')

Would that work for you?

Nope.
Traceback (most recent call last):
  File "***\pandocd.py", line 3, in <module>
    output = pypandoc.convert('paper.md', 'pdf', latexEngine='xelatex')
TypeError: convert() got an unexpected keyword argument 'latexEngine'
[Finished in 0.2s with exit code 1]

@gregist Yeah, I meant I can implement that parameter there unless someone comes up with a nicer API. :+1:

@bebraw Well... that should work.
nice project btw.

@gregist Never mind... Looks like the functionality we need is already there. Try

output = pypandoc.convert('paper.md', 'pdf', extra_args=['--latex-engine=xelatex'])

I think that should be equivalent to what you wrote earlier.
There's so much functionality I forgot about that. :+1:

Traceback (most recent call last):
  File "***\try.py", line 3, in <module>
    output = pypandoc.convert('paper.md', 'pdf', extra_args=['--latex-engine=xelatex'])
  File "C:\Python27\lib\site-packages\pypandoc.py", line 94, in convert
    outputfile=outputfile, filters=filters)
  File "C:\Python27\lib\site-packages\pypandoc.py", line 135, in _convert
    outputfile=outputfile, filters=filters)
  File "C:\Python27\lib\site-packages\pypandoc.py", line 208, in _process_file
    'Pandoc died with exitcode "%s" during conversation: %s' % (p.returncode, stderr)
RuntimeError: Pandoc died with exitcode "9" during conversation: pandoc: Unknown writer: pdf

To create a pdf with pandoc, use the latex or beamer writer and specify

an output file with .pdf extension (pandoc -t latex -o filename.pdf).

[Finished in 0.2s with exit code 1]

@gregist I made it discard --to parameter for pdf and published 0.9.9. That should fix it.

@bebraw COOL. It works!
pypandoc won't recogonize some formatting argument though. e.g.
output = pypandoc.convert('paper.md', 'pdf', outputfile='paper.pdf', extra_args=['-V geometry:margin=1.5cm'])
No error coughed. Just that the formatting isn't right.

@gregist If you look at the current implementation extra_args are just passed to subprocess.Popen. Can you try against that or os.system so we get something to compare with again?

@bebraw
the code:
import pypandoc
import os
output = pypandoc.convert('paper.md', 'pdf', outputfile='paper.pdf', extra_args=['-V geometry:margin=1.5cm'])
os.system("pandoc "+'paper.md -o paper2.pdf -V geometry:margin=1.5cm')

the output:

pypandoc

os.system

I don't know how to implement with subprocess.Popen, or you can pass me the code and I'd be happy to test it for you.

That's enough to debug. Thanks. I'll look into it.

@gregist Looks like it's very strict about parameters. The following works for me:

output = pypandoc.convert('demo.md', 'pdf', outputfile='paper.pdf', extra_args=['-V', 'geometry:margin=1.5cm'])

I'm not sure if this a bug in pypandoc itself or how to even start to fix it since it's so fundamental. There's probably a good reason why it works this way, though.

@bebraw

The following works for me:

can confirm. cheers!
Yes it's a little weird why seperated quotes work while otherwise does not. Maybe we should 'feature' this bug by including it in the documentations.

Sure. I'll add a brief note about it.


Why can't I use the latest version?

Hello,
In the docker-compose.yml file, when I change the images to latest, then the program does not work properly. Why?
Thank you.

WDYM? I work with latest and it all works

WDYM? I work with latest and it all works

Hi,
Thank you so much for your reply.
I mean the docker-compose.yml file is similar to the https://pastebin.mozilla.org/DMSssS1d.


github username: Mitz190

Hello I'm Mithali Redkar
I would love to join the Organzition as I love open source contribution ,Open Source provides means For Learner to learn from others the beauty here is even a beginner can contribute into the projects .
I love to solve the pattern problems.
github username:Mitz190

Thank you for checking our project. You have been invited to join the organization. :)


REQUEST: New membership for cicoyle

GitHub Username
cicoyle
Requirements

[x] I have reviewed the community membership guidelines (https://github.com/dapr/community/blob/master/community-membership.md)
[x] I have enabled 2FA on my GitHub account. See https://github.com/settings/security
[x] I have subscribed to the Dapr community Discord server
[x] I am contributing (any of the following apply: issues, discussions, PRs, reviews) to 1 or more Dapr repositories

List of contributions to the Dapr project

distributed scheduler proposal
opened an issue for removing the conformance test requirement for binding components && updated the docs
helped with the v1.12 endgame: updated quickstarts/tutorials to check rc && tested/validated dashboard
reviewed this zeebe components certification test PR

+1 nonbinding and only partially biased 😉  👯♀

+1

+1 non-binding

+1

+1 binding

+1

Invite sent.


ENH: make Wavelet, WaveletPacket, WaveletPacket2D and ContinuousWavelet pickleable

closes #545
This PR makes the Wavelet and WaveletPacket class pickleable.
If this seems reasonable, we should extend this to WaveletPacket2D as well
TODO:

[x] make additional classes pickleable
[x] add tests

All green, in it goes. Thanks @grlee77


Material Search Bar within a Navigation Controller

Hello! Is there an example of how to place the Material Search Bar in a Navigation Controller, inside of a Tab Bar Controller? So basically, a setup like Facebook mobile, but instead of their version of the search bar, it would show the Material search bar. I didn't see any examples, so I figured Id ask

Hey, You should be able to add a SearchBar to the navigationItem for the NavigationController.
Please update to Material 2.3.17, and then try the following code.
let searchBar = SearchBar()

navigationItem.centerViews = [searchBar]
navigationItem.contentViewAlignment = .full // Tells the NavigationController to spread the contentView (centerViews) across all the available space.

// Example buttons.
navigationItem.leftViews = [menuButton]
navigationItem.rightViews = [starButton, searchButton]

That should work :)

Okay, then I must be missing something because with this code:

Your code isn't visible, can you share again please?

self.searchBar = SearchBar()
self.navigationItem.centerViews = [self.searchBar]
self.searchBar.delegate = self
self.navigationItem.contentViewAlignment = .full
var buttons = IconButton
buttons.append(IconButton(image: Icon.cm.menu))
buttons.append(IconButton(image: Icon.cm.moreVertical))
self.navigationItem.leftViews = buttons
Nothing appears on the navigation bar at all
Heres what a po of the search bar produces:
<Material.SearchBar: 0x7faee5402ae0; frame = (0 0; 0 49); autoresize = W; layer = <CALayer: 0x6080000226a0>>

Where is this code placed? Is it in the rootViewController of the NavigationController?

Yes it is, in -viewDidLoad

That is strange. Would you be able to share a sample project with me at<EMAIL_ADDRESS>

Sure, I'll send over exactly what I'm looking at in a few minutes! :)

Thank you :)


Hint at package name with dash/underscore mismatch

Say you have this in your Cargo.toml
[dependencies]
rustc_serialize = "*"

You'll get this error:
error: no matching package named `rustc_serialize` found (required by `my_crate`)
location searched: registry https://github.com/rust-lang/crates.io-index
version required: *

It would be nice if cargo instead said something like “did you mean rustc-serialize”?

Agreed! Would be an awesome improvement to have.

It seems fixed now:
error: no matching package named `rustc_serialize` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
perhaps you meant: rustc-serialize
required by package `my_crate v0.1.0 (/path/to/my_crate)`

Yes. Thanks!


Fix call stack exceeded bug

Description
Whenever a component re-renders, the new component links to the old component's node ref and if this repeats long enough, the call stack will be exceeded.
Rather than link, the new component should reuse the existing node ref
Fixes https://github.com/yewstack/yew/issues/1449
Checklist

[x] I have run cargo make pr-flow
[x] I have reviewed my own code
[x] I have added tests

Oops, will fix


Install Elm Packages when building Docker

We know at build time which Elm packages we'll need to run the app, so let's install them as part of the image rather than wait until it's needed (for instance, by CI).

From Circle it doesn't appear that this is working (it's still installing the elm dependencies), but it still seems like not a terrible idea. 🤔


limit result set

Hi, wondering what the best approach would be for the client to query the following:

Sort by field
Limit to n records

Thanks

have you tried page[size]=n ?

Enable the paged paginator (https://github.com/cerebris/jsonapi-resources#pagination) and specify the page size. Use the sort option for the field.

worked!


Fix miscellaneous related with type def files

What: fix inconsistency of type def files

Checklist:

[x] Documentation
[x] Tests
[x] Ready to be merged
[x] Added myself to contributors table

Codecov Report

Merging #296 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #296   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          10     10
  Lines         170    170
  Branches       46     46
=====================================
  Hits          170    170

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f35d7b2...cc12209. Read the comment docs.

@luke-john Pardon me? No-newline marks are removed, aren't they?

From the diffs in Files Changed it looks like they've been removed in a number of files.

@luke-john
No-newline marks are removed, that's not a problem (at least I believe), so I can't get the problem.
Sorry for my ignorance, and what's your point?

No sorry you're absolutely right.
I was completely misreading the diff, apologies.


fix(standalone): use aria label from input

Summary

Use aria-label property from given input element on the new input element.
Result

When using standalone version (with places.js for instance) the aria-label on the target input element will be forwarded to the input element created by autocomplete.js.

Coverage remained the same at 88.947% when pulling 9c1ee1ee2083560c806c4efd2051cc3b21dc61ea on kontrollanten:fix-aria_label into 84b17140ae34e0a96558efc04adba7a935841857 on algolia:master.

This looks correct, but I will not have time to review this request until after the holidays. Could you describe which behaviour you are trying to fix? I thought there were already labels?

Can we have a release for this soon? :)

I released 0.36.0 earlier today


Chapter 3 - Classification - shuffling should happen first?

Just a quick question - in the Chapter 3 - Classification we split the data to training and test set, then shuffle the training set to ensure all cross-validation folds are similar without missing digits.
Shouldn't it be shuffle first, then split? As far as I understand - and because the data seems to be sorted from 1s to 9s - if we first split and then shuffle, we end up with the training set containing only 1, ..., 8 instances  and the test set containing only 9s.
Wouldn't we want both sets to have proportionally the same amount of every instances?

Hi @mnylen ,
Great question! In general, you are absolutely right, it's better to shuffle first and then split, but in the case of MNIST, it is a standard dataset that has already been split into a training set and a test test, and if you want your results to be comparable to other people's results, it's best to use the same training/test set as everyone else. The first 60,000 instances are the training set (with about 6,000 images of each digit, indeed ordered from 0s to 9s), and the last 10,000 are the test set (with about 1,000 images per digit, and again ordered from 0s to 9s).
Note that TensorFlow's input_data() function to load MNIST actually returns a Datasets object that gives you easy access to the training set and the test set, already split and shuffled for you.  Note that it uses 5,000 instances from the training set as the validation set.
Hope this helps!

Closing this issue, please comment if you need any further explanations.


[FLINK-6665] Pass a ScheduledExecutorService to the RestartStrategy

… avoid any sleeps.
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the How To Contribute guide.
In addition to going through the list, please provide a meaningful description of your changes.

[ ] General

The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
The pull request addresses only one issue
Each commit in the PR has a meaningful commit message (including the JIRA id)

[ ] Documentation

Documentation has been added for new functionality
Old documentation affected by the pull request has been updated
JavaDoc for public methods has been added

[ ] Tests & Build

Functionality added by the pull request is covered by tests
mvn clean verify has been executed successfully locally or a Travis build has passed

@tillrohrmann Thank you for your review, I have remove restart(ExecutionGraph executionGraph), and use ScheduledExecutor instead of ScheduledExecutorService you metioned. Thanks

We have to also consider anther aspect: Changing this from a synchronous call to an asynchronous callback makes new types of races possible, against which we need to guard.
We need to make sure that the restart() call cannot succeed if there was another cycle of failure that goes into RESTARTING.
I would suggest to address https://issues.apache.org/jira/browse/FLINK-6667 first, so that the callback can check that the globalModVersion is unchanged upon restart. Then there is no anger in moving this to an asynchronous callback.

@StephanEwen Thank you for your suggestion, I think you raise an important problem about this issue. I agree with you, I will pick FLINK-6667 and try to fix it before this PR. Thanks :) cc @tillrohrmann

I have taken and adapted the code for the patch in #4364

Thank you for the contribution!
The code form this PR has been improved and merged in this commit: 65400bd0a0f6a3bdd3e0bad05e2179534eaf6e9e
If you agree, could you close this PR?

@StephanEwen Thank you for your merging :)