matmul with large matrices fails with float16, but succeeds with float32

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
Yes. see below

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Linux Ubuntu 16.04

TensorFlow installed from (source or binary):
This bug occurs with every version of GPU tensorflow I've tried or built, including several binaries and currently building from source.

TensorFlow version (use command below):
v1.12.0-rc0-2836-gd63e3ea

Python version:
2.7

Bazel version (if compiling from source):
0.19.1

GCC/Compiler version (if compiling from source):
5.4.0

CUDA/cuDNN version:
Currently building from source with CUDA 10.0 and cuDNN 7.4, but previously the bug occurred with binaries using CUDA 9.0 and cuDNN 7.4 and cuDNN 7.0

GPU model and memory:
GeForce GTX 1070 8GB

Exact command to reproduce:
Run the following python script:

import numpy as np
import tensorflow as tf

xvar = tf.placeholder (tf.float16, name="x",shape=[None,2])

weight_init = tf.uniform_unit_scaling_initializer (factor=1,dtype=tf.float16)
W = tf.get_variable ("W",[2,2],initializer=weight_init,dtype=tf.float16)

output = tf.matmul (xvar,W)

init = tf.global_variables_initializer ()

with tf.Session () as sess:
    sess.run (init)
    o = sess.run (output, feed_dict = {xvar:np.zeros ((8500000,2),dtype=np.float16)})

Describe the problem
Matmul fails with large float16 matrices. If you reduce the size of the matrix to 8e6, the code runs. If you keep the size of the matrix the same, but change all the data types to float32, it runs. The limiting factor seems to be the single largest dimension. For example, doing the matmul with sizes (8e6,100) and (100,100) works in float16
Perhaps this is not a Tensorflow issue, but an internal CUDA issue?
Source code / logs
WARNING:tensorflow:From float16_demo.py:6: __init__ (from tensorflow.python.ops.init_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.initializers.variance_scaling instead with distribution=uniform to get equivalent behavior.
WARNING:tensorflow:From /home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
2018-11-19 15:49:01.787213: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:993] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-11-19 15:49:01.787660: I tensorflow/compiler/xla/service/service.cc:149] XLA service 0x4863200 executing computations on platform CUDA. Devices:
2018-11-19 15:49:01.787676: I tensorflow/compiler/xla/service/service.cc:157]   StreamExecutor device (0): GeForce GTX 1070, Compute Capability 6.1
2018-11-19 15:49:01.787882: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7465
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 7.52GiB
2018-11-19 15:49:01.787894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2018-11-19 15:49:01.788392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-19 15:49:01.788401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2018-11-19 15:49:01.788405: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2018-11-19 15:49:01.788537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7316 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-11-19 15:49:03.017490: E tensorflow/stream_executor/cuda/cuda_blas.cc:652] failed to run cuBLAS routine cublasSgemmEx: CUBLAS_STATUS_EXECUTION_FAILED
Traceback (most recent call last):
  File "float16_demo.py", line 15, in <module>
    o = sess.run (output, feed_dict = {xvar:np.zeros ((8500000,2),dtype=np.float16)})
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Blas GEMM launch failed : a.shape=(8500000, 2), b.shape=(2, 2), m=8500000, n=2, k=2
	 [[node MatMul (defined at float16_demo.py:9) ]]

Caused by op u'MatMul', defined at:
  File "float16_demo.py", line 9, in <module>
    output = tf.matmul (xvar,W)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 2272, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4763, in mat_mul
    name=name)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 512, in new_func
    return func(*args, **kwargs)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3268, in create_op
    op_def=op_def)
  File "/home/ilia/tensorflow/venv-source-nightly-2018nov19/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1831, in __init__
    self._traceback = tf_stack.extract_stack()

InternalError (see above for traceback): Blas GEMM launch failed : a.shape=(8500000, 2), b.shape=(2, 2), m=8500000, n=2, k=2
	 [[node MatMul (defined at float16_demo.py:9) ]]

I was able to run your code snippet successfully. I suspect it's not a TensorFlow related issue, however you can try shutting down all the processes/notebooks that utilize your GPU and run the code again.

I am running this in multi-user.target, so other GPU utilization is
non-existent. Do you have the same GPU or different? Perhaps on other GPUs
the limit is higher? I understand that there may be a limit to how big
matrices can be, it just doesn't make sense for that limit to be lower for
fp16 than it is for fp32.
On Mon, Nov 26, 2018, 7:17 PM ymodak<EMAIL_ADDRESS>wrote:

I was able to run your code snippet successfully. I suspect it's not a
TensorFlow related issue, however you can try shutting down all the
processes/notebooks that utilize your GPU and run the code again.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/23861#issuecomment-441849075,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYkCWqbB3uJe2p33kKdyomI2cP4goJJ0ks5uzISJgaJpZM4Yp5CY
.

Could it be because of a change in the float type in Python 2.7 vs Python 3.6? I personally use Python 3.6 and it works with no problem... I know they tweaked some little details with floats in the newer version, but I think it's just related to int division (now giving floats instead of rounded integers), so it should not have anything to do with it. Still, that could be something to try out for you!

/CC @chsigg, can you take a look?

Tried on python3 with the stable 1.13 binary release and CUDA 10.0, identical behavior

Identical behavior on python3 TF 2.0 alpha with the following code:
import numpy as np
import tensorflow as tf

@tf.function
def myMatMul (x,W):
    return tf.matmul (x,W)

W = np.ones ((2,2), dtype=np.float16)
x = np.zeros ((8500000,2), dtype=np.float16)

output = myMatMul (x,W)

As before, reducing the large dimension to 8e6 results in a successful computation, or keeping it at 8.5e6 and changing the data type to np.float32

@ilia-nikiforov Is this still an issue for you? I ran your code in colab with GPU and don't see any issue. I guess your GPU may had less memory when compared to colab GPU.
There were lot of performance improvement over the time and TF2.3 and tf-nightly has some performance upgrades. Thanks!

I am closing this issue as this was resolved in tf-nightly. Please feel free to reopen the issue If i am mistaken. Thanks!


fix: use the lineWidth option for line breaking in flow collections

This should address issues #288 and #507.

I also amended the commit to include two testcases, with one that wraps and another test that does not.


Query user's entries by sample name

From Jim Bowring

We would like to be able to query a user's entries by sample name (not
IGSN) to check for possible duplications - the API does not show this
option - does it exist?
Thanks, Jim

REST API for retrieving IGSN according user code and sample name is implemented.

Tested the following using  usercode = IESER
Successfully able to search by user code and narrow down by name.  Used name
"SMA1958-010" to find one sample.  "Green Grossular 1" returns three samples (as expected).
Searching for "Green Grossular" returns zero results (partial name).
Consideration - Can there be a way to search for partial matches? (this might be too difficult or outside scope of tool).
Failed aspects
When 'hideprivate' is set to 1, it should not display private samples.  If I do a search for sample name="Green Grossular 1" and user code 'IESER', it returns "IESER001M",  "IESER001J", "IESER001R".  "IESER001R" has a release date of 2022-06-01 and should not be visible.
Tested 'hideprivate'=1, limit =50, and user code="IESER'.  Results include samples which should be hidden "IESER000V".

Tested the following using usercode = IESER
Successfully able to search by user code and narrow down by name. Used name
"SMA1958-010" to find one sample. "Green Grossular 1" returns three samples (as expected).
Searching for "Green Grossular" returns zero results (partial name).
Consideration - Can there be a way to search for partial matches? (this might be too difficult or outside scope of tool).
Failed aspects
When 'hideprivate' is set to 1, it should not display private samples. If I do a search for sample name="Green Grossular 1" and user code 'IESER', it returns "IESER001M", "IESER001J", "IESER001R". "IESER001R" has a release date of 2022-06-01 and should not be visible.
Tested 'hideprivate'=1, limit =50, and user code="IESER'. Results include samples which should be hidden "IESER000V".

I thought about this. 'IGSN' itself is always public regardless of the private or public flag. 'private' only refers to the metadata. Let me see if I can hide those IGSNs.


Getting errors through take_while! or similar loop

I have something that looks like this:
many_till!(complex_subparser,
           simple_end_marker)

What I was hoping to get is something like the following pseudocode:
let mut result = vec![];
while(simple_end_marker is not matching) {
    result.append(try!(complex_subparser))
}
result

That is, as long as the simple_end_marker does not match, I expect the complex_subparser to match. If it doesn't, I want the error from it.  Currently, if both the simple_end_marker and the complex_subparser fails, all I get is a ManyTill error.
I'm using the verbose-errors feature.
My actual code, for context: https://github.com/kaj/ructe/blob/feature/improve-error-handling/src/template.rs#L63

maybe this could work
separated_nonempty_list!(simple_end_marker, complex_subparser)

@juggle-tux , no, that would require the simple_end_marker between each complex_subparser instead of a single simple_end_marker after them all.  Hm.  But maybe I can create a non-consuming negated version of the simple_end_marker?  Since my end markers are generally just eof or the '}' character, I guess that should be easy enogh.
Thanks for the suggestion, @juggle-tux !  :-)


How to efficiently test the huge inherited Dataset API

What is your issue?
xarray.Dataset has a ginormous API, and eventually the entire thing should also be available on DataTree. However we probably still need to test this copied API, because the majority of it will be altered via the @map_over_subtree decorator. How can we do that without also copying thousands of lines of tests from xarray.tests?

@jhamman made a good suggestion today: reroute as much as possible internally through a node._to_temp_dataset() private method, similarly to what we do in xarray.DataArray.
This would mean that every method which is intended to act the same as it does on Dataset will essentially be guaranteed to act the same, because internally it calls the corresponding Dataset method. This could make the number of tests required for #41 much lower.

I think this issue is outdated - after #9585 we no longer blindly inherit methods, and the plan is to add tests for them as we re-add them.


correct var is pattern

Fixes #11804

@Thraka It gets a lot more useful in C# 8 when you have the property patterns: you can assign a variable to any property of an object.

It’s also useful in cases like this:
bool Test(SomeType x)
    => x.CalculateSomething() is var y && y.Value1 < y.Value2;


add a callback on upload progress

Give the options to have a callback that gets ping on upload progress

sure, within ops is cool as well


[DO NOT MERGE] Fix no error when the font collection is null and the font doesn't exist

This is experimental rework of https://github.com/mono/libgdiplus/pull/220. It doesn't work on OS X, but I want to check if the font substitution makes any difference on Linux.

Seems like the font substitution doesn't solve anything even on Linux. 👎


must-gather: add ceph config for each osd in the cluster

Signed-off-by: RAJAT SINGH<EMAIL_ADDRESS>

everything works as expected but my cluster destroyed at the very last moment, I will test this once again just do double check and remove the WIP label

It works and ready to merge.

If you believe this is ready, please remove the [WIP] tag from the PR.
/assign @crombus

/cherry-pick release-4.8


Don't Merge! PR real time test for target branch full build

Docs Build status updates of commit ea8566f:
:clock10: Pending: waiting for processors (7 builds ahead of you)
:warning: Docs Build is busy, currently there are 7 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit ea8566f:
:clock10: Full build: the average full build time is 5 min(s) 14 sec(s), this is based on the last 30 build(s) for this repository.

Docs Build status updates of commit ea8566f:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-conceptual/testing.md
:warning:Warning
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)
Details

docs-ref-conceptual/testing.md

[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.
[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.
[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Documentation] Write up review and customize letter function

Add this to docs/comment in functions
https://github.com/ProgramEquity/amplify/discussions/62#discussioncomment-2146534

Can you provide more details about the problem we're trying to solve and the desired outcomes?

Hi @manishapriya94 just wanted to check in on this and see if I can get some further information on this issue.

Just updated, also good to look at PRs by @tdo95 and @sammychinedu2ky

I found this PR . I'm not seeing any code level documentation, are you looking for code level documentation or is there an external doc that needs to be updated?


New feature: Related Posts

New Features

Display related posts

Minor Updates

Trivial changes in search bar

And what if the category attribute is not given?

I'd found it was far to merge; The additional updates I did:

Handle the post on which user does not specify the posting date. Also, in that case, I marked '-' on date field (in both pagination.html and post.html).
Fix the width of #related-post as 100%

@cvlian
Honestly I have bunch of thing to do about style in mobile layout. So, let me handle them later all at once.


[BUG] benchmark_runner.py calls to gc.collect throw "RuntimeError: IOLoop is closed"

Running gc.collect calls on all workers via client.run results in a crash:
Using default arguments
Connected to ucx://<IP_ADDRESS>:8786
Pure Dask Queries
01
/home/rgelhausen/conda/envs/rapids-tpcx-bb/lib/python3.7/site-packages/cudf/core/frame.py:1965: UserWarning: When using a sequence of booleans for `ascending`, `na_position` flag is not yet supported and defaults to treating nulls as greater than all numbers
  "When using a sequence of booleans for `ascending`, "
Traceback (most recent call last):
  File "benchmark_runner.py", line 67, in <module>
    client.run(gc.collect)
  File "/home/rgelhausen/conda/envs/rapids-tpcx-bb/lib/python3.7/site-packages/distributed/client.py", line 2487, in run
    return self.sync(self._run, function, *args, **kwargs)
  File "/home/rgelhausen/conda/envs/rapids-tpcx-bb/lib/python3.7/site-packages/distributed/client.py", line 833, in sync
    self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
  File "/home/rgelhausen/conda/envs/rapids-tpcx-bb/lib/python3.7/site-packages/distributed/utils.py", line 304, in sync
    raise RuntimeError("IOLoop is closed")
RuntimeError: IOLoop is closed

Closed by https://github.com/rapidsai/tpcx-bb/pull/48


Support for Symfony 4

#SymfonyConHackday2017

Sure, although you may have noticed :blush: that I no longer maintain this package, hope it helps.


Add command: Email and Address fields are now optional | resolves #10

Solving issue #10
Email and address of specific foods are sometimes not required to be ordered. Therefore omitting these fields should be necessary.

Adds dummy text to replace missing email and address field.
Removes assert fail test cases where email and address is missing

Pull request applied from branch instead. New PR posted, this is now a duplicate PR. Closed.


Local driver creation attempt if remote driver is not created

Hello,
We are having issues with remote driver.
If, by some reason, we cannot connect to remote driver, local driver is being created automatically.
After QAF source code analysis I found this:
https://github.com/qmetry/qaf/blob/b15a00381203cfe1b61d11bdb78ad66f3337897e/src/com/qmetry/qaf/automation/ui/UiDriverFactory.java#L248
Constructor<? extends WebDriver> constructor = of.getConstructor(Capabilities.class);
return constructor.newInstance(capabilities);

I already fixed it for myself by changing to:
Constructor<? extends WebDriver> constructor = of.getConstructor(URL.class, Capabilities.class);
return constructor.newInstance(new URL(urlStr), capabilities);

Just wanted to clarify whether it was designed like that or it's a typo?

it's by design. Did you faced any issue because of that or just posted as question?

Hi Chirag,
We have an issue that we have added a workaround for and I just wanted to clarify whether it was designed like that.
The issue is that if we work with remote server (remote.server) and if by some reason remote server is unavailable, QAF tries to create a local Appium server(which in our case we do not need). After source code analysis I found and absolutely sure that this line (please find my comment above) of code is responsible for that.
My question is: Was it designed like that? And if so, is there a way not to create a local driver if it fails to connect to remote.
You can close this issue if it was designed that way.

Yes, in case of appium/mobile if you have local appium server running by design it should connect to  appium server on <IP_ADDRESS> if it is running.
If you are using remote driver (by specifying remote in driver name) that should not connect to local server if remote server connection failure.
I am keeping this issue open to get opinion from different users so that the default behavior can be changed if required depending on users requirement.


Decide on ordering of Tx/Rx pairs in APIs

We should decide on an order, and add it to the API guidelines, then check that all APIs that return or consume a tx/rx pair use that order.

Well, obviously tx, rx is the only right way to do it :)
Btw we might want to think about this a bit more generally, there are some inconsistent things in the HAL, like:

(timer, clocks, vs clocks, timer)

embassy-net and smoltcp both seem to be using rx, tx.

In our internal meeting we decided to try and be consistent with the ecosystem, we will use rx, tx.

Well, obviously tx, rx is the only right way to do it :)
Btw we might want to think about this a bit more generally, there are some inconsistent things in the HAL, like:
 (timer, clocks, vs clocks, timer)

I guess after https://github.com/esp-rs/esp-hal/pull/1999 clocks/timer is no longer an issue


name?

how to get the name (and maybe size) of the torrent when a hash is found?

Use https://github.com/whtsky/mala


[WIP] Tests: replace hard coded (e.g. /tmp) paths

On macOS and Linux system you always have /tmp. On Windows you don't.
This PR fixes all the tests that assumed these and similar paths to also work on Windows.

          1 Warning

:warning:
Pull Request is Work in Progress

  Generated by :no_entry_sign: Danger


Enhancement: Provide mqtt information if connection to sensor failed or failed to retrieve data fro sensor.

Hello,
Scenario:
MiFlora work for some time and returns data but battery goes off and data stop to flow.
Openhab display value last returned over mqtt. User newer knows that sensor is offline.
I would like to propose enhancement:
In the event of lost connectivity to the sensor or failure to retrieve data, deamon could returned either "N/A" or "connection lost" value in all measurements for lost sensor.
Do you think it would be possible ?
Thanks
Jarek

It is possible to add topic "timestamp" for device. And in openhab write a rule that checks last timestamp...
This is already done for some mqtt conventions in daemon.

Adding a timestamp is one way to go but then as you have mentioned, we would need another thing (rule) in openhab to check timestemp, while adding N/A or connection lost to a toppic would give user general idea of what have happened without additional rules in openhab.

I have rethink  this:

adding a timestamp would be great to get information when data where gathered
then on top of that - updating toppic with N/A or connection lost if timer would not be available.

@wojciej I have something like this kinda-sorta working in my branch https://github.com/insertjokehere/miflora-mqtt-daemon/tree/device_registry - it sets an availability topic in the home assistant discovery message. Currently all sensors that the daemon have discovered are marked as "available" if the daemon is up and connected; I've not implemented any logic around detecting if a specific sensor hasn't been contacted in a while


Program exits when framing is incorrect.

If we set the option for framing with \n, framing: {Nerves.UART.Framing.Line, separator: "\n"}, but send no new line we get this error:
** (exit) exited in: GenServer.call(#PID<0.166.0>, {:read, 3000}, 3100)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:834: GenServer.call/3
    (hello_uart) lib/hello_uart.ex:24: HelloUart.uart_loop/1

The example can be found in: https://github.com/DisruptiveAngels/uart-example/blob/d990251ce6fd4992f91c3fee490253e0f1a8d06d/lib/hello_uart.ex

So I was debugging the error more and noticed I was using the version 1.0.1 when I updated it started working as expected


Can't edit preferences

    First time installing and I get an error everytime I try to edit my preferences saying "module not installed"
    -
    Hyper 1.4.8
    Electron 1.7.8
    win32 x64 10.0.16299

@PlanetVaster can provide more information 🙏

@PlanetVaster It is a very frequent problem in the windows version, since it tries to execute the file
.hyper.js.  in the canary version it is configured to open notepad or any text editor that has been previously configured

@PlanetVaster  You can change to the Canarian channel by adding this line of code to the .hyper.js file: updateChanel:  'canary'

Another solution would be to change the default action for .js-files in Windows, to open in your favorite JavaScript-editor. When I configured so that .js-files opened in Visual Studio Code, this problem went away.

Hey there, please try again with the new Hyper v2 release and open a new issue if the problem stills exists. Thank you!


integer 32/64 bit files have the wrong range

The numbers in the files integers-32 have the wrong range, some don't fit into int. And there are no small numbers (like 1 or 2).

"integers-64 should not exclude the 2^32 range because that's stupid"

This is no longer relevant, as we have changed the test files


Seaborn Facetgrids do not show

Describe the bug
title
Environment
{
"marimo": "0.6.10",
"OS": "Linux",
"OS Version": "6.5.3-coreweave",
"Processor": "x86_64",
"Python Version": "3.10.0",
"Binaries": {
"Browser": "--",
"Node": "v20.9.0"
},
"Requirements": {
"click": "8.1.7",
"importlib-resources": "6.3.0",
"jedi": "0.19.1",
"markdown": "3.6",
"pymdown-extensions": "10.8.1",
"pygments": "2.17.2",
"tomlkit": "0.12.5",
"uvicorn": "0.28.0",
"starlette": "0.36.3",
"websocket": "missing",
"typing-extensions": "4.10.0",
"black": "24.4.2"
}
}
Code to reproduce
tips = sns.load_dataset("tips")
g = sns.FacetGrid(tips, col="time",  row="sex")
g.map(sns.scatterplot, "total_bill", "tip")
g.figure

What is the expected?
This is what i see

https://marimo.app/l/3dhfag

Okay super weird! I think it was a different bug. I'm using the VSCode extension via Remote-SSH, and sometimes I have to go through, stop the server, and remove all port forwards, or else the app will bug out. It sometimes displays an empty screen, and I guess in this case the plots did not show. Sorry about that. Maybe it's a rogue marimo server that starts on the same port or some other nonsense that's happening.


[Feat Request] SRCREV should be derived from ${PV}

The cleanest way to doing this when a vendor is tagging revision numbers, even if there's branch info is to use the value in PV to derive the results.
Suggest that you consider doing something similar to this:
# Compute branch info from ${PV} as Base PV...
BPV = "${@'.'.join(d.getVar('PV').split('.')[0:2])}"
# Since they tag off of something resembling ${PV}, use it.
SRCREV = "v${PV}"
SRC_URI = " \
    git://github.com/tensorflow/tensorflow.git;branch=r${BPV} \
As you can see, this programmatically cracks off the first two digits every time ${PV} is supplied and then lets you leverage TensorFlow's tagging and branching to make a generic for this.  I arrived here as a result of my (Sadly, failed) work on trying to make one unifying solution for all.  It's not so much that it's un-doable...I just don't do anything with TF itself except at work- and I needed CUDA 10.0 support and 2.4.x requires CUDA 10.1 or later with Tensor Cores support.  When they roll out new versions, you need only handle the patch cases as needed and validate it's not broken and stomp out a version without so much as lookng at commit hashes, etc.
I may have a bit more to add as a pull request later on.  There probably should be something for supporting the C/C++ bindings, etc. for at least the latest (2.4.0, master...) as it's likely to be as useful as this is for people on targets.

The cleanest way to doing this when a vendor is tagging revision numbers, even if there's branch info is to use the value in PV to derive the results.

Thank you for the very useful information !
With the release of v2.4.1 for TensorFlow, I was able to work fine with PV values.
https://github.com/NobuoTsukamoto/meta-tensorflow-lite/blob/b6d4f29f301d47062d0b15022294aaf226c42187/recipes-framework/tensorflow-lite/python3-tensorflow-lite_2.4.1.bb

The cleanest way to doing this when a vendor is tagging revision numbers, even if there's branch info is to use the value in PV to derive the results.

Thank you for the very useful information !
With the release of v2.4.1 for TensorFlow, I was able to work fine with PV values.
https://github.com/NobuoTsukamoto/meta-tensorflow-lite/blob/b6d4f29f301d47062d0b15022294aaf226c42187/recipes-framework/tensorflow-lite/python3-tensorflow-lite_2.4.1.bb

I needed CUDA 10.0 support and 2.4.x requires CUDA 10.1 or later with Tensor Cores support.

Could you tell us a little more detail?
Do you need the TensorFlow with CUDA Yocto recipe? (Is it for NVIDIA JETSON? or x86 PC?)
If you need anything in this repository, please let us know the details.
I'm sorry if I misunderstood.
Thanks.

I needed CUDA 10.0 support and 2.4.x requires CUDA 10.1 or later with Tensor Cores support.

Could you tell us a little more detail?
Do you need the TensorFlow with CUDA Yocto recipe? (Is it for NVIDIA JETSON? or x86 PC?)
If you need anything in this repository, please let us know the details.
I'm sorry if I misunderstood.
Thanks.


Add Time Mocking, No Deps

I noticed that adding deps may not be appreciated so this is an alternate to #327
It adds zero deps, with the tradeoff that there's no convenient way to perform mocked-time tests within the package.  (Seems like it might be useful for things like #321 or #325 but that's not my call.)
I did leave the schedule behavior test in there, behind a build tag, so you can try it out with go test --tags=clock if you like.  I can also remove that file, but I thought it gave a good example of how to perform mocked-time tests, including the adapter needed to wire in the mixer/clock.

Any thoughts, @robfig ?


Lights: only on/off/dimming?

Is it somehow possible also to add colour changes to the API? That would be so useful! Also, milights can also change the white balance, is this possible? I guess it might not since I'm not sure whether this is possible with the hue lights.
I now have my MiLight bulbs connected to the bridge, and made a small API from the node-milight-promise that allows to set colours/disco microphone lights/light alarms and so on (I'm thinking of publishing it opensource once it is more refined and easy to setup).
The ha-bridge then makes http requests to my API which controls the lights, and it works very well!

This is already referenced in #327


New task added

program for Text Wrapping in python

#hacktoberfest2020 contribution

The functionality of this code is not clear


feat: Automate location extraction and english translation

Summary:
Closes #618, which includes adding translations for location names (country, subdivision, and municipality) to the FeedSearch materialized view and improving the functionality for geocoding locations from GTFS feeds.
Changes include:

Database Schema Changes:

Introduced a new country column to the Location table.
Introduced a new Translation table with type, language_code, key, and value columns to store translations for various location elements.
Modified the FeedSearch materialized view to incorporate translations for country, subdivision_name, and municipality into the searchable document field.

Backend Logic:

Created the GeocodedLocation class to include methods for handling location extraction and translations.
Implemented enhancements to the update_location function to integrate location translations into the database.

Testing:

Added unit tests for the new translation functionality.

Expected behavior:
The system should now support searching feeds using English-translated names for countries, subdivisions, and municipalities. When a feed has associated locations with translations available in the Translation table, these translations will be included in the search index, enabling users to find feeds using either the original or translated location names. This change aims to improve the searchability of feeds for users who might use different languages.
Feed locations are also now automatically extracted from reverse geolocating using five points from the dataset: the extreme points (the ones with extreme lat/lon which give four points but can be less if one point represents two extremes) and the point in stops.txt closest to the center of the bounding box. Additional points are randomly selected to complete the count of five. The decision on the subdivision or municipality is based on majority voting. If there's no majority at the subdivision level, the country level is included, and multiple countries are included if necessary.
Testing tips:
Use the PR preview URL to search for locations. Example tests:

Espana vs España vs. Spain
Cairo vs القاهرة
日本 vs Japan
ประเทศไทย vs Thailand
Bayern vs Bavaria
Aotearoa vs New Zealand

Please make sure these boxes are checked before submitting your pull request - thanks!

[x] Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
[x] Add or update any needed documentation to the repo
[x] Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
[x] Linked all relevant issues
[ ] Include screenshot(s) showing how this pull request works and fixes the issue(s)

Note that the integration tests are currently expected to fail due to the database changes. The corresponding API updates should be covered in issues #622 and #623.

@cka-y From a QA perspective, this looks great! (Solves the core problem we were trying to fix, where people can input full country name in either original language or English, accents or no accents). Much better user behaviour that I look forward to telling our usability testers we incorporated!
Couple outstanding questions:
I see a few examples of countries that when I include the original language name vs. the English, I get a different number of results:

Hungary - 4 results vs. Magyarország - 5 results
Spain - 34 results vs. Espana (34 results)
Bavaria - 7 results vs. Bayern - 9 results

Is this because the location data in the search UI is inaccurate? Or another reason? In these cases, it looks like the feeds are all based in the same country, so it's not because of text in the feed name or transit provider that matches where the location does not.

@emmambd
Bavaria vs. Bayern (difference in the red square):

It appears that the two feeds not returned when searching for "Bavaria" (the translation) are due to the following reasons:

First Feed: It lacks a valid dataset because the producer URL returns an HTML file instead of the expected data.

Second Feed: This is a very large feed, and we do not have sufficient memory in the cloud function to process it, specifically to open the stops and parse it for extracting either a bounding box or location. Currently, the memory allocation is 8Gi, but we can consider increasing it if we find it valuable, albeit at a higher cost.

In the first element, the feed is returned when searching for "Bayern" despite "Bayern" not appearing in the feed. This discrepancy is likely due to string transformation processes during the search query. For the feeds related to Spain, there may be a similar reason.
Also, searching for "Spain" returns 34 results, while "España" returns 31. The extracted location is always in the regional language (in this case, "España"), suggesting that the difference is not due to location mismatches but rather the search process (otherwise España should return more results than Spain).

@cka-y - Got it. If I understand this correctly, basically this is occurring either due to 1) issues with parsing the location because of the feed, whether it be missing data or size or 2) changes we still need to make to the API side?
I'm fine living with this as is for now.

@emmambd
I've done a deeper dive to understand the differences in search results. For Bavaria vs Bayern, the issue is with the feed data: either the file is too large to process or the dataset is missing because the producer URL does not return a valid zip file. However, for Spain vs Espana, I discovered that our search was only using the first element of the location data. This approach was initially fine because we typically had only one location per feed.
The problem arose with feeds containing multiple locations (e.g., feeds covering several countries including Espana). If Espana wasn't the first country listed, it would not be included in the search document, leading to discrepancies. Additionally, the English translation (Spain) was consistently appearing as the first element in the translations, further skewing the results.
I've now fixed this issue! Both Spain and Espana queries return the correct number of feeds, which is 35. I've double-checked this with the database to confirm the accuracy.

@davidgamez @cka-y Does that mean that merging this is blocked until #622 is done?

@davidgamez @cka-y Does that mean that merging this is blocked until #622 is done?

Yes, this is why my suggestion is to comment/ignore the code until the follow-up issue is completed.

@davidgamez @emmambd I could comment the location filtering integration tests as part of this PR, merge and then generate/uncomment tests as part of #622. Thoughts?


doc: fix logo size to full width of container

For some reason the new logo defaults to 0x0 px in the sphinx theme.

For some reason the new logo defaults to 0x0 px in the sphinx theme.

This might be an issue with the SVG itself. In https://github.com/simbricks/simbricks/pull/122, GitHub also showed a size of 0x0 in the changes view. Do we want to fix that instead?

You are welcome to look into it if you like. But having the size depend on the browser-interpretation of an SVG in the rendering seems a bit dangerous in any case.

I just tested your changes in Firefox and they don't actually work for me. Instead, I now looked into what has changed on the SVG side compared to the old logo. Turns out that we are no longer setting the width and height attributes on the top-level svg tag. I'll add a commit to fix that.

We also no longer need the fix in the CSS. I tested both Chromium and Firefox for this.


Convert a 12-bit ADC reading to a 15-bit data.

The adcIn.read() generates 12bit data. I modify the code to convert a 12-bit ADC reading to a 15-bit data by multiplying by 8. It can enables us to get more big output sound and more better dynamic range. According to my test, just converting 12-bit to 16-bit by multiplying by 16 causes noise when over-ranging. The one bit may be remaining for filtering.

Thanks, this sound really interesting.
I'm working in the new version, and
here are the results of the latest developments of the Raspberry Pico Pi based DSP filter.  In the attached videos there are some tests on the 3 filters made and the simple noise reduction.  This example implements the signal acquisition with the internal ADC and the reproduction takes place digitally via I2S protocol on MAX 98357A amplifier with a remarkable result in terms of audio power and reproduction fidelity, only using a few components and these two modules it is possible drive both medium impedance headphones and 4 to 8 ohm speakers and deliver up to 3 W, as you can hear in the videos ( no power amplifier was used, the audio signal comes from the Max ).
As soon as possible all the source code Will be shared on GitHub, just some small modifications and I Will share it as new project with all new filters, and of possible can repeat the tests on the filters.
I create new videos, please check It and give me your impressions.
Videos:
CW https://youtu.be/CpewdCyXoGU
SSB1 https://youtu.be/-HOMVBYqSmQ
SSB2 https://youtu.be/sMhqdDxAlq8
--
Inviato da Libero Mail

I watched the video and it's really awesome! I bought both PCM5102A and MAX98357A. I will conduct the tests of new DSP filters.

Great ! In the afternoon I Will commit the new sources.
--
Inviato da Libero Mail

The new project SimpleAudioFilter07 Is available on GitHub.
Bye.
Ik8yfw
Inviato da Libero Mail


21.04 does not exist in templates anymore, need to change to 22.04

$OSVERSION 21.04 results in failure when trying to set the TEMPLATE array. Changing OSVERSION to 22.04 fixes the issue.

Actually appears that 22.04 isn't available, either. But 21.04 isn't working.  22.04 passes the error in the array but then you can't get it.


Add functionality to handle HGVS strings with the reference sequence specified (see details)

home page. search for "NM_007294.3(BRCA1):c.-20+11C>T". In the
javascript console I get the exception:

Error: Invariant Violation: Missing "c" parameter for path
"/variants?search=NM_007294.3(BRCA1):c.-20+11C>T"
(note: specifying the reference sequence is what seems to throw off the search)

The javascript error is a separate issue from correctly handling the reference id. I believe the javascript error has been resolved, but the feature request remains. So, not a bug.


FALCON-1853 : Add spark process workflow builder

Following is the pull request that will perform the integration to execute the Spark application through Falcon. Please review.
XSD changes for integration are already committed through pull request #89 .

I have verified the Falcon build using this pull request on secure cluster and it works successfully. Spark process entity runs with no issues.

Thanks @vrangan for reviewing the pull request. Pull request has been updated after incorporating the review comments.

[INFO] Method org.apache.falcon.oozie.process.SparkProcessWorkflowBuilder.checkDataFeedsAsArgument(Process) seems to be useless [org.apache.falcon.oozie.process.SparkProcessWorkflowBuilder] At SparkProcessWorkflowBuilder.java:[line 144] UC_USELESS_VOID_METHOD

Can you fix this and make sure you run complete fix and test with the latest master and use
-Ptest-patch,hivedr,hdfs-snapshot-mirroring,adf to make sure we have all components are covered

@vrangan I have fixed the issue and now build is successful. Also I have tested the patch it is working.

I have checked the jenkins build failure and they are not related to this patch.

@peeyushb   - The patch no longer applies.  Can you please rebase and reupload

It looks like multicommit pull requests don't apply if we get a patch - works ok if we do the final diff

Thanks @pavankumar526  for reviewing the pull request. I have incorporated the review comments.

+1
Thanks @peeyushb  for diligently working on this and responding to comments.
Can you check the one failure in jenkins runs - It does not look related - just review and comment

Thanks @vrangan for reviewing the patch in pull request.
I have done a check for failure in Jenkins and it does not look related.

+1 patch looks good. @peeyushb It will be good to have one integration test using Falcon Unit, so that it will cover end to end. You can do it as part of other jira.

Will merge this patch in some time, if there are no further comments


Cannot make any get requests

When I try to make a get request (for example HTTPotion.get("http://www.google.com")) I get the following error / stacktrace:
** (ArgumentError) argument error
 stacktrace:
   :ets.lookup(:ibrowse_lb, {'www.google.nl', 80})
   src/ibrowse.erl:317: :ibrowse.send_req/6
   deps/httpotion/lib/httpotion.ex:123: HTTPotion.request/5
   lib/octolixir/parser.ex:11: Octolixir.Parser.parse_gist/1
   test/parser_test.exs:14: ParserTest."test parse_gist should return succesfully with a valid id"/1

Anyone knows what's going on?

What do you think about mentioning HTTPotion.start in the README? And incidentally, where would you put HTTPotion.start, in, say, a phoenix app?

@gkop It should be mentioned! I think it even was mentioned at some point...?
I guess it should go into app deps in mix.exs:
  def application do
    [mod: {HelloPhoenix, []},
      applications: [:phoenix, :cowboy, :logger, :httpotion]]
  end

Please test it and tell me whether it works!

@myfreeweb yes, it works, thank you!

I'm getting error even after running:
mix deps.get
iex -S mix

The error is:
** (ArgumentError) argument error
    (stdlib) :ets.lookup(:ibrowse_lb, {'www.google.com', 80})
             src/ibrowse.erl:328: :ibrowse.send_req/6
             lib/httpotion.ex:295: HTTPotion.request/3

@collegeimprovements You need to start httpotion, either with HTTPotion.start as mentioned by @kwijibo or if you use phoenix, see @myfreeweb 's comment.

It solved my issue. Thanks.


Fix extension_dir configuration

Description
The extension_dir value in the default php.ini files has changed from
; extension_dir = "ext" to ;extension_dir = "ext". The replace regex
now matches both formats.
Motivation and Context
The current method does not actually configure the extension_dir value in PHP 7.3.0 upwards
How Has this Been Tested?
Tested locally in au
Screenshot (if appropriate, usually isn't needed):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Migrated package (a package has been migrated from another repository)

Checklist:

[x] My code follows the code style of this repository.
[ ] My change requires a change to documentation (this usually means the notes in the description of a package).
[ ] I have updated the documentation accordingly (this usually means the notes in the description of a package).
[x] All files are up to date with the latest Contributing Guidelines
[x] The added/modified package passed install/uninstall in the chocolatey test environment.
[x] The changes only affect a single package (not including meta package).

:x: Package verification failed, please review the Appveyor Logs and the provided Artifacts before requesting a human reviewer to take a look.

This is failing because php cannot be uninstalled:
2019-12-06 19:08:41,458 2952 [DEBUG] - Running 'C:\ProgramData\chocolatey\lib\php\tools\chocolateyUninstall.ps1'
2019-12-06 19:08:41,458 2952 [DEBUG] - Checking for uninstall text document in C:\ProgramData\chocolatey\lib\php\tools\..
2019-12-06 19:08:41,505 2952 [DEBUG] - Running Uninstall-ChocolateyZipPackage -packageName 'php' -zipFileName 'php-7.4.0-nts-Win32-vc15-x64.zip'
2019-12-06 19:08:41,521 2952 [ERROR] - ERROR: Cannot bind argument to parameter 'Path' because it is null.

@johnstevenson your changes have been merged, thanks for your contribution 👍
I will need to look into the uninstall issue before we can push a new version with these changes.


Allow wrapped functions to inherit original "this" context

PURPOSE
Allow the "this" argument to be inherited when calling a function wrapped by a limiter.  This allows more natural usage of limits when applied to modules or class methods, e.g.:
var object = {
    value: "Hello World",
    hello: limit.wrap(function () {
        return this.value;
    })
};

// When calling `object.hello()`, `this` = `object`
object.hello().then(console.log);

DETAILS
The majority of ways you can schedule a task can't inherit this, because we don't have that context.  That includes schedule, submit, and the withOptions variant of a wrapped function.  But, when calling a wrapped function directly, we do have that context, so we might as well inherit it.
This is especially useful for wrapping class (prototype) methods, so I included a test to make that use case clear.
(There may also be a better/more appropriate place to make this fix, this is just my suggestion!)

Hmm, worked on my machine :).  I'll see if I can make heads or tails of the failures in the Travis build.

Travis has different latencies than local development and sometimes needs to be re-run. I'm slowly tweaking the tests to make them more resilient.
Very nice tests! I like the feature a lot. I think it would also work with withOptions, same changes: -> instead of => and fn.bind(this)? That would minimize the inconsistencies between the different ways to add a job.

I'd be interested to hear your ideas on how to improve the Readme when it comes to contexts. In my experience, most JS devs simply do not understand how it works: prototypes, classes, function vs =>, etc. I recently added a small paragraph to the 'Gotchas' section due to how frequent those mistakes are.

I don't think it'll work for options unfortunately, only because of the calling syntax:
object.fn.withOptions({ });

In this case wrapped.withOptions, even if we did bind, would receive fn (a prototype method) and not object.  Not sure if there is a good way around that one.
Good points on readme, I'll think about it!

I'll try it out when I have time this weekend, I think I can use some apply() magic to make it work.

I think you have different use cases here.  The use case where options actually matter is probably the one where you are using bottleneck like it is a job scheduler, maybe even with redis, giving jobs IDs and priorities, etc.
With my use case, which is more just "hey, I don't want to spam AWS", I can get away with something like this:
myModule.downloadFile = new Bottleneck({ maxConcurrent: 10 }).wrap((bucket, key, options) => {
    // basic AWS SDK call here
});

So the "lack" of options isn't a big deal.  If I really wanted to use options and keep context, that's when the caller has to step in:
// With bind
myModule.downloadFile.withOptions.bind(myModule)({}, bucket, key);

// Or with call
myModule.downloadFile.withOptions.call(myModule, {}, bucket, key);

One approach could be a new method that takes options for "the next call", which then resets them.  In fact, you could even support a "permanent" option set and a "next call" variant... Might look like this:
// No options
object.fn().then(...);

// Permanently set expiration for this fn
object.fn.setOptions({ expiration: 1800 });
object.fn().then(...);
object.fn().then(...);

// Set ID and Priority only for the _next_ call
object.fn.nextOptions({ priority: 1, id: uuid });
object.fn().then(...);

// This call has no id or priority.  Expiration still exists.
object.fn().then(...);

For schedule and submit, I think nothing needs to change -- those APIs already make plenty of sense when your use case is a more robust "queue" of tasks.  I think the easiest advice for executing any/all class methods there is just the arrow function:
limiter.schedule({ priority: 1 }, () => object.fn()).then(result => ...);

With my use case, which is more just "hey, I don't want to spam AWS", I can get away with something like this:
So the "lack" of options isn't a big deal. If I really wanted to use options and keep context, that's when the caller has to step in:

You're totally right, I'm convinced.

One approach could be a new method that takes options for "the next call", which then resets them. In fact, you could even support a "permanent" option set and a "next call" variant... Might look like this:

I don't think this is worth the complexity and added confusion.
I'm merging this and I'll play with it this weekend and release it soon.

This has been released as part of 2.16.2


Https code school url validation

Description of changes
Small change to the Makefile with comments that can help anyone debugging to output verbose tests or test just one when running make test
Updated the code_school.rb model to have URL validation for HTTPS. Uses Regex to accomplish this.
Added a test to ensure that it will throw an error when a HTTP address is given.
Updated the Code School Factory Faker for URL to generate HTTPS urls for mock data in testing
Issue Resolved

Fixes #436

Whoops. Didn't mean to close the un-merged PR... :confused:


Can't select daemon

I can't select daemon because there two starts with same substring.
 german$ lunchy restart syncthing
Multiple daemons found matching 'syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
Multiple daemons found matching 'syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
 german$ lunchy restart homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing

You should be able to throw a -x on there for exact matching.

I did, but got the same behavior.
german$ lunchy restart -x homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing

Can you verify your lunchy version?  That -x feature is relatively new.
lunchy -v

will show the version at the top.  It should be 0.10.x or greater.  If it
isn't try updating lunchy and see if that solves the issue.
On Mon, Aug 3, 2015 at 10:52 AM, German<EMAIL_ADDRESS>wrote:

I did, but got the same behavior.
german$ lunchy restart -x homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
Multiple daemons found matching 'homebrew.mxcl.syncthing'. You need to be more specific. Matches found are:
homebrew.mxcl.syncthing-inotify
homebrew.mxcl.syncthing
—
Reply to this email directly or view it on GitHub
https://github.com/eddiezane/lunchy/issues/62#issuecomment-127348471.

german$ lunchy -v
Lunchy 0.10.1, the friendly launchctl wrapper

Thanks.  We'll look into it.
On Mon, Aug 3, 2015 at 4:30 PM, German<EMAIL_ADDRESS>wrote:

german$ lunchy -v
Lunchy 0.10.1, the friendly launchctl wrapper
—
Reply to this email directly or view it on GitHub
https://github.com/eddiezane/lunchy/issues/62#issuecomment-127431010.


Update sbt, scripted-plugin to 1.4.7

Updates

org.scala-sbt:sbt
org.scala-sbt:scripted-plugin

from 1.4.4 to 1.4.7.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt" } ]

labels: library-update, semver-patch

Superseded by #58.


re-org http components into gobblin-http module

Move newly introduced http components from gobblin-core to gobblin-http
Decouple AsyncWriteRequestBuilder from write, making one general for both async read and write

@ibuenros @yukuai518 Please review. This will unblock kuai from working on the ambry join converter


Update pallet repository

Update to pallet commit https://github.com/integritee-network/pallets/commit/2dbe761f0bb09fcb731e5ce16b25d6fff1b9e85b (most recent that is < metadata update v14)

Confirmed it to be working with the updated worker, so I'll merge


[SR-15176] Fixit for override func that should be override var

Previous ID
SR-15176

Radar
rdar://28337871

Original Reporter
@CodaFi

Type
Improvement

Additional Detail from JIRA

Votes
0

Component/s
Compiler

Labels
Improvement, DiagnosticsQoI, StarterBug

Assignee
jiaren wang (JIRA)

Priority
Medium

md5: b17661d61829d04b65f822d78b07388a

Issue Description:
Consider
class C {
  var canBecomeFirstResponder: Bool

  init() {}
}

class D: C {
    override func canBecomeFirstResponder() -> Bool {
    }
}

We should add a special case to diagnoseGeneralOverrideFailure that tries to see if it can correct no-args FuncDecls to VarDecls and vice-versa.

Hello @AnthonyLatsis -  is this issue still open, can I give it a shot? Thanks!

You're welcome to! Make sure it still reproduces though. Want we to assign you?

I got busier and did not complete it yet, but I did make progress on this and was planning on opening the PR soon.

Let's try this!

Hey @LucianoPAlmeida @AnthonyLatsis , I managed to find the error call site:
File --> /lib/Sema/TypeCheckDeclPrimary
(Case 1: When the method is overridden instead of the property)
FD->diagnose(diag::method_does_not_override, isClassContext)
                .highlight(OA->getLocation());

(Case 2: When the property is overridden instead of the method)
VD->diagnose(diag::property_does_not_override, isClassContext)
                .highlight(OA->getLocation());

So I think fixing the actual issue remains. I will try to do it!

@AnthonyLatsis I have a little off-topic issue, I have come across this earlier also, what I did was to delete my workspace and again create it, but is there any other solution to this?
<unknown>:0: error: compiled module was created by an older version of the compiler; rebuild 'Swift' and try again:

PS: Rebuilding fixed the issue!

is there any other solution to this?

This means the binary module format version associated with the just-built compiler is different than the one that was used to serialize the Standard Library. Most likely, you pulled in the latest changes and rebuilt just the compiler (swift-frontend) rather than everything, including the Standard Library. Re-creating the workspace is almost never the answer, because the workspace or the generated Xcode projects are not responsible for building anything — the Ninja build is.

Yes, also one more thing, I am suddenly encountering problems with colouring and type hints not working properly in Xcode files while I am making changes, I tried clearing the developers derived data folder too, which could potentially re index the files, but this isn't still working...

I have always seen these functional issues come up randomly when navigating the code base in Xcode. They usually get worse the older and more outdated the generated project becomes. Re-generating the projects after every checkout is probably the right answer, but I wouldn’t know for certain: I am too lazy to always keep the generated projects in sync, and tend to fall back to searching or spawning tabs whenever these issues show up.

Ok I managed to fix the problem with some workaround, finally!

Actually in my case, it was a different issue, so actually if you need a detailed explanation, here you go:
Basically, due to some of my older instances of projects, when I built the Xcode project, randomly the old instance of workspace and project would get created, so as soon as it got created, I deleted it, I also cleared the derived data, and finally after re launching Xcode, the hang error happens, this really made me happy, as I was aware of this one, which can be easily fixed (this was the first part of fix), by deleting autosave information, next I re launched and then everything was working again...
So actually the initial error which I said (related to syntax colouring, code completion, features like jump to definition), which weren't working, I noticed from that part, when only some of the functions weren't able to jump to original file, then, I realised I must check in the Xcode's console log, where I found the bug, Xcode wasn't able to find the files itself related to some dependencies...
So overall, if anyone encounters such issues in the future, follow the below steps:

Quit Xcode
Delete Derived data from Developers folder
Re launch Xcode
If the issue persists, build the project again
Then check if in the relative path (where your workspace was created), any extra files randomly pop there, delete them.
Quit Xcode

Now, you will see this nice hanging error (which is actually a good sign, that the problem is fixed)
All you need to do now is delete AutoSave information in the Library folder, quit and re launch, you should be fine.

@AnthonyLatsis I awkwardly thought it as an issue, it wasn't an issue, actually when I was looking for type hint to get this FD->getResultInterfaceType()->getRValueType(), I typed FD->getInterfaceType(), and was expecting to show a type hint for it. Such a weird mistake I made, this made me think something was wrong with my work area, anyway.

Also, are there any docs related to these specific types of files, meaning, learning about the Types, Decl, etc, to grab the pointers for exactly what I am looking for, or do I have to manually learn and figure out?

Like in this issue, I will have to find a match for the function parameter and its declaration in the parent class, to correctly identify whether this override can be fixed...same goes for the property...

Like in this issue, I will have to find a match for the function parameter and its declaration in the parent class, to correctly identify whether this override can be fixed...same goes for the property...

There is a little bit of info here

Hey thanks a lot @LucianoPAlmeida! Looks like Swift Forums is the go for asking such questions!

There is a compiler category on the forums where you can ask more general questions about development in the compiler.
But if those are related to the issue is fine to keep the discussion here or on PRs =]

actually when I was looking for type hint

What do you mean by "type hint"? Code completion hints?

Also, are there any docs related to these specific types of files, meaning, learning about the Types, Decl, etc, to grab the pointers for exactly what I am looking for, or do I have to manually learn and figure out?

I do not know of any detailed write-ups on the AST design or the parts of semantic analysis that do not involve subtle and complex algorithms among our docs. This is where the inline documentation and comments come in handy. But we do have a list of external resources, many of which I found very helpful back in the day, with many more yet to read. Check out the article by Slava Pestov about how stuff is represented in the compiler.

Thanks! @AnthonyLatsis
I will go through the external resources!
And yeah I meant code competition and syntax colouring, I had fixed it by changing the target, it got set to a different one, maybe I had check marked by mistake while editing code or setting breakpoints due to which only files related to those dependencies would show up in the colouring and code competition, other lines(in the same file or different one) would be white, and the jump to definition and other related features wouldn't work, which I misinterpreted that it was due to some other issue in my code base.
Why was this issue closed and re opened?

I have added the notes in the diagnostic file, I hope the changes are relatively fine:
Commits

Have you tested them? Please open a pull request if you want your changes to be reviewed.

Coming soon!

I have one question, regarding the super class inherit, should we consider the case for only one base class or can it be multiple classes, like, for example, class D could inherit from two classes B and C, where the function could be from either of them...

 I just need to provide a fixit now!
Here is the output:
/Users/rajveersingh/Desktop/swift-project/build/Project-Swift-Compiler/Debug-Area.swift:15:19: error: method does not override any method from its superclass
    override func canBecomeFirstResponder() -> Bool {
    ~~~~~~~~      ^
/Users/rajveersingh/Desktop/swift-project/build/Project-Swift-Compiler/Debug-Area.swift:15:19: note: did you mean to override the property 'canBecomeFirstResponder'?
    override func canBecomeFirstResponder() -> Bool {
                  ^

I have one question, regarding the super class inherit, should we consider the case for only one base class or can it be multiple classes

Swift does not have multiple inheritance.

Oops! My bad! I misinterpreted it with C++ as it has the 'virtual' keyword to handle the diamond problem.

Hi @CodaFi. I'm working in a full-stack development cohort with a requirement for an open source contribution. I would love to try this one as my first one as I have an interest in learning Swift.

@wesleykebrown Assigned you.


Assign release privileges to<EMAIL_ADDRESS>

gem owner isodoc-i18n --add<EMAIL_ADDRESS>This is holding up the entire release

@ronaldtse I have tried to add the<EMAIL_ADDRESS>with command above, no success


My friends can't join

When I create a LAN port, I get the link and send it to my friends. However, when they try to join, they can't because Minecraft says "connection lost." I don't know what to do. Can someone help me?

I think the thing is broken, we'll have to wait for it to be fixed ig

what servers are you on ?

@acewins727 i play on my survival world, if you mean that

@acewins727
Does the mod work for you?

it doesn't since yesterday

and I meant what e4mc servers, like EU, US, OC and JP

@acewins727 same, doesn't work since yesterday and i use EU servers

I'm trying to know if it's a EU problem or a e4mc problem

the server status page says it's working well so the servers are working

https://status.e4mc.link/

but just now there have been a few orange dots which would normally mean people have been connecting

the japanese one just had a red dot show up so probably this one is working

it looks like people are using it rn but it still doesn't work for me
what country do you live in ?

maybe doing /e4mc restart over and over and trying to get a working one could work...??

I live in germany

france, so it's not a country thing either

I really think we have to wait until it works again, I've tried it with only e4mc and it doesn't work

yea, i hope it will fixed

maybe tomorrow ? I hope

I messaged the dev, a few hours ago.
It is now being worked on.

damn, I didn't find where to message her directly

skyevg — Today at 3:08 PM
hopefully it's fixed, I'm currently too tired to test
jucaTHOR — Today at 3:08 PM
ill test it
it works

ok nice

i'll test if it works


[GLUTEN-5611] [VL] Avoid trigger Spark memory listener when native memory request can be handled internally

What changes were proposed in this pull request?
(Fixes: #5611)

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_5631_time.csv
log/native_master_05_06_2024_72e787c64_time.csv
difference
percentage

q1
37.37
36.58
-0.795
97.87%

q2
23.87
23.74
-0.131
99.45%

q3
37.25
38.97
1.717
104.61%

q4
38.58
38.09
-0.493
98.72%

q5
71.25
68.56
-2.692
96.22%

q6
7.42
6.78
-0.641
91.37%

q7
84.59
83.29
-1.297
98.47%

q8
85.26
87.34
2.077
102.44%

q9
121.85
122.36
0.512
100.42%

q10
45.14
46.26
1.122
102.49%

q11
20.45
20.36
-0.093
99.55%

q12
27.86
27.48
-0.382
98.63%

q13
54.28
54.80
0.518
100.95%

q14
22.09
20.20
-1.895
91.42%

q15
29.44
30.34
0.899
103.05%

q16
14.28
15.88
1.596
111.18%

q17
102.57
101.90
-0.667
99.35%

q18
148.57
146.06
-2.515
98.31%

q19
13.49
14.65
1.167
108.66%

q20
28.66
26.35
-2.310
91.94%

q21
288.33
291.97
3.637
101.26%

q22
15.06
15.01
-0.049
99.68%

total
1317.69
1316.98
-0.713
99.95%


Split Test and Build in Different Stages

Split UT and Build Stages in different jobs of travis CI so that it's easy for user to traces what went wrong in CI

This change is

@asifdxtreme Don't know we could do that! Cool!

LGTM

@Random-Liu Thanks for the merge, yes this is a beta feature by Travis CI and very useful one


Fix DISTINCT (and other aggregates) in SQL Layer for single table queries

Single Table Queries are broken in several ways:

no aggregates
column spec not used (all cols returned)
automatic order clause on ALL columns (slow!)

will be resolved by new sql


MySql Guide for the .NET Core version.

I need help connecting to the database in the new ASP.Net Core version.
What i had done:

Changed appsettings.json data in:

"Data": {
"Default": {
"ConnectionString": "Server=localhost; Port=3306; Database=serene; Uid=root; Pwd=12345678",
"ProviderName": "MySql.Data.MySqlClient"
},
"Northwind": {
"ConnectionString": "Server=localhost; Port=3306; Database=serene; Uid=root; Pwd=12345678",
"ProviderName": "MySql.Data.MySqlClient"
}
},

Added mysql connector using: npm install 7.0.6-IR31

In project.json it added

"MySql.Data.EntityFrameworkCore": "7.0.6-IR31"

After   "dependencies": {},
When im trying to connect i get the error:

ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: providerInvariantName
Serenity.Data.DbProviderFactories.GetFactory(string providerInvariantName)

Can anyone help with this issue?

Only SqlServer (remote) is supported yet

I've updated guide for MySql:
https://volkanceylan.gitbooks.io/serenity-guide/content/mysql.html
There is no code generator support for MySql yet, though

will you be adding code generator support for MySql anytime this year?


Tooltips Feature

Simple CSS based tooltips feature implemented in 'Reading now' and 'Show people online' to show email in 'data-tooltip' instead of 'title' attribute. Usage explained in new page template.
New page template: http://take.ms/KzLjE
Reading now: http://take.ms/mo6qv
Show people online: http://take.ms/PGd27

Thanks for the pull request. There are a number of things that must be fixed before I can merge it to master.

npm start displays a warning:

./src/components/person/Person.js
  Line 6:  Links must not point to "#". Use a more descriptive href or use a button instead  jsx-a11y/href-no-hash

Please fix this warning, possibly by not using <a> tags to display a tooltip. Do not use <a> tags if you do not link to other pages.

Tooltips work differently for the same text:

Please do not use vendor prefixes in the CSS code (see: /src/css/tooltips.css).

@bolchowka thanks for checking, sorry for the issues

npm start is now free of any warnings, it was a good point - I moved <a> elements to <span>

The issue here was word-wrap: break-word inherited from content--markdown css class. I added it to all tooltips (because it's a good idea to break words there) + added a new clas tooltip-auto which contains width: auto for long words (like emails)

removed all vendor prefixes

@bolchowka all issues had been fixed - can you review the request again?

@bolchowka all issues had been fixed - can you review the request again?

@quarties just merged to master. Thanks for contribution!


Wrong realization of TypedArray's toLocaleString()

Executing following code in ch 1.10 :
Uint8Array.prototype.toLocaleString.call(0)

The output is ""
However, in specific :
"ValidateTypedArray is applied to the this value prior to evaluating the algorithm. "
So, a TypeError should be thrown.
BT group
2018.08.29

Interestingly there's a spec note here that if the ECMA-402 Intl API is present (which CC does implement) then the algorithm should instead be based on that of Array.prototype.toLocaleString - does that then invalidate the requirement to invoke ValidateTypedArray, I wonder?  Intuitively one would expect that caveat to still apply but specifications can be fickle things.

@fatcerberus It is a bit oddly worded, but in this case ValidateTypedArray must still be applied to the typed array, since the algorithm will need to access internal slots instead of a "length" property, and will need to make sure that the typed array has not been detached.

Alright, thanks - that’s what I intuitively expected but you can never be too sure with cases like this. :)


set_next doesn't work when builtin_marks are enabled

When builtin_marks are set, set_next doesn't work. Works fine when builtin_marks are empty.
I use Neovim HEAD.

Moving to builtin marks is disabled, since there is a bad interaction with moving to and then setting the ‘ mark at the same time. However, it should be possible to allow movement to the other builtin marks. I can take a look when I have time.

Perhaps, I don't understand. I'm trying to set new marks using set_next, not move to marks. When builtin_marks are enabled, that API doesn't work.

Whoops, I misread that as get_next. It should be a simple fix.


[Request] Theme Suggestion

HI,
Please consider using tabs for showing HSLA, HEXA etc. This will save much space and look cooler
Image:

REF: Simple tabs
https://www.w3schools.com/w3css/tryit.asp?filename=tryw3css_tabulators_grid&stacked=h
Great Plugin BTW!!!

https://codepen.io/mafar/pen/RwbqYBE?editors=0010
here is my take with qtip2 integration . Please add to documentation as it may help someone


[network] move directional connections to different metric

I mistakingly pushed this to the wrong branch, so this should update the info from this PR to master
https://github.com/libra/libra/pull/5399

@bors-libra r=davidiw

:pushpin: Commit 0e03565 has been approved by davidiw

:hourglass: Testing commit 0e035655f5a335c026facf5446eb108eef68f822 with merge 3c379c9325aedb4b9b63275d77338c85711d68e4...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: davidiw
Pushing 3c379c9325aedb4b9b63275d77338c85711d68e4 to master...


Improve sorting in module part context

Purpose
$subject
Fixes #37661
With this PR we have done multiple refactoring in the completion providers.

Add type symbols to completion items like string , int, etc which were previously considered modules (pre-declared) instead of types. So the completion item type has changed with this addition. So this affects the test cases in many contexts.

Improve the sorting order of completion items in module level as follows.

main snippet
service snippet
service template snippets
service keyword
function snippet
open record snippet
close record snippet
other snippets
types
keywords
modules
Other

Samples

Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[x] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[x] Increased Test Coverage
[x] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Created https://github.com/ballerina-platform/ballerina-lang/issues/38817 to track 2.a and 2.b

Previously I had a hard time getting the string completion item which is used more often than stream, and it was annoying as a user(hope this is relatable to others as well).

But with this new refactoring we get the string completion item before stream and it so satisfying.

Maybe we can use this as an example to improve UX in future. It can be a whole new different scenario, but can make a huge difference for a regular user.

Is there a specific reason to include service keyword after the service snippets but not function keyword after function snippet?

Is there a way that we can improve following scenarios(only those of which are within the scope of this PR/issue)
a. getting main function snippet in test.bal
b. not getting relevant completions after public (here we get error after public)
c. getting main function snippet after public

Screen.Recording.2022-11-24.at.02.51.53.mov

service is a module level member in ballerina. That's why we have special cased it. We can special case the function keyword as well. +1

Shall we send this to 2201.3.x as well?


Puppet agent don't apply my own chocolatey module

I created a module and the init.pp it looks like:
class msoffice365 {
package { 'office365homepremium' :
ensure => installed,
provider => 'chocolatey',
}
it is at .../etc/modules/msoffice365/manifests/
But when i run the puppetagent it recognizes but don't install anything just finish with no errors. There is something that is missing? When i run the debug agent it only mentions the module at:
Debug: /Package[office365homepremium]: Provider chocolatey does not support features virtual_packages; not managing attribute allow_virtual.

It worked changin 'office365homepremium' for 'office365proplus'

I am also hopeful you used puppetlabs/chocolatey as compared to this module


Check and update architecture documentation

Description
Check documentation for completeness
Acceptance criteria

Update documentation (new API, etc)

Definition of done (DoD)

Added only after week 5
The same for all features
Here goes the project specific part

DoD general criteria

Feature has been fully implemented
Feature has been merged into the mainline
All acceptance criteria were met
Product owner approved features
All tests are passing
Developers agreed to release

Updated the information and structure of the project's wiki page, including all subpages.


Make the request package more flexible, consistent and simpler - a "Discussion PR" containing 5 potential future PRs for review

This is a pure "discussion-PR" as it is too big to be applied with one merge. It is presented in this way,
as some of the PRs build on top of the others, so they cannot be split out.
It is therefore best to review the 5 commits one by one, as each of them is meant to become a separate PR.
It is the result of the work on a rainy weekend in London. :-)
Motivation
The aim of this series of PRs is to improve the simplicity, consistency and flexibility of the functionality contained in the request package. The quick summary of the 5 commits:

Add support for reusable and combinable validation rules (e.g. RequiredIntParam("foo").should(bePositive and beSmallerThan(5)))

Improve DecodeRequest API to return Try[T] instead of Option[T] for improved error handling

Make DecodeRequest more broadly useful through adding an implicit as[T] method to RequestReader[String], RequestReader[Option[String]] and RequestReader[Seq[String]]

Unify error handling based on just 3 error types and refactor the request package for a much simpler implementation

Introduce deprecations for the API elements which are now covered in a more generic way by the new features

The following sections provide more detail for each of the PRs
PR 1: Add support for reusable and combinable validation rules
This corresponds to the feature set discussed here: https://github.com/finagle/finch/issues/26#issuecomment-72353887, so it should hopefully not be controversial.
This PR leaves everything in place that Ryan already added for validation logic. He recently introduced the option to declare validation rules inline:
RequiredIntParam("foo").should("be positive")(_ > 0)

This PR adds to this work the following:
Negation:
RequiredIntParam("foo").shouldNot("be positive")(_ > 0)

Reusable Rules:
val bePositive = ValidationRule[Int]("be positive")(_ > 0)

RequiredIntParam("foo").should(bePositive)

Combinable Rules:
val bePositive = ValidationRule[Int]("be positive")(_ > 0)
def beSmallerThan(n: Int) = ValidationRule[Int]("be smaller than $n")(_ < n)

RequiredIntParam("foo").should(bePositive and beSmallerThan(5))

The old ValidationRule reader has been removed. Due to the name conflict there
was no option to deprecate it first.
This PR also includes a new section in docs.md for this feature.
Please ignore everything around error handling in this PR, this is covered by PR 4.
Breaking API changes: Only the removal of the old ValidationRule.
PR 2: Improve DecodeRequest API to return Try[T] instead of Option[T] for improved error handling
This is also very unlikely controversial. The existing API was insufficient for proper error handling.
Returning an Option is usually not a good idea in APIs where details about the failure may be helpful.
The new API is simply:
trait DecodeAnyRequest {
  def apply[A: ClassTag](req: String): Try[A]
}

This PR also makes all necessary adaptions for the various JSON integrations (Argonaut, Jawn, Jackson, Finch JSON).
Note that Finch JSON still does not report any error detail, as the underlying public API (Scala SDK JSON parser) is very limited. It should be possible to extract more detail through using a more low-level API here, but this is not yet covered by this PR.
Breaking API changes: Only for those who manually implemented a DecodeRequest. Anyone using the various JSON integration modules should not be affected.
PR 3: Make DecodeRequest more broadly useful through adding an implicit as[T] method to all string based readers.
This has not yet been discussed anywhere, but is hopefully very obviously attractive. The use of the DecodeRequest API has previously been limited to body parsers. However, this limitation feels arbitrary. It is sometimes just as useful to have for parameters or for headers. As an example, in our current project almost each route contains two parameters which need to be converted to a Joda DateTime. Composing this outside of a reader would be cumbersome and feel like the wrong abstraction.
The new as[T] method makes it easy to do this together with other parameter validations and conversions:
val user: RequestReader[TimeSeries] = for {
  groupBy <- RequiredParam("groupBy")
  startDate <- RequiredParam("startDate").as[DateTime]
  endDate <- RequiredParam("startDate").as[DateTime]
} yield TimeSeries(groupBy, startDate, endDate)

The above will work as long as an implicit DecodeRequest[DateTime] is in scope.
For headers this feature may come in handy, too. In particular for those where you write custom parsers based on RFCs, like the Accept header for example.
The beauty of this feature comes from the fact that the as[T] method is not hardcoded anywhere, but implicitly available for any string based reader. The RequiredCookie reader for example cannot be directly used with as[T] as it is of type RequestReader[Cookie], but the following will work without problems:
RequiredCookie("visits").map(_.value).as[Int]

The as[T] method is available for the following reader types:
RequestReader[String]         --> RequestReader[T]
RequestReader[Option[String]] --> RequestReader[Option[T]]
RequestReader[Seq[String]]    --> RequestReader[Seq[T]]

Breaking API changes: Just one (hopefully minor): RequiredParams is now a RequestReader[Seq[String]] instead of List to make it more generic and matching to the implicit as for readers producing a Seq[String].
Otherwise this PR is non-destructive, it only adds to the feature set. In my eyes the old RequiredBody[T] should at some point be deprecated in favor of RequiredBody.as[T] to make it all feel more unified, but this is not covered by this PR yet.
PR 4: Unify error handling based on just 4 error types and refactor the request package for a much simpler implementation
The error handling has already been discussed here: https://github.com/finagle/finch/issues/26#issuecomment-72385723
It unifies all error types of Finch to just these 4:
case class RequestReaderErrors(errors: Seq[Throwable]) // when multiple request item were invalid or missing

case class NotFound(item: RequestItem) // when a required request item (header, param, cookie, body) was missing

case class NotParsed(item: RequestItem, targetType: ClassTag[_], cause: Throwable) // when type conversion failed

case class NotValid(item: RequestItem, rule: String) // when a validation rule did not pass for a request item

where RequestItem is this ADT:
sealed abstract class RequestItem(val kind: String, val nameOption:Option[String] = None) {
  val description = kind + nameOption.fold("")(" '"+_+"'")
}
case class ParamItem(name: String) extends RequestItem("param", Some(name))
case class HeaderItem(name: String) extends RequestItem("header", Some(name))
case class CookieItem(name: String) extends RequestItem("cookie", Some(name))
case object BodyItem extends RequestItem("body")
case object MultipleItems extends RequestItem("request")

This makes the life for both Finch users and Finch maintainers easier, as it is very consistent and intuitive.
Since this was a bigger change, it was best to combine this work with a bigger internal refactoring. Apart from the change in error types, this refactoring does not change any public APIs, but it greatly simplifies the implementation of the request package, which should become immediately obvious when skimming over the diff. Through the use of the features introduced with the previous PRs and the introduction of just a handful of new convenience methods, many of the built-in reader implementations could be reduced to just 1 line.
This PR may be the first to be slightly controversial because of the internal rework being quite radical. However I hope that most people will immediately see that the code is much easier to read and maintain now.
Breaking API changes: Everything around Exception types
PR 5: Introduce deprecations for the API elements which are now covered in a more generic way by the new features
This may be controversial, but I still hope to convince Finch maintainers and users to reduce the API.
In particular I'm in favor of deprecating all the conversion readers like RequiredIntParam in favor
of the more generic RequiredParam.as[Int]. I think it is more consistent, and more functional style
to allow for composition of these features instead of hard-coding just a bunch of them while making it
non-obvious how to do the same for other types if required.
PR 4 also introduced implicit DecodeRequest instances for all the types that were previously supported
throug hard-coded readers. The decoders are all just one line, replacing hundreds of lines of plumbing
for the old hard-coded readers.
Since those were very central APIs which might be used in dozens of places in some projects, I feel it is
a bit rude to just remove them. Even though we are pre-1.0 and could do theoretically do anything we want,deprecation is easy to do in this case. I'm already happy if we get these readers removed for 1.0.
With the cleanup and unification, the Finch request package can be explained in 10 minutes:

Use one of the following 6 base reader types (Xx stands for Required/Optional)

XxParam
XxParams
XxHeader
XxCookie
XxStringBody
XxArrayBody

Convert types of any string based reader with as[T] combined with an implicit DecodeRequest[T]

Validate with should or shouldNot methods, defining rules inline or with reusable validators

Compose in applicative syntax with ~ or in monadic syntax with for-comprehensions/flatMap

Transform readers with map or mapFuture

Handle any errors based on just 4 unified error types

The mapFuture method was introduced with PR 4. It is slightly unusual, but extremely useful, the internal implementation uses it a lot, as it is more convenient than flatMap in almost all cases.
Regarding naming XxStringBody could be renamed back to XxBody now, if we remove the old XxBody[T] type instead of just deprecating it. I also feel XxBinaryBody would be clearer than XxArrayBody, but this is just a minor detail.
This PR also updates docs.md to use the new API instead of the deprecated one.
Breaking API changes: None, this PR just introduces deprecations.

And again, as a reminder: Please review the 5 commits one by one, it will be overwhelming otherwise
Btw: as you noticed I had some weird problems with whitespace that make some files appear as completely changed

Coverage decreased (-1.83%) to 90.16% when pulling 5f58b1fcdcd1b92142dab631fd1e7c1f241e9e92 on Sphonic:requestreader-improvements into 828c8fe43ac445b66edd1ab6e969e6db8c870c97 on finagle:master.

OMG! Let me review this one-by-one. Amazing amount of job!

:+1:  on PR1 rather then a couple of scaladoc comments.

:+1:  on PR2! Very solid job!

:+1: PR3 looks pretty good to me. Do you think we may consider using RequiredParam("a").as[Int] instead of RequiredIntParam()?

Regarding PR 3, yes, I suggest deprecating RequiredIntReader in PR 5 for unifying the API.
And RequiredParams("ids").as[T] works with a simple DecodeRequest[T], see the docs I wrote for this PR and the tests. It also works for optional string readers.
Need to catch some sleep, will extract PR 1 tomorrow in the morning and do the little fixes you asked for.

Just saw it! Amazing!  Don't worry. Take a break. I will carefully review the rest of the changes.

@rodrigopr, @rpless, @benjumanji, @BenWhitehead I need some help here :)

I would go with renaming DecodeRequest to Decode and DecodeAnyRequest to DecodeAny since it's no longer just about bodies but about any request parts/items.

Also, @jenshalm please go ahead with renaming StringBody to Body and ArrayBody to BinaryBody. This will make things more uniform since there is always XY.as where X is either Required or Optional and Y is either Body, Param, Header etc.

I don't think I'll be able to review tonight, but will try and get to it tomorrow.
Based on skimming the description I think a lot of things make sense but I'll spend more time digging in tomorrow.

Awesome changes!
Missed most of the discussion, will try to catch up tomorrow.

One thing will be broken with new API. Previously we was able to read params/etc like this:
val a = RequiredIntParam("a")(req)

Which may be rewritten using the new API as:
val a = (RequiredParam("a").as[Int])(req)

We have to use extra-brackets around reader, since as method takes an implicit parameter. We can either try to solve  by adding an extra method in implicit classes with as function. We need a method like this:
def as[A](req: Req)(implicit ev: Req => HttpRequest, DecodeMagnet[A]): Future[A]

Although, This is not a big deal. I think we can just mention it in the documentation.

Also, @jenshalm could you please fix a build. It doesn't compile for Scala 2.10:
[error] /home/travis/build/finagle/finch/core/src/main/scala/io/finch/request/package.scala:280: overriding method applyOrElse in trait PartialFunction of type [A1 <: com.twitter.util.Try[T], B1 >: io.finch.request.package.NotParsed](x: A1, default: A1 => B1)B1;
[error]  method applyOrElse has incompatible type
[error]       else RequestReaderErrors(converted collect { case Throw(e) => NotParsed(reader.item, tag, e) }).toFutureException
[error]                                                  ^
[error] one error found
[error] (core/compile:compile) Compilation failed.

You can check it locally by running sbt +test.

Regarding RequiredIntParam("a")(req), yes I noticed you added the workaround for RequiredBody[A](req). If this is seen as essential we can add similar workarounds for the three as[T] implementations. However, is this often used in app code? It's probably more likely a shortcut for unit tests?
Yes, I did not run the tests for 2.10, as this was first a mere discussion PR. I will have a closer eye on these things when I extract the individual PRs. I'll also have a look then which of them causes the decrease in coverage.

Regarding the rename from DecodeRequest to Decode. I think for the semantics it is not strictly necessary, as headers, cookies and params can be seen as part of the request technically (it is not named DecodeBody after all). However, I find the shorter name nicer nevertheless. I would just keep it in sync with renaming EncodeResponse to Encode then.

Regarding RequiredIntParam("a")(req). Let's just note in the docs that there're extra brackets required, if the request passed directly do primitive (means the reader, which is not composed by user but implemented directly in Finch) reader.
Regarding the Encode/Decode-request. Let's just keep the names for now.
There is one thing that worries me about as method. We do have support of Jackson, which implements the DecodeAny and EncodeAny. So, it might be used as an implicit decode in as method, since it support any type (including Int, etc). @jenshalm could you please add a test to jackson package that reads a simple integer param with as method. I'm just curious how the compile will handle the situation if there two implicits available in the scope. If it doesn't compile, we at least have to provide a workaround for users who use Jakcson.

Ah, this is good to know, I will make sure this is tested. This is for PR 3, until that is "hot" I have had the time to look into it (very likely tomorrow evening).
Is the DecodeAny for Jackson optional or is it the only way to decode with Jackson? An implicit for Any is usually not a good thing to have. Even if the implicit resolution for Int for example works (e.g. through the compiler picking the closest match), the downside would still be that if someone uses a type for parameter decoding without matching implicit in as[A] she would get the Jackson Any decoder then instead of an error message.

Yeah. DecodeAnyRequest doesn't actually returns Any, but has a method with type parameter:
def apply[A](s: String): Try[A]

This is why there is a decodeMagnet everywhere. For Jackson we have to provide a ClassTag as well. Thus, decodeMagnet is actually created from both implicit values: decodeAnyJackson and ClassTag.
The reason we have both Decode and DecodeAny is that there are JSON libraries that works with case classes and other types directly via reflection. They don't really need new instances of decode per each class the used.

Wow, thanks for all the hard work @jenshalm I'm going to reserve doing detailed reviews for each of the individual PRs so that it's easier for me to follow the contained changes.
Most things seem reasonable.
One thing that did stand out to me though is that the new as[T] can only be applied to strings.  It would be nice if it could also apply to Array[Byte] particularly for the scenario where a binary body is used (perhaps protobuf) that can't be deserialized from a string.  Maybe this is a change for a different PR but I think we should avoid limiting to string when it's completely reasonable to have something be binary.

I think we're totally fine with as method only on string-based readers. In fact, all the base readers we have are string-based, which feels pretty reasonable to me, since the HTTP protocol is string-based itself. There is only one thing that doesn't feet into this concept - BinaryBody or ArrayBody previously. What I think we should do is to deprecate/remove it in favour of just Body, which is string-based. Thus, users will always be able to get a binary representation by just mapping the reader to _.getBytes().
This approach also works well with protobuf, since we're always ending up with writing an implementations of DecodeRequest, which is also string-based.

One problem with that approach though is that there may be weirdness in how the string is encoded possibly leading to loss of data.  I agree that HTTP is primarily string based, however this characteristic does not hold true for the request/response body.
It is very practical to have binary bodies. An example of a binary response body might be to serve an image thumbnail of a users profile image. You don't want to encode to a string from a png, and in the reverse you don't want to decode to a string before trying to decode to a png.
Another scenario I can think of where the server would need to handle a binary body would be if the client posts a gzipped request body, using finch's readers it would very feasable to do something like the following:
val reader = for {
  bytes <- RequiredArrayBody
  decompressed <- Decompress(bytes)
  bigObject <- Decode(decompressed).as[BigObject]
} yield bigObject

To be honest, I haven't thought about binary bodies for two reasons: I haven't used them for years and I've built as[T] on top of the existing DecodeRequest[T] functionality which is entirely string based.
I think we should keep RequiredBinaryBody as calling getBytes on a RequestReader[String] feels wrong to me and could be misleading, as in reality it is the other way round, we start with bytes and might turn them into a string.
I don't think we would want a fully generic as[T] as this would mean we'd need to move to DecodeRequest[A,B] which would overcomplicate the design and implicit resolution and for very specific transformations, you'd always have map and flatMap on the readers.
What we could do, if there is support for that idea, is to introduce DecodeBinary[T] and EncodeBinary[T] later. The good news is, should we go that route, it will be trivial to extend as[T] to be usable with those, too. However, this is beyond the scope of this PR galore, as we'd also need to enhance the other side, TurnIntoHttp[T] which is also currently entirely string based.
Should there be interest in making working with binary protocols more convenient, I'd suggest to create an issue and describe the use cases.

PR 1 is already merged now.
PR 2 is now active and under review: https://github.com/finagle/finch/pull/165

PR2 #165 and PR3 #168 are merged.

PR4 #170 is merged

I'm closing this PR now, as it was only meant to exist for review and discussion.
4 PRs have been merged now, the 5th is active for review now here: https://github.com/finagle/finch/pull/173.
The last PR deals with deprecating APIs, so it would be nice to get feedback from those who have an existing code base using Finch.