ENH add RubinPSF class

This PR adds a class wrapping the Rubin PSF models.

OK @rmjarvis, this one is ready for a review!
I have added a basic wrapper around the Rubin PSF interface along with the standard tests.
I also verified that the PSF centering is correct (always at the sub-pixel position) and that galsim can recenter the PSFs pretty well. We would never recenter them when actually using them of course.

Sorry for the extra commits. I found a few typos and thought of one more test. Now I'm done.

I'm gonna merge this one. Happy to accept comments and fix things later!


failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED

My environment：cudatoolkit 9.0 cudnn 7.3 ubutu18 tensorflow 1.14.0
Unfortunately, the following error occurred：failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED
who can help me？Thank you very much！

Two common reasons for this issue:

If you use RTX 30 series GPUs, the GPU driver does not support cuda 9.0. Please refer to this answer: https://github.com/qqwweee/keras-yolo3/issues/332#issuecomment-738773653
If not, it may be caused because you didn't install the patches for cuda 9.0. Suggest to install patch as follows: https://developer.nvidia.com/cuda-90-download-archive

Hope it helps :)


Add IBM Watson Content Hub CMS

Sounds like the type should be API Driven?

Hi. Thanks for the quick response. Wasn't sure if you could put anything in the field, or what the options are. I'd say it is more "Business user experience driven, but with great API support too", but that's quite long....

Still, happy to do that if it helps

LGTM


[SPARK-48205][PYTHON] Remove the private[sql] modifier for Python data sources

What changes were proposed in this pull request?

This PR removes the private[sql] modifier for Python data sources to make it consistent with UDFs and UDTFs.
Why are the changes needed?

Does this PR introduce any user-facing change?

No
How was this patch tested?

Existing tests
Was this patch authored or co-authored using generative AI tooling?

No

cc @HyukjinKwon

Merged to master.


New release?

Would it be possible to cut a new bijection release? (I'm particularly interested in picking up https://github.com/twitter/bijection/pull/224.)

I'd love to fix #199 before we publish...

Will take a stab...

https://github.com/twitter/bijection/releases/tag/v0.9.0


when I use ALE 1D, ValueError: arange: cannot compute length

Thanks for raising this issue! However, this specific repo is no longer actively managed as the code base has been re-branded as scikit-explain (https://github.com/monte-flora/scikit-explain). If you continue to have this error with the new package, please raise the issue on that GitHub page.  If you do raise an issue, it is helpful to provide the full error code, including the line where the error was raised. If you have any questions, feel free to ask!


Code

Hello, I'm confused by those code, are those because of one-hot label?
target_boxes = torch.cat([t["boxes"][v[1]] for t,v in zip(targets, indices)], dim=0)
target_classes_o = torch.cat([t["labels"][v[1]]
for t,v in zip(targets, indices)])#1d class index  sorted by tgt_ind
    #compute normalizer for postive
    pos_idx = self._get_src_permutation_idx(indices)
    pos_idx_c = pos_idx + (target_classes_o.cpu(), )
    src_boxes = pred_boxes[pos_idx]

why       pos_idx_c = pos_idx + (target_classes_o.cpu(), )    have a "+"

sorry for late reply, this is for a concatenation between tuples. Thus , the result pos_idx_c is a tuple of index in  (batch, sample , category) dimension.


Independent workflow job for dbt-postgres

Description & motivation
resolves #213
resolves #212
resolves #211
Checklist

[x] This code is associated with an issue which has been triaged and accepted for development.

Merging even though GitHub Actions CI isn't passing. It appears to be completely unrelated to this PR and others are investigating.
The key thing for this PR is that CircleCI is passing ✅


Update SimBrief Briefing Blade

Fixed wrong EET and ENDURANCE values at IVAO & VATSIM prefile forms with a php function converting seconds to HHmm format.
Also added View Route At SkyVector link (uses auto zoom to fpl route on High Enroute Charts) , thx Maco for his idea and original link.

You will need to rebase your other PR - meaning pull this down and update the changes in that one to fit this one, otherwise there might be conflicts.

Which other PR you are reffering @nabeelio ? If we are talking about the simbrief_form.blade, they are completely independent from each other, this is just the simbrief_briefing blade dispalying the results as you know.
Best wishes,

Ah, nevermind - I got confused 👍

Ok, no problem :)


GeniusReader loader

Description
Added GeniusReader loader, which allows for loading of documents from the Genius API
Fixes # (issue)
Type of Change
Please delete options that are not relevant.

[X] New Loader/Tool

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Added new unit/integration tests
[ ] Added new notebook (that tests end-to-end)
[X] I stared at the code and made sure it makes sense

Suggested Checklist:

[X] I have added a library.json file if a new loader/tool was added
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] I ran make format; make lint to appease the lint gods

Thanks for the contribution @ThibaudARoy. Looks like test our failing due to import errors. Can you follow a similar pattern to whats been used in the past, which is to do lazy imports and throw an  error if the module is not installed?
See here: https://github.com/run-llama/llama-hub/blob/364feb5b9c41a081da53aae02e894c6caa2f478d/llama_hub/llama_packs/gradio_agent_chat/base.py#L36

@ThibaudARoy is there an update on this?

@jerryjliu updated the import statement. Let me know if there's anything else I can fix.

sorry for delay @ThibaudARoy seems like there's some minor linting errors you can fix with make lint; make format!


Different kernel_size and strides values on DepthwiseConv2D (with padding=same) will trigger significant inconsistent results across Tensorlfow, CNTK and Theano, even using norml parameters like kernel_size=2,strides=1.

Different kernel_size and strides values on DepthwiseConv2D (with padding=same) will trigger significant inconsistent results across Tensorlfow, CNTK and Theano, even using norml parameters like kernel_size=2,strides=1.
System information

Have I written custom code (as opposed to using example directory):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 & Linux Ubuntu 18.04
Tensorflow backend (yes / no): yes
Tensorflow version：1.14.0-gpu
Cntk version: 2.7-gpu
Theano version:1.04-gpu
Keras version: 2.3.1
Python version: 3.6.9
CUDA/cuDNN version: -
GPU model and memory: -

Describe the current behavior
When I use DepthwiseConv2D with the parameter padding=same, and change the kernel_size and strides, I notice obvious output differences across 3 backends in some cases, as shown in the following tables.  (See the codes below to reproduce the issue.)
Note that, we take the sum of absolute diffenece (SoAD) as the metric in these tables. (See SoAD implementation in code snippet below.)

kernel_size=1

CNTK behaves differently from Tensorflow and Theano in some cases when kernel_size=1.

strides
Tensorflow-CNTK
Tensorflow-Theano
CNTK-Theano

1
0.
0.
0.

2
0.
0.
0.

3
0.
0.
0.

4
897.31089687
0.
897.31089687

5
0.
0.
0.

6
0.
0.
0.

7
446.60348872
0.
446.60348872

8
316.42553336
0.
316.42553336

9
273.88436383
0.
273.88436383

10
0.
0.
0.

kernel_size=2

When strides=1, three backends show quite different outputs. Similar issues occur when strides=4/7/8/9.  And Theano behaves quite differently from Tensorflow and CNTK in other cases.

strides
Tensorflow-CNTK
Tensorflow-Theano
CNTK-Theano

1
1.66629908e+04
1.89978224e+04
1.94654342e+04

2
3.41665281e-13
4.25164804e+03
4.25164804e+03

3
1.90125585e-13
2.15956529e+03
2.15956529e+03

4
1.13031484e+03
1.16128682e+03
1.22253624e+03

5
6.84723274e-14
8.51269990e+02
8.51269990e+02

6
4.63077927e-14
6.57431127e+02
6.57431127e+02

7
4.37231640e+02
4.89286548e+02
4.55944453e+02

8
2.53817713e+02
2.63568541e+02
2.46146475e+02

9
2.76400556e+02
3.37932890e+02
3.35156225e+02

10
2.37899977e-14
3.07664662e+02
3.07664662e+02

3）kernel_size=3
Three backends do not show definitive similarity facing different strides values.

strides
Tensorflow-CNTK
Tensorflow-Theano
CNTK-Theano

1
2.35979202e-12
3.30831253e-12
2.94574548e-12

2
4.64192581e+03
4.64192581e+03
8.69110122e-13

3
2.38899066e+03
2.38899066e+03
3.62354989e-13

4
1.48203932e-13
1.12537857e+03
1.12537857e+03

5
9.62017411e+02
9.62017411e+02
1.33954073e-13

6
7.45779210e+02
7.45779210e+02
9.36620573e-14

7
6.25862209e-14
4.90622760e+02
4.90622760e+02

8
2.70831408e+02
2.80985657e+02
2.92329633e+02

9
2.56000644e+02
3.00206323e+02
3.10693253e+02

10
3.37504933e+02
3.37504933e+02
4.37499429e-14

Key insights
It is explictly mentioned in Conv2D's docs that Note that "same" is slightly inconsistent across backends withstrides != 1,(see keras\layers\convolutional.py line391 ). This also applies to the case of DepthwiseConv2D,  but we found no similar warnings for DepthwiseConv2D. I think Keras needs to update its comments here.
Worse still, even when strides = 1 (e.g., kernel_size = 2 and strides = 1), there shows significant difference across CNTK, Tensorflow and Theano, as shown in the table above.  Dose it indicate an implementation bug in DepthwiseConv2D?  This may lead to potential risk when we use DepthwiseConv2D in real applications.
Code to reproduce the issue
import os
import numpy as np
import keras.layers as L
import keras.backend as K
import importlib
from keras.models import load_model
from keras.engine import Model, Input

backends = ['cntk','tensorflow','theano']

# SoAD, Sum of Absolute Difference
def acc_abs_diff(output1, output2):
    assert output1.shape == output2.shape
    abs_diff = np.abs(output1-output2)
    return  np.sum(abs_diff)

def set_keras_backend(backend='tensorflow'):
    if K.backend() != backend:
        os.environ['KERAS_BACKEND'] = backend
        importlib.reload(K.load_backend)
        importlib.reload(K)
        assert K.backend() == backend

kwargs={'kernel_size': 3, 'padding': 'same', 'strides': 1, 'dilation_rate': (1, 1), 'data_format': 'channels_last'}
kwargs['kernel_size']=2
listdiff=[]
for stride_value in range(10):
    kwargs['strides']=stride_value+1
    input_data = (10 * np.random.random((1,32,32,16)))
    input = input_data.astype('float64')
    set_keras_backend('tensorflow')
    layer = L.convolutional.DepthwiseConv2D(**kwargs)# you can also use DepthwiseConv2D
    x = Input(batch_shape=input.shape)
    y = layer(x)
    bk_model = Model(x, y)
    model_path = os.path.join('./', 'model.h5')
    bk_model.save(model_path, bk_model)
    output = {}
    for bk in backends:
        try:
            set_keras_backend(backend=bk)
            model = load_model(model_path)
            output[bk] = model.predict(input)
        except:
            print('error result')
    try:
        diff1 = acc_abs_diff(output['tensorflow'], output['cntk'])
    except:
        diff1 = None
    try:
        diff2=acc_abs_diff(output['theano'], output['cntk'])
    except:
        diff2=None
    try:
        diff3=acc_abs_diff(output['theano'], output['tensorflow'])
    except:
        diff3=None
    listdiff.append([diff1,diff2,diff3])
arraydiff = np.array(listdiff)
print(arraydiff)
print('finish')

@Jup11ter
We see that you are using old version of tensorflow which is not actively supported now, please verify on 2.x and open a new issue in case you face any errors, hence moving this to closed status.


detect broken on master

Discovered during #55

The issue was on the refactor branch and has been fixed. I'm still not happy with the way meters, producers and devices are setup so this will likely see another iteration.