Stuck after printing 'Successfully opened dynamic library libcublas.so.10.0'

Description
I run this command 't2t-trainer --problem=librispeech --model=transformer --data_dir=~/dataset/t2t/librispeech/ --output_dir=. --hparams_set=transformer_librispeech --worker_gpu=1' and it's stuck after printing 'Successfully opened dynamic library libcublas.so.10.0'.
Then I set TF_CPP_MIN_VLOG_LEVEL=2, it keeps printing
'tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 20480 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 40960 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 60000 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 60000 ms.'
...
Environment information
OS: <your answer here>

$ pip freeze | grep tensor
# your output here
mesh-tensorflow==0.0.5
tensor2tensor==1.13.4
tensorboard==1.14.0
tensorflow-datasets==1.0.2
tensorflow-estimator==1.14.0
tensorflow-gpu==1.14.0
tensorflow-metadata==0.14.0
tensorflow-probability==0.7.0
$ python -V
# your output here
Python 3.6.5 :: Anaconda, Inc.

For bugs: reproduction and error logs
# Steps to reproduce:

# Error logs:
See descriptions

Description
I am having the same issue for both tensorflow version 1.14.0 and tensorflow version 1.14.1 that were built using CUDA 10.1.
Environment information
OS: Ubuntu 16.04.6 LTS

$ pip freeze | grep tensor
mesh-tensorflow==0.0.5
tensor2tensor==1.13.4
tensorboard==1.14.0
tensorflow==1.14.0
tensorflow-estimator==1.14.0
tensorflow-metadata==0.14.0
tensorflow-probability==0.7.0

$ python -V
Python 2.7.12

For bugs: reproduction and error logs
# Steps to reproduce:
t2t-trainer --problem=librispeech --model=transformer --data_dir=~/datasets/t2t/librispeech/ --output_dir=~/trainoutput/librispeech/ --hparams_set=transformer_librispeech --worker_gpu=1

# Error logs:
[...]
session_manager.py:500] Running local_init_op.
session_manager.py:502] Done running local_init_op.
basic_session_run_hooks.py:606] Saving checkpoints for 0 into ~/trainoutput/librispeech/model.ckpt.
tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 20480 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 40960 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 60000 ms.
tensorflow/core/framework/model.cc:440] Starting optimization of tunable parameters
tensorflow/core/framework/model.cc:482] Number of tunable parameters: 0
tensorflow/core/kernels/data/model_dataset_op.cc:172] Waiting for 60000 ms.

I stuck at the same step when trying running LibriSpeechCleanSmall+Transformer. And I can't receive any log like 'Starting optimization of tunable parameters'. It just stuck and keep occupying GPU without unexpected termination.
Enviroment:
OS: Ubuntu 16.04.5 LTS

$ pip freeze | grep tensor
mesh-tensorflow==0.0.5
tensor2tensor==1.13.4
tensorboard==1.14.0
tensorflow-datasets==1.1.0
tensorflow-estimator==1.14.0rc1
tensorflow-gpu==1.14.0rc1
tensorflow-hub==0.4.0
tensorflow-metadata==0.14.0
tensorflow-probability==0.7.0

$ python -V
Python 3.5.2

For bugs: reproduction and error logs
# Steps to reproduce:
t2t-trainer --problem=librispeech_clean_small --model=transformer --hparams_set=transformer_librispeech --data_dir=/data/tensor2tensor/data/librispeech_clean_small --output_dir=/data/tensor2tensor/exp --train_steps=1000 --eval_steps=100 --verbosity=0

# Error logs:
[...]
2019-08-06 06:28:42.670483: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-08-06 06:28:43.090613: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-08-06 06:28:43.266243: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-08-06 06:28:43.328620: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-08-06 06:28:43.830164: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-08-06 06:28:44.081212: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-08-06 06:28:44.819371: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-08-06 06:28:44.826114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-08-06 06:28:44.834796: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-08-06 06:28:44.839837: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-06 06:28:44.839860: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0
2019-08-06 06:28:44.839867: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N
2019-08-06 06:28:44.854920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11596 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:8c:00.0, compute capability: 3.7)
2019-08-06 06:28:49.295630: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
I0806 06:28:56.309820<PHONE_NUMBER>96832 session_manager.py:500] Running local_init_op.
I0806 06:28:56.744457<PHONE_NUMBER>96832 session_manager.py:502] Done running local_init_op.
I0806 06:30:06.414134<PHONE_NUMBER>96832 basic_session_run_hooks.py:606] Saving checkpoints for 0 into /data/tensor2tensor/exp/librispeech_transformer_clean_small/model.ckpt.
2019-08-06 06:30:50.528524: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0

Did anyone find a fix for this? Getting the exact same problem using t2t-decoder, depending on the model, it either hangs on Successfully opened dynamic library libcublas.so.10.0 or Successfully opened dynamic library libcudnn.so.7. This was not happening for me 2-3 weeks ago - I'm not sure what's changed

Description
I also have the same issue. After "Successfully opened dynamic library libcublas.so.10.0", nothing even after 3 days.
  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
I0823 03:15:32.888142<PHONE_NUMBER>74464 session_manager.py:500] Running local_init_op.
I0823 03:15:33.431387<PHONE_NUMBER>74464 session_manager.py:502] Done running local_init_op.
I0823 03:15:55.611809<PHONE_NUMBER>74464 basic_session_run_hooks.py:606] Saving checkpoints for 0 into /tmp/t2t.work/librispeech_clean_small.20190823/model.ckpt.
2019-08-23 03:16:33.482639: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0

Same here

Continued here:
tensorflow/tensorflow#32017

@rachellim at TF was able to reproduce and resolve the hang issue (parallel_interleave_dataset_op.cc doesn't handle iterator creation errors correctly when the sloppy=True).
With her fix in place (or setting sloppy=False) the training now halts with Conv2D errors.
Other interesting clues (reported by @huang-haijie ) are setting  audio_add_delta_deltas from True to False, OR
seting audio_preproc_in_bottom from False to True prevents the Conv2D error.
It still seems like it may be a TF issue, as the same T2T code works fine with TF 1.13.2, but fails with the Conv2D issues on TF 1.14.0.  Any suggestions for next steps would be appreciated...

Continued here:
tensorflow/tensorflow#32691

Description
I also have the same issue. After "Successfully opened dynamic library libcublas.so.10.0", nothing even after 3 days.
  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
I0823 03:15:32.888142<PHONE_NUMBER>74464 session_manager.py:500] Running local_init_op.
I0823 03:15:33.431387<PHONE_NUMBER>74464 session_manager.py:502] Done running local_init_op.
I0823 03:15:55.611809<PHONE_NUMBER>74464 basic_session_run_hooks.py:606] Saving checkpoints for 0 into /tmp/t2t.work/librispeech_clean_small.20190823/model.ckpt.
2019-08-23 03:16:33.482639: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0

Did the issue resolved?
Running in Google Colab with GPU.

The same happens with tf.n.conv3d used inside a map(...) on a tf.data.Dataset. Any updates on how to solve this?

@ramonemiliani93 , what version of tensorflow are you running?

@ramonemiliani93 , what version of tensorflow are you running? I was not able to reproduce this issue with the following dataset:
def make_tensor(sizes):
  return np.asarray([f * 1.0 for f in range(1, np.prod(sizes) + 1)]).reshape(sizes)

filter = make_tensor([1, 1, 1, 3, 3])
x = make_tensor([10, 2, 3, 1, 3])
dataset = tf.data.Dataset.from_tensors((x, filter))
dataset = dataset.map(lambda input, filter: tf.nn.conv3d(input, filter, strides=[1, 1, 1, 1, 1], padding="VALID")
print(list(dataset))

So it doesn't seem to be an issue with using tf.nn.conv3d inside map. Can you provide a minimal repro?

Based on https://github.com/tensorflow/tensorflow/issues/38100 and https://github.com/f90/FactorGAN/issues/1, I suspect this may be a problem with your CUDA installation.

+1. Same issue. It used to work and died without warning on EC2. Later when I reload the pretrained model it hangs with no output.

@harishkashyap - what version of tensorflow as you using? If you use an older version, does it still work? (Trying to diagnose whether it's an issue with your CUDA installation or a regression in TF)

EC2 pytorch AMI tensorflow 2.3

No idea. I just used an AMI instance with linux 2. Was working fine and now fails to load the pre-trained model.

@sanjoy, can you reassign this to someone on the GPU team to investigate?

Same issue.

I'm trying to run tf object detection model. Getting the same issue; Stuck after Successfully opened dynamic library libcuda.so.1
Please someone helps.

I have this problem using the anaconda cudatoolkit. I ended up using nvidia-docker instead for my cuda/cudnn installation and now it works.

Same issue. Have cuda builded from aur. Dont know what is changed because it suddenly stopped to work.

same issue

same issue on Ubuntu 20.04 with RTX3090. TF was installed using anaconda.

Same issue with Ubuntu 20.04.
Python version: Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46)
Tensorflow version:
❯ conda list | grep tensorflow
tensorflow                2.4.1           gpu_py39h8236f22_0
tensorflow-base           2.4.1           gpu_py39h29c2da4_0
tensorflow-estimator      2.4.1              pyheb71bc4_0
tensorflow-gpu            2.4.1                h30adc30_0

Same issue with Ubuntu 20.04.
Python version:
Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46)

Tensorflow version:
❯ conda list | grep tensorflow
tensorflow                2.4.1           gpu_py39h8236f22_0
tensorflow-base           2.4.1           gpu_py39h29c2da4_0
tensorflow-estimator      2.4.1              pyheb71bc4_0
tensorflow-gpu            2.4.1                h30adc30_0

❯ python
Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import tensorflow as tf
2021-04-30 14:32:10.400682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1

>>> tf.add(1, 2)
2021-04-30 14:32:26.991352: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-04-30 14:32:26.993581: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-04-30 14:32:27.026516: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.027085: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 960M computeCapability: 5.0
coreClock: 1.176GHz coreCount: 5 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 74.65GiB/s
2021-04-30 14:32:27.027104: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-04-30 14:32:27.028731: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-04-30 14:32:27.028771: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-04-30 14:32:27.030183: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-04-30 14:32:27.030438: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-04-30 14:32:27.032093: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-04-30 14:32:27.033044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-04-30 14:32:27.036642: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-04-30 14:32:27.036780: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.037699: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.038214: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-04-30 14:32:27.038509: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-04-30 14:32:27.038916: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.039462: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 960M computeCapability: 5.0
coreClock: 1.176GHz coreCount: 5 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 74.65GiB/s
2021-04-30 14:32:27.039482: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-04-30 14:32:27.039507: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-04-30 14:32:27.039522: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.10
2021-04-30 14:32:27.039536: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-04-30 14:32:27.039550: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-04-30 14:32:27.039563: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-04-30 14:32:27.039577: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-04-30 14:32:27.039590: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.7
2021-04-30 14:32:27.039645: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.040215: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-04-30 14:32:27.040651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2021-04-30 14:32:27.040677: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1

same issue on Ubuntu 20.04 with RTX3090. TF was installed using anaconda.

Same for me

same issue with CUDA=10.0 CUDNN=7.4 Tensorflow=1.14.0. Object detection algo stuck at:
I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10
and sometimes stuck at:
I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
Anyone figue it out? REALLY NEED HELP!

For such cases, it'd be useful to get a stacktrace of where TF is stuck.
You can obtain this using gdb: start gdb, attach to the hung TF process, and then get a backtrace.

Hi @sanjoy , thx for your advice.
I've tried to find out the process id of running python program and gdb attach PID; then use bt to get backtrace. However,  it returns No stack.
Then I tried ps -ef | grep tensorflow-gpu | grep -v grep
and this line returns nothing.
I am wondering does it mean such a problem has nothing to do with tensorflow?
I've also tried with PyTorch on the same machine, and it reflects a similar situation that it also takes a long time to load some libraries from Cuda.
Below are the details of my situation:
GPU: GeForce RTX 3060
Driver Version: 460.73.01
CUDA Driver Veresion: 11.2
Tensorflow: tensorflow-gpu 1.14.0
CUDA Runtime Version: 10.0
cudnn:  7.4.1
(CUDA Runtime and cudnn version fits the guide from Tensorflow official documentation)
I've tried the following test about TensorFlow and it all works:
tf.test.is_built_with_cuda() and tf.test.is_gpu_available()
My situation is the program will stuck at 2021-06-05 12:16:54.099778: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10 for several minutes.
and sometimes stuck at another loading process 2021-06-05 12:21:22.212818: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 for even longer time.
You can check the attached for details.
log.txt
After waiting for around 30 mins, the program will continue running and WORK WELL.
So the MAJOR PROBLEM is that it takes a long time in loading libraries related to cuda (I guess).
And I don't know how to locate the problem and resolve it.

@alanzyt311 I missed that you're running TF 1.14.  1.14 is very old and does not have native support for your GPU (which I believe is Ampere based).  So TensorFlow blocks at startup as it JIT compiles PTX to SASS for your GPU which can take 30+ minutes.
Can you please try running with TF 2.5?

Thx. I've tried TF-gpu 2.0 just now, still not worked (same problem as above). Now gonna try for TF 2.5.

Same problem, I have to wait more than 15 mins to see my training.
And it throws loss==Nan.
I have to quit training and check files for another training.
Same thing occurs again.
Been two days stuck here, 1) takes around 10 ~ 15 mins to start training 2) Throws loss=Nan.
Any help?
I also decreased lr_rate, batch size, nothing works.
I am using docker with following details
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

 0  GeForce RTX 3060    Off  | 00000000:0B:00.0  On |

N/A |
|  0%   60C    P2    41W / 180W |  11391MiB / 12031MiB

This is my output while waiting for training to run:
021-06-10 07:48:07.953350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2021-06-10 07:48:07.953360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2021-06-10 07:48:07.953478: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-10 07:48:07.953996: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-10 07:48:07.954461: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10923 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3060, pci bus id: 0000:0b:00.0, compute capability: 8.6)
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
I0610 07:48:07.957259<PHONE_NUMBER>81216 mirrored_strategy.py:500] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: None
I0610 07:48:07.961450<PHONE_NUMBER>81216 config_util.py:552] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0610 07:48:07.961535<PHONE_NUMBER>81216 config_util.py:552] Maybe overwriting use_bfloat16: False

@Laudarisd Are you using TF 2.5?  If not, can you please try with TF 2.5?

@sanjoy right now I am using tf 2.2 , I will try to use 2.5. But I am not sure whether tf 2.5 will use my gpu or not.
Thanks for you answer.
Will be back after training.

@sanjoy @michyzhu I think this is a problem from mismatched versions of libraries.
I upgraded tf to 2.5 but I could't run training.
So to confirm,  I trained my data in colab everything runs perfectly and smoothly. This is my output in colab while running training:
INFO:tensorflow:{'Loss/classification_loss': 0.7730327,
 'Loss/localization_loss': 0.2685254,
 'Loss/regularization_loss': 0.2532684,
 'Loss/total_loss': 1.2948265,
 'learning_rate': 0.001}
I0611 07:42:00.161126<PHONE_NUMBER>85248 model_lib_v2.py:701] {'Loss/classification_loss': 0.7730327,
 'Loss/localization_loss': 0.2685254,
 'Loss/regularization_loss': 0.2532684,
 'Loss/total_loss': 1.2948265,
 'learning_rate': 0.001}
.........
..........

Right now I am running training in colab.

Sorry for the late update. I reset my environment (including the correct matching among GPU version, cuda, cudnn and TensorFlow version), then the problem gets solved.

@alanzyt311 You are right, need to match all the settings and libraries to run training quicker and smoothly. Thanks.
For me tf 2.5.0
cuda 11.2

@sanjoy Thx for your advice, I follow your tip and solve this problem successfully.
Same issue for me, which was solved after updating tf == 2.4 to tf ==2.5.
Environment information:
OS ubuntu 18.04
python 3.7
CUDA Version 11.4.48
CuDNN Version 8.2.1
Tensorflow-gpu == 2.4
only tensorflow>=2.5.0 supports CUDA11.2 on https://tensorflow.google.cn/install/gpu <\strong>

Same problem for me with a configuration:
GPU: Tesla T4 | NVIDIA driver: 460.106.00 | CUDA: 11.2.152 | CUDNN: 8.1.1 | Tensorflow: 2.8.0
2022-02-28 03:38:14.118011: I tensorflow/stream_executor/plugin_registry.cc:247] Selecting default DNN plugin, cuDNN
2022-02-28 03:38:14.118120: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2022-02-28 03:38:14.867374: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:14.867435: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:14.867501: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:38:14.867540: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:14.867557: I tensorflow/core/framework/model.cc:1774] Waiting for 2560 ms.
2022-02-28 03:38:17.211820: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:17.211878: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:17.211934: I tensorflow/core/framework/model.cc:147] Setting tunable parameter MapAndBatch(id:2):: parallelism to 4
2022-02-28 03:38:17.211965: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:17.211978: I tensorflow/core/framework/model.cc:1774] Waiting for 20480 ms.
2022-02-28 03:38:17.427702: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:17.427751: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:17.427807: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:38:17.427849: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:17.427867: I tensorflow/core/framework/model.cc:1774] Waiting for 5120 ms.
2022-02-28 03:38:22.548024: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:22.548080: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:22.548130: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:38:22.548162: I tensorflow/core/framework/model.cc:1787] Optimized for 1 ms.
2022-02-28 03:38:22.548176: I tensorflow/core/framework/model.cc:1774] Waiting for 10240 ms.
2022-02-28 03:38:32.788310: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:32.788363: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:32.788417: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:38:32.788448: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:32.788478: I tensorflow/core/framework/model.cc:1774] Waiting for 20480 ms.
2022-02-28 03:38:37.692113: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:37.692169: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:37.692223: I tensorflow/core/framework/model.cc:147] Setting tunable parameter MapAndBatch(id:2):: parallelism to 4
2022-02-28 03:38:37.692252: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:37.692264: I tensorflow/core/framework/model.cc:1774] Waiting for 40960 ms.
2022-02-28 03:38:45.229087: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8101
2022-02-28 03:38:53.268617: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:38:53.268674: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:38:53.268727: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:38:53.268758: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:38:53.268786: I tensorflow/core/framework/model.cc:1774] Waiting for 40960 ms.
2022-02-28 03:39:18.652405: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:39:18.652461: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:39:18.652515: I tensorflow/core/framework/model.cc:147] Setting tunable parameter MapAndBatch(id:2):: parallelism to 4
2022-02-28 03:39:18.652544: I tensorflow/core/framework/model.cc:1787] Optimized for 0 ms.
2022-02-28 03:39:18.652555: I tensorflow/core/framework/model.cc:1774] Waiting for 60000 ms.
2022-02-28 03:39:34.228917: I tensorflow/core/framework/model.cc:1873] Starting optimization of tunable parameters with Hill Climb.
2022-02-28 03:39:34.228966: I tensorflow/core/framework/model.cc:1880] Number of tunable parameters: 1
2022-02-28 03:39:34.229020: I tensorflow/core/framework/model.cc:147] Setting tunable parameter ParallelMapV2(id:1):: parallelism to 4
2022-02-28 03:39:34.229048: I tensorflow/core/framework/model.cc:1787] Optimized for 1 ms.
2022-02-28 03:39:34.229081: I tensorflow/core/framework/model.cc:1774] Waiting for 60000 ms.
2022-02-28 03:39:37.490747: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.000798: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.000922: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.000958: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.000988: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.268880: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.268963: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269095: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269120: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269139: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269157: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269176: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.269199: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.270723: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.270755: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1
2022-02-28 03:39:41.270776: I tensorflow/stream_executor/cuda/cuda_dnn.cc:781] Requesting grouped convolution: 1

@harishkashyap - what version of tensorflow as you using? If you use an older version, does it still work? (Trying to diagnose whether it's an issue with your CUDA installation or a regression in TF)

I can't go as far back as 1.3 or below because latest Python 3.9+ don't pull them. I tried using tensorflow 2.0+ versions. Edwardlib still has demo notebooks that use tensorflow.contrib which no longer exists. However without that it's tough to use this library since distributions are missing which are critical.

Same here. I am using 8 GPUS on one machine.
python   3.9.12
tensorflow-gpu   2.4.1
cudatoolkit    10.1.243
cudnn    7.6.5
I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
After 1-hour model started to train.
Epoch 1/10
3/3 - 3389s - loss: 1.0588 - tp: 8669090.0000 - fp: 4140984.0000 - tn: 32559178.0000 - fn: 9680990.0000
Epoch 2/10
3/3 - 1s - loss: 0.5463 - tp: 12156823.0000 - fp:<PHONE_NUMBER> - tn: 36215892.0000 - fn: 6193257.0000
Epoch 3/10
3/3 - 1s - loss: 0.2260 - tp: 17219144.0000 - fp:<PHONE_NUMBER> - tn: 36224584.0000 - fn: 1130937.0000
Epoch 4/10
3/3 - 1s - loss: 0.1627 - tp: 17855504.0000 - fp:<PHONE_NUMBER> - tn: 36230252.0000 - fn:<PHONE_NUMBER>
Epoch 5/10
3/3 - 1s - loss: 0.1416 - tp: 17883972.0000 - fp:<PHONE_NUMBER> - tn: 36234520.0000 - fn:<PHONE_NUMBER>

error this because

test.record and train.record not create or have 0 byte in object_detection/
not have train_labels.csv and test_labels.csv in object_detection/image/
not have images train/test folder in object_detection/images

Solving ..
try run

python xml_to_csv.py
python generate_tfrecord.py --csv_input=images/train_labels.csv --image_dir=images/train --output_path=train.record
python generate_tfrecord.py --csv_input=images/test_labels.csv --image_dir=images/test --output_path=test.record

after edit .config file ( such as use faster_rcnn_inception_v2_pets.config) and run training model again with
python model_main.py --logtostderr --model_dir=training/ --ipeline_config_path=training/faster_rcnn_inception_v2_pets.config
then it will run successfully ..

same issue on Ubuntu 20.04 with RTX3090. TF was installed using anaconda.

I got the same problem, Did anyone solve it?

Same issue here for TF-gpu 1.14. Any fix? I would really like to use this tf version as i want to run old code an i dont want to fiddle around with tf.compat.v1.

error this because
1. test.record and train.record not create or have 0 byte in object_detection/

2. not have train_labels.csv and test_labels.csv in object_detection/image/

3. not have images train/test folder in object_detection/images

Solving .. try run
1. python xml_to_csv.py

2. python generate_tfrecord.py --csv_input=images/train_labels.csv --image_dir=images/train --output_path=train.record

3. python generate_tfrecord.py --csv_input=images/test_labels.csv --image_dir=images/test --output_path=test.record

after edit .config file ( such as use faster_rcnn_inception_v2_pets.config) and run training model again with python model_main.py --logtostderr --model_dir=training/ --ipeline_config_path=training/faster_rcnn_inception_v2_pets.config
then it will run successfully ..

I had this issue, changed dataset's, and the issue disappeared. So at least in my case, something was wonky with the dataset. I'll update this comment when I figure it out.

在带有 RTX3090 的 Ubuntu 20.04 上也有同样的问题。TF 是使用 anaconda 安装的。

我也是

请问你解决这个问题了吗

在带有 RTX3090 的 Ubuntu 20.04 上也有同样的问题。TF 是使用 anaconda 安装的。

我也是

请问你解决这个问题了吗

I am facing the same issue. Have you solved it?


Include api version when fetching within sanity

Reproduce by navigating to 'News'

Need to review how we use the sanity client for fetching in general, I see we are not consistent with it.
This should probably be handled by web/lib/config.ts, which we use for the preview hook and PortableText in web/lib/sanity.ts - maybe make a reusable wrapper for the client where this config is passed?

Added api version isuniqueWithinLocale.js where its missing

We should at some point add the api version to a shared config or something


Remove BPF_F_CURRENT_CPU macro

LGTM
This is to support older kernels (4.4) that don't support the special value BPF_F_CURRENT_CPU.


Add slogan "Meltano does for data what docker did for containers"

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/316
Originally created by @bencodezen on 2019-01-22 20:42:10

This should be integrated into the docs site

View 3 previous comments from the original issue on GitLab


clustering predictions beyond redshift 1.5

Hello,
I am using this fantastic tool you have developed. Would it be possible to extended the predictions of clustering to redshift 2 or 3 ?
Indeed ongoing spectroscopic surveys are gathering large samples there and having this tool to interpret these would be great.
Best wishes

Thanks for the inquiry. Unfortunately, the current emulator is built based on simulation data below z<1.5, and higher redshifts are not supported. We are now performing the 2nd series of simulations, and they will cover up to z~3. I expect that it will take some time, but hopefully we can make it in the near future.

Thank you, I am looking forward to it.


Fix #125 invalid character (backspace) in output

If you clear text in a UITest by pressing backspace, Xcode logs output into console with the UTF-8 character \x08.
This results into the following error message when parsing JUnit output:
org.dom4j.DocumentException: Error on line 496 of document  : An invalid XML character (Unicode: 0x8) was found in the element content of the document. Nested exception: An invalid XML character (Unicode: 0x8) was found in the element content of the document.

To fix this, I’ve replaced the character with the HTML Entity counterpart:
t =    10.54s Type '' into "TestTextField" TextField

This fixes #125.
As I didn’t write any production Objective-C code yet as I started with Swift only please let me know if there’s anything I can improve or needs to be changed.
I couldn’t find any specific unit test for this method to adjust. If there is any I would try to make it test my change. Otherwise I could also provide an XML file with one test case where this specific problem occurs.

/azp run

Is this still an issue?


Push from_name and lookup down into two intermediary temporary classes and change behavior slightly

This splits the way we do lookups into two ways based on the class

Functions and classes – these always require an object "tag"
All other objects – we assume these are always single-object apps

The intermediary classes are temporary – we're giong to push everything down into each specific subclass shortly. This is just some prep work that makes it easier.
It relies on some analytics I did which indicates that there's a strict correspondence between object types and lookup methods. If you analyze all calls to AppLookupObject, function & class lookups always have the "tag" set, but all other entities never have the "tag" set. This basically ends up enforcing this, and is a step towards specializing the implementation of each class (which later will allow me to remove the single-object app code from the client)

Passes all integration tests as well.

@prbot approve

Will revert this if it's causing issues. I'll start working on the server-side implementation of ClassGet and FunctionGet next.


Using custom FaceNet model combined with a Siamese Architecture

Hey
The FaceNet model is performing well as of now but I would like to improve this by feeding the vectors that we get from FaceNet into a Siamese like network which would print out the difference between two images. So essentially I would have two FaceNet heads of whose result I will take the difference of and feed it into a few more layers and which will give me the distance. I am wondering how could I go about this with the level of customizability that I am talking about.

https://www.pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/
It is 3 series article. Read them all, and rule!

https://www.pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/
It is 3 series article. Read them all, and rule!

@konstantinos779 Thanks, would definitely take a look at this.

@konstantinos779 Thanks, would definitely take a look at this.


E-mail confirmation behaviour

When register or login and user is not verified, we should redirect him on a specific view

Done


How to scroll to the bottom of a target element but only as far as not to scroll the top out of view?

How to scroll to the bottom of a target element but only as far as not to scroll the top of the element out of view?

Basically a "try and scroll this whole div into view but only as far as not hiding the top of it" scenario.

You should ask these questions on sites like StackOverflow, there are many more people available there. Github issues are only for bug/feature/enhancement requests.
Regards

But it's a question about using your library? I'm asking whether this is possible with your library. Or if you have any insight into how this can be achieved? If the answer is 'no' then this ticket could lead onto an enhancement request.
Asking the author of the plugin I have a question about seems far more logical than asking a community of people who may or may not have experience of using your plugin.


"Scope to element" would open a new editor with only the specified scope (function/class) visible

Steps:

File x.py has multiple classes, methods, functions.
Wish to edit a specific method or class somewhat extensively or with focus.
a. For example, to be able to search, replace, jump to top/bottom of document, etc., while "Staying within the scope" of the method/class.
Right click the name of the class or method and choose Scope to element.
A new (virtual) editor is opened and populated with the source of the specified element. Any changes in this editor are as if done through a "proper" editor with a view to the entire parent file, and vice-versa.
a. But, in comparison, all other editor commands affect only the source of the "element" scoped into, even when they should affect "the whole document".

Process suggestion:
Unsuccessful (but reasonably well defined) backlog candidates should be put in a separate milestone for:

Future reference for backlog idea mergers or in a case a good usecase arises
Inspiration for extension authors, where the functionality is at all possible to provide through an extension.


Most recent lectures require preamble installs to run on Google Colab

We need to update:

tax_smoothing_1
tax_smoothing_2
tax_smoothing_3

with
pip install quantecon cell blocks.

Well spotted, thanks @mmcky !
@AnjuJoon, could you please address this (and try to remember to add in future)?
It would be great if we had a test for successful colab execution, although I can't think of any way to do it...   Perhaps it's worth opening an issue, @mmcky , as to the need for such a test if we can work out a good way?

@AnjuJoon Please cancel that request --- I just saw #537

Well spotted, thanks @mmcky !
@AnjuJoon, could you please address this (and try to remember to add in future)?
It would be great if we had a test for successful colab execution, although I can't think of any way to do it... Perhaps it's worth opening an issue, @mmcky , as to the need for such a test if we can work out a good way?

@jstac Sure, I will remember that (missing that in new lectures was a big blunder from my side :( Won't happen in future)

This has been fixed in #537


Wallet address is not prefilled on Banxa fiat on-ramp form

Problem

Create a new wallet

Click on Buy ADA

Go back to copy & paste your address here

Solution

Work with Banxa to let the address field in their form to be prepopulated by URL query string
Edit the Buy ADA button link href to include the known address.

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://github.com/Jostino/RimWorld-it/releases/tag/fix
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx


Cannot ENABLE any API when there is a pending request to DISABLE any API

I've been trying to use the Cloud Datastore API with Java, however it's been throwing this error:
com.google.gcloud.datastore.DatastoreException: The Cloud Datastore API is not enabled for the project <project-id>
However, when I try to enable it in the Google Cloud Console API Manager (https://console.cloud.google.com/apis/api/datastore.googleapis.com/overview), it shows another error:

This is persistent and happens every time, and I don't have any pending requests to disable any APIs.
The end result?  Absolutely no way to use the Cloud Datastore API!!

Thanks for the report, we've escalated to the team that handles service management, I'll let you know when we get more information.

Just tried again this evening and the great news is that it's been resolved.  Thanks for looking into it.

@Queatz Would you be willing to share the project ID you were using when you saw this error? It'd help us ensure this issue doesn't recur. You can email me at<EMAIL_ADDRESS>Thanks!

@eddavisson yup - sent it your way.


Document the *_encode_signing_data_to functions

If one writes signing data using one of the two
*_encode_signing_data_to functions then creating the message to sign is slightly nuanced and different for each of the functions. For Taproot one must use a specific tagged hash and for ECDSA one must use a sha256d hash.
Add documentation that explains the hashing requirements for each function.
Close: #2703

May have prevented some of the debugging outline in #2701

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.131%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
19195

Relevant Lines:
23090

💛  - Coveralls

Can one-ack merge since this is docs only.


FR: Add indexing option to additional parameters

These parameters should also be indexed:

addReqHeader
addResHeader
delReqHeader
delResHeader
setReqHeader
setResHeader

The project moved to https://github.com/docker-flow/docker-flow-proxy. Please recreate the issue there if it's still valid.


feat: Off-peak NeTEx change

Description
NeTEx created with time restrictions include an additional FareStructureElement to include the additional restrictions to access rights of product. This should only apply to the FareStructureElement that includes the qualityStructureFactors. The other FareStructureElements relating to access (i.e. include GenericParameterAssignments) should remain unchanged.
Testing instructions
Currently this FareStructureElement includes:
<TypeOfFareStructureElementRef version="fxc:v1.0" ref="fxc:access"/>
This needs to be replaced with:
<TypeOfFareStructureElementRef version="fxc:v1.0" ref="fxc:access_when"/>
Code review required

Waiting for Steven to review

Ready for review


Nullpointer Exception in PDF build due to invalid href

Expected Behavior
Error message should be displayed that referenced item cannot be found.
Actual Behavior
Nullpointer Exception is thrown if output is PDF, error message is shown if output is HTML5
Possible Solution
Catch the exception and issue an error message.
Steps to Reproduce
Compile testmap.ditamap from this Gist
Copy of the error message, log file or stack trace
Error log in Gist
Environment

DITA-OT version: 3.6.1
Operating system and version: Win32NT             10.0.18363.0 Microsoft Windows NT 10.0.18363.0
How did you run DITA-OT?
dita.bat -f pdf -i .\testmap.ditamap -d
Transformation type:
PDF

@Snafu I don't understand what the DITA should be doing based in the Gist because it doesn't include all files.
<topicref keyref="content"/>
<keydef keys="content" href="#./sdfsf"/>

would resolve to
<topicref href="#./sdfsf"/>

But

Topicrefs cannot reference topic element, only topics.
This tries to reference topic element with ID sdfsf. Maps cannot have element level IDs.

Closing this. Please add a full set of DITA files if you need this to be reopened.


Using request DTOs instead of domain objects in APIs

Summary
While going through the codebase I observed that in controllers, at many places, we are using domain objects instead of a separate DTOs for handling API requests.
For example https://github.com/appsmithorg/appsmith/blob/c1b498f6b81dd67d707fd45c7dd40d6990486438/app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/UserController.java#L54
Moreover, there are no validations in the object. (Ideally, they should not be present on domain objects)
Motivation

The first thing is using DTOs for requests separate the resource/controller layer from the service layer. DTO and domain objects can change independently from each other.
We can avoid littering our domain objects with Jackson and Java bean validation annotations.
Having a request DTO will help a backend engineer or a contributor to quickly understand what this API needs instead of understanding the domain object (In most of the cases domain objects contain some extra fields populated in the service layer).

Additional Context
For example, I wanted to explore the backend system locally (without digging into/running the frontend) so to access many of the APIs I had to create a User through the UserController APIs. But when I went to the create method I had to understand by looking into the User domain object that what fields to set, what is ignored etc. DTOs make this thing simple.

@pc9795 Thanks for opening this issue. You do make a good point.
I have 1 question though: Does creating a request DTO for each API mean that now we'll have to perform a deep copy of the object from the DTO to the domain object on each request? Vice-versa, will we have to perform a deep copy from domain object to DTO ( for GET API calls )? Is there a way to avoid all this copying?
Also, by any chance is there a reference code base you can point me to

@mohanarpit No need for manual copying. You can look into https://mapstruct.org/

Some code reference where I have used it: https://github.com/pc9795/portfolio/blob/master/src/main/java/com/prashantchaubey/dto/mappers/ContactMapper.java

@pc9795 Thanks for the pointer to mapstruct. #TIL
On reading a little more about it, in the generated code section, I can see that MapStruct implicitly creates the implementation class to copy data from the domain class to the DTO. This is the copy that I'm trying to avoid. It creates unnecessary garbage in the JVM and slows things down. A lot of this doesn't show up in the early days, but with a little load on the machine, the cost of these copies become apparent very quickly.
By design, the Appsmith backend server should be as light and performant as possible. Hence, if there's a better way to implement the "separation of concerns" principle that you suggested, I'd be more than willing to do that.

hmmmm, I have not thought in this way. I only used Map struct or wrote mappers by scratch.  For the starters there is this excellent article. I can dig in more but not sure as of now. But yeah using DTOs is debatable but IMO keeping things separate has its own benefits.

Closing this issue for now since we are not prioritizing this code change.


Added support for Postgres v14

What does this PR do?
This PR adds support for Postgres v14
Motivation

Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached

Thanks @mawais54013! Looks great. Can you please attach the graphs of KPIs from our integration environment?

PG14 graph in staging env:

https://a.cl.ly/NQuNbLvv
Sample host for PG 14:
https://a.cl.ly/YEurKqpQ

Staging Environment Comparing postgres v14 to v13.3 on agent v7.34:
Agent health:

https://a.cl.ly/OAu6Ok28
Core metrics:

https://a.cl.ly/lluEZPXl
Database health:

https://a.cl.ly/6quZkjXJ

@mawais54013 Internal metrics are looking, the only thing that stands out to me is this new error we're seeing in version 14 and it's occurring frequently.

I think it's worth investigating to see what the source is. I'd suggest looking at the agent logs for clues and then look at our  core queries (i.e. metric, sample, and activity) in the integration to see if any of those may be the problem.


Getting error while building code

Hi, I am using the latest version of zoom (1.0.3). While building project I am getting following error:
Execution failed for task ':app:checkDebugAarMetadata'.
> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
   > Could not find :commonlib:.
     Searched in the following locations:
       - file:/Users/sam/Documents/Office/flutter/.pub-cache/hosted/pub.dartlang.org/zoom-1.0.3/android/libs/commonlib.aar
     Required by:
         project :app > project :zoom
   > Could not find :mobilertc:.
     Searched in the following locations:
       - file:/Users/sam/Documents/Office/flutter/.pub-cache/hosted/pub.dartlang.org/zoom-1.0.3/android/libs/mobilertc.aar
     Required by:
         project :app > project :zoom

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Seems like it is not able to locate commonlib and mobilertc plugins used by the zoom plugin.
Please guide.

After install the library, must run the follow script to get some sdk stuff for the first time:
flutter pub run zoom:unzip_zoom_sdk

Thanks for the quick response. This solved the problem.

For using it on a real device and for Simulators on an M1-Chip (ARM) based MacBooks. you need to use flutter pub run zoom:unzip_zoom_sdk.
For simulators of Intel-based Macs, you need to use flutter pub run zoom:unzip_zoom_sdk dev

According to documentation i run flutter pub run flutter_zoom_sdk:unzip_zoom_sdk but still same issue

Could you show me your console log after run“flutter pub run zoom:unzip_zoom_sdk”，  Please make sure you run the right command "flutter pub run zoom:unzip_zoom_sdk"，not "flutter pub run flutter_zoom_sdk:unzip_zoom_sdk"

According to documentation i run "flutter pub run flutter_zoom_sdk:unzip_zoom_sdk" but still same issue

did you solve the problem? I have the same issue

flutter pub run zoom:unzip_zoom_sdk


Passing width and height to swatches in SketchPresetColors

As far as I'm aware, <SketchPresetColors /> only accepts the props Colors, onClick, and onSwatchHover. Trying to add styling to that element modifies the container holding the swatches rather than the swatches themselves. The only way to modify the size of the swatches is to go into SketchPresetColors.js and directly modify the swatchWrap CSS style, which I'm trying to avoid at all costs. I'd really appreciate the ability to just have width and height (for the swatches) as props of SketchPresetColors.
For reference, I'm going off the advice from https://github.com/casesandberg/react-color/issues/343

Hmm. Unfortunately there is no easy way to do this right now without duplicating the component and making your own and changing those values.
There is a really hacky way of doing this with CSS until I can come up with a better fix. You can target those swatches like this:
.sketch-picker .flexbox-fix:last-child div {
    width: 40px !important;
    height: 40px !important;
}

Hmm. Unfortunately there is no easy way to do this right now without duplicating the component and making your own and changing those values.
There is a really hacky way of doing this with CSS until I can come up with a better fix. You can target those swatches like this:
.sketch-picker .flexbox-fix:last-child div {
    width: 40px !important;
    height: 40px !important;
}

it really solve my problem. thanks


New-PartnerAccessToken  throws error for -UseDeviceAuthentication if web app instead of native app is used: AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'

If -ApplicatonId points to a confidential client (web application) error will be thrown because
New-PartnerAccessToken does not admit -Credential paramenter with -UseDeviceAuthentication switch.
The New-PartnerAccessToken with -UseDeviceAuthentication switch therefore only works if app is a public client (native app)
This logic is undocumented and possibly a bug as confidential clients are being recommended for implementing Secure Application Model in general.
Steps to reproduce
PS C:\Users\morcinim> $token = New-PartnerAccessToken -ApplicationId $appId -Scopes https://api.partnercenter.microsoft.com/user_impersonation -Tenant $tenantId -UseDeviceAuthentication
WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code D8RNFN5HH to authenticate.
New-PartnerAccessToken : AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.
Trace ID: c252087f-8a47-4029-844a-d93f641d3800
Correlation ID: 773a28fd-e5bf-46d9-91b3-ccf707ae8c3f
Timestamp: 2019-09-26 09:11:51Z
At line:1 char:10

$token = New-PartnerAccessToken -ApplicationId $appId -Scopes https:/ ...

     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [New-PartnerAccessToken], MsalServiceException
FullyQualifiedErrorId : Microsoft.Identity.Client.MsalServiceException,Microsoft.Store.PartnerCenter.PowerShell.Commands.NewPartnerAccessToken

Expected behavior
Same as when ApplicationId refrerences native app:
PS C:\Users\morcinim> $token = New-PartnerAccessToken -ApplicationId f0ab7600-4dd1-yyyy-zzzz-f8527d4dxxxx -Scopes https://api.partnercenter.microsoft.com/user_impersonation  -Tenant $tenantId -UseDeviceAuthentication
WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DQUM484H3 to authenticate.
PS C:\Users\morcinim> $refreshToken = $token.RefreshToken
PS C:\Users\morcinim> $azureToken = New-PartnerAccessToken -ApplicationId f0ab7600-4dd1-yyyy-zzzz-f8527d4dxxxx  -RefreshToken $refreshToken -Scopes 'https://management.azure.com/user_impersonation'  -Tenant $customerTenantId
$graphToken = New-PartnerAccessToken -ApplicationId f0ab7600-4dd1-yyyy-zzzz-f8527d4dxxxx  -RefreshToken $refreshToken -Scopes 'https://graph.microsoft.com/.default'  -Tenant $customerTenantId
Actual behavior

What is the behavior observed?
When ApplicationId refrerences web app:

AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.
Diagnostic logs
Environment

Please share additional details about your environment.
Version
Install-Module -RequiredVersion 2.0.1909.5 -Name PartnerCenter -AllowClobber -Scope CurrentUser -Force
Install-Module -RequiredVersion 6.13.1 -Name AzureRm -AllowClobber -Scope CurrentUser -Force
Install-Module -RequiredVersion 5.8.3 -Name AzureRm.profile -AllowClobber -Scope CurrentUser -Force

Makes sense to me, should be fixed!

I have this issue as well. Not sure when it occurred. Using a token acquired through REST directly works

When you specify the UseDeviceAuthentication parameter you are telling the module to use device code authentication. This particular flow only supports a public client, which is commonly referred to as a native client. If you try to use a web application with this approach the client secret value, even if it is included in the contents when request an access token, will be ignored by Azure Active Directory.
With the release of Partner Center PowerShell 2.0 several changes were made to the authentication to ensure that the best solution for this particular problem would be available for PowerShell 5.1 and PowerShell Core. The default authentication behavior will launch a web browser (e.g. Edge, Chrome, etc.) where you can complete the request. This approach supports both a native and web application because it is leveraging the authorization code
Since this is a limitation of the device code flow no changes can be added to this module to address the issue. If you are planning to use the UseDeviceAuthentication parameter and a custom Azure Active Directory application it must be a native application.

Great, thank you for the explanation. It would be great to include a note in help file of New-PartnerAccessToken powershell command that UseDeviceAuthentication flag implies ApplicationId parameter has to point to a native app to avoid confusion.


test: use "makeSut" for test setup around

Conveeeeeeentions

#307  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @davidlj95 and the rest of your teammates on  Graphite


Take over Google's wdl_runner

This copies the code from https://github.com/googlegenomics/pipelines-api-examples/ and renames "wdl_runner" to "cromwell-on-google" to be more specific.

@mbookman FYI

The most recent commit removed runners/cromwell_on_google/workflows/wdl_pipeline.yaml.
That is needed to run workflows from gcloud.
Can that file be restored?

@mbookman I updated this, afaict everything works (with Cromwell 29). Would you care to review?

@mbookman Updated the PR based on our discussion earlier today

Good eye, @mbookman -- thanks for catching these. I think I addressed all your points.

Thanks @mbookman


fix: fixed swift lint

:tada: This PR is included in version 1.0.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Promo banner template

Designed to be easily interchanged for any event/market. The Title, subtext and image are the elements that will need to be switched out.
Desktop https://www.figma.com/file/buzGz2v44DfkUQf112KlcO/Landing-Page-Complete-Design?node-id=3300%3A98962
Mobile https://www.figma.com/file/buzGz2v44DfkUQf112KlcO/Landing-Page-Complete-Design?node-id=3324%3A0

Few things I noticed:

Lines missing in the top left
Gradient overlay should sit above image, not the lines otherwise it washes out the colours of the diagonal lines
Padding in button to left and right of text looks too big


provide way to verify  images in quay

Is your feature request related to a problem? Please describe.
we need way to verify images in quay is correct.  This is a similar to verifying gold master on spectrum scale, We document md5sum for package.
Describe the solution you'd like
Evan suggested following
checking what is uploaded to quay, then you might want to know the build ID, git commit, git tag/version, etc
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
nstalled v1.1.0 on openshift 4.2 via OLM.  it pulled following images. how to verify that images are correct?
ibm-spectrum-scale-csi-driver 8b6cb1a1743e
ibm-spectrum-scale-csi-operator ced0267b0a45
(edited)
Evan Dunn:working_remotely_0671922247:  2 hours ago
not sure what you mean, or are trying to accomplish
but here is how docker works -
docker manifest inspect quay.io/ibm-spectrum-scale/ibm-spectrum-scale-csi-operator:v1.1.0
this is the fat manifest and lists the manifests per platform
if you take the linux/amd64 manifest and then inspect that...
docker manifest inspect quay.io/ibm-spectrum-scale/ibm-spectrum-scale-csi-operator@sha256:6dfeb08e2e377efc147f9a5b793ac0dbaecc9c1e1a3bafb9a6170d7ab97326c0
notice: you have to reference the manifest by digest as that is how the fat manifest references it
this platform specific manifest then has the digest of the image itself
"digest": "sha256:ced0267b0a4537376ff677f5c25b0532c69896aedd44ffc3f18b9150728ba927"
and you can see that matches with what you list (you are showing the truncated hash)
Evan Dunn:working_remotely_0671922247:  2 hours ago
this doesn't really serve much purpose as we could easily enough swap the tag to some other manifest (aka what I did to add z & p to the x only manifest)
tags are just references to the digest, digests reference a manifest or an underlying image
from the OLM perspective John has it pointed to just the v1.1.0 tag -- not the digest itself
Sanjay Gandhi  2 hours ago
I wanted to validate that images uploaded on quay is correct and OLM is pulling correct images . I don’t know it is required for docker images  but we use md5cksum for rpms.
Evan Dunn:working_remotely_0671922247:  2 hours ago
the digest is the sha256 of the image, so its akin to checking md5
Evan Dunn:working_remotely_0671922247:  2 hours ago
for checking what is uploaded to quay, then you might want to know the build ID, git commit, git tag/version, etc
sadly, we don't dynamically add that -- we should

Implemented a fix for this which adds a git_commit  label to both the driver and operator images.

Solution has  been merged.


Issue 6766 - Forward reference error for default struct/class arguments

https://issues.dlang.org/show_bug.cgi?id=6766
Also fixes issue 11166 because the issue root is same.
Issue 11166 - Forward reference error when alias of template instance is private

Auto-merge toggled on

This has introduced a regression: https://issues.dlang.org/show_bug.cgi?id=15726


docker attach all for container output

During Docker Hack Day we discussed options around Docker having the ability to automatically attach to all containers and provide their output to a single stream that can then be sent to an external service such as syslog.
A couple of scenarios came about:

leave it up to a client application to attach to the streams; but this needs to be done with care in order to not lose any output between starting the container and attaching to the log stream

implement the logic within docker; but how to provide it in a general enough way so it can be used for miscellaneous use-cases

Additionally, Docker has two facilities for getting information: events and attach.  The events command output appears to be for logging docker actions and attach appears to be for output generated by the containerized application.  Is extending attach the right place for this feature?
Finally, attaching to stdin doesn't seem to apply here, should "attach all" explicitly exclude stdin?
Please share your thoughts.
Thanks!

I think the solution here is the docker logging drivers. In 1.6 you can send all container logs to syslog.
Journald has been added sense then, and others are being developd by other parties (such as fluentd).
I think we can close this one now.

In fact, I think I'm going to go ahead and close it, but can discuss if you don't feel like this solves the issue.


Modify install command to perform 'reinstall' if invoked with no arguments

This should be the default; all other command line tools (bower, npm, etc) default to this behavior.
It's really annoying have to explain to people repeatedly that typescript is special and doesn't work the same way as all the other standard tools.
It should be a trivial change; instead of:
Tok:hello_world doug$ tsd install

-> an error occured!

pass at least one query pattern
VError: pass at least one query pattern

Simple detect no arguments were passed and run reinstall instead.

I started it here https://github.com/Diullei/tsd/commit/4671696899fb089340d652c10d41189693cbed84

I have just run into this myself. I noticed that gulp/tsd.js is missing. Was this removed in the most recent update of the generator? A work around for the problem was for me to run tsd reinstall before running gulp.


Add missing inFlightConn TCP middleware CRD

What does this PR do?
This PR updates the middlewaretcp CRD to have access to missing inFlightConn middleware.
Motivation
Have the possibility to use more TCP middleware
More

[X] Yes, I updated the chart version

Can changes to CRDs be properly announced? Majority of users aren't aware of this change, because it's patch release. And helm doesn't upgrade CDRs by design, so upgrade should be applied manually via kubectl. See how kube-prometheus-stack handles CRDs upgrades https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#upgrading-chart


create-certificate to also check "additional names"?

v nice lib - thank you @brandonweiss
it would be nice to have create-certificate also check "Additional names" in addition to the DomainName
similar to how  create-distribution checks on Aliases
use case: i would like to reuse the same certificate across my dev.*, stage.* and www.* environments (rather than having to configure certificates for each env)
i can send a PR if you would consider this behavior.
cheers.

@mvberg Thanks!
Hmm… 🤔
So the way I designed it is when the certificate gets created, the primary domain name is whatever you put as the domain in your configuration (e.g. domain.com), and the “additional names” is a wildcard on that domain (*.domain.com). So… shouldn’t it just work?
Or is the issue that you already have a certificate that was manually created, and what you’re using as the domain name doesn’t match the primary domain on the certificate, but it is in the “additional names”?

Or is the issue that you already have a certificate that was manually created, and what you’re using as the domain name doesn’t match the primary domain on the certificate, but it is in the “additional names”?

exactly this. i have a "wildcard" certificate that i would like to reuse. but thinking about this again, what about adding a option to set the certificate ARN in the .discharge and doing something like:
module.exports = {
  title: "Create certificate",
  skip: async (context) => {
    let domain = context.config.domain

    if (context.config.certificate_arn) { // existing certificate
      context.certificateARN = context.config.certificate_arn;
      return "Using Certificate from config"
    }

Before I shipped Discharge I waffled back and forth about wether or not to expose configuration options for declaring the IDs of certain, possibly existing resources, like a certificate or distribution. Ultimately I decided not to, in part because it wouldn’t be necessary most of the time, but also because I’m generally allergic to adding options like this unless absolutely necessary—I prefer convention over configuration.
I think there might be a good way to solve this, though, without configuration, I’m just not sure I have all the right information yet. For your existing certificate, can you tell me exactly what the values of DomainName and SubjectAlternativeNames are?
Thanks!


Allow FQDNs in usernames

As noted in #198, we're seeing that folks using the
IndieWeb-to-Fediverse bridge https://fed.brid.gy/ as their Mastodon
Server aren't detected as valid profile URLs for Mastodon, resulting in:

No handles were found on your profile

We can make sure to include the . as a valid character in a URL,
allowing us to extract @{fqdn}@{fqdn}.
Closes #198.

Merged, tested and live. Thanks for improving fedifinder.

Thank you for the speedy merge, I can see it working now 👏🏽


switch to Alpine as environment distribution

I think the tests for this fail, because Alpine by default installs QEMU 6.1.0. If you try to start RustyHermit with the MicroVM on this version, the kernel will not boot and the QEMU process just gets stuck (which is why there is no log output and why the delete command claims that the container was never stopped)

Ah, I understand. In this case I have to open an issue at libhermit.

Yes. This might also be an issue with the bootloader because that does not emit any log messages either.


Rewrite Sprightly's engine

Make sure that the rewrite achieves the following:

The engine should be written in TS.
Follow a functional approach to implement the engine, as opposed to the OOP one implemented currently.
Use JS's new method .replaceAll() to implement the transformation logic.
Currently, if you have the views directory set to ./views and the dir is laid out as follows:

./views
+- file.spy
+- subfolder
+---- nested-file.spy

If subfolder/nested-file.spy wanted to include file.spy as a partial, it would have to reference it as follows ./file.spy instead of ../file.spy (since it's one level up in the tree) because the files are resolved relative to the pre-set ./views directory. You need to fix this behavior and implement the correct one mentioned first.

Implement a good error tracing strategy.
Implement a caching strategy.
Support escaping syntax.
The API should take the form:

function sprightly(
  entryPoint: string,
  data: Record<string, string>,
  options: { cache: boolean }
): Promise<string> {
  ...
}

Support accessing nested properties. Example:
Given

{{ nested.property }}

and
const data = {
  nested: {
    property: "Osid's awesome"
  }
}

should resolve in:

Osid's awesome

Support express through the .__express property. Read more here and here.

These are the change in requirements after writing the initial tests:

There's a keyFallback option that is used as follows:

If a referenced key is not found in the data object and keyFallback is not set, then the resolved value should be an empty string.
If a referenced key is not found in the data object and keyFallback is set, then the resolved value should be the value assigned to the option.

There's a throwOnKeyNotfound that does the following: If a referenced key is not found in the data object and this option is enabled, then the engine should throw a custom error containing a string with tracing info (can be used to render in express).
We need to support array indexing syntax as well. So this syntax for example is supported:

const data = {
 property: {
   nested: [{ anotherProperty: ["foo", "bar"] }]
 }
}

// in template
property.nested[0].anotherProperty[1] // results in "bar"

If the entry point isn't found, it throws an error containing that.
If a referenced component isn't found, it throws an error containing tracing info for that.
I will comment the specific requirements for supporting escaping later when I know them 😃.

@OsidAbu-alrub.

What can cause errors in the application? Missing components or wrong file paths for example?


improve dtr index page

fix #1766
I also want some screenshots added back into here, but not these huge ones that prevent you from seeing all the contents. Something more specific and arranged like the left-right hipster marketing pages ;)

Who would tell that @vikstrous has some marketing chops? Nice!


Scaling issue

While running the command
pyfractal.GUI().run() from jupyter notebook on a windows machine, the canvas generated is not scaled as per screen size. Pinch too zoom did correct the canvas size later, put initial left side canvas was too big to see the output.
Maybe the canvas can be adjusted as per the window size of the user.
Thanks

Can reproduce, this is a breaking issue. The author @deut-erium should fix it ASAP.


Change approach to test cleanup to maintain go 1.13 compatibility

Signed-off-by: Gary Brown<EMAIL_ADDRESS>

why do we need 1.13 compatibility?

We use it in our internal build system.


Firefox

What old version of FF is supported by framework?

last 2 versions of browser

Are you planning on supporting later versions? Example, 29 version.

No, use your own babel config

I tried, but I get error 'SyntaxError: missing variable name' in FF with use any import function from 'effector'

It is enough to compile node_modules with the correctly configured preset-env


[BUG] cannot execute binary file: Exec format error after install on ubuntu/aarch64

I performed a fresh install today and cannot execute protostar due to an 'exec format error.' I've tried reinstalling and have the same issue. I don't see an option to build from source, so have been unable to try that.
uname -a returns the following for my system:
Linux a20eec2a910c 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 12:56:31 UTC 2022 aarch64 GNU/Linux
Protostar version

The latest version (I cannot execute protostar -v but I grabbed the latest from the install script as of today).

Name
Version

Protostar
PROTOSTAR_VERSION

Cairo-lang
0.8.1

Git
2.30.2

Current Behavior
protostar -v errors with the message bash: /home/vscode/.protostar/dist/protostar/protostar: cannot execute binary file: Exec format error
Expected Behavior
protostar -v should return the version of protostar
Steps to Reproduce

Install protostar: curl -L https://raw.githubusercontent.com/software-mansion/protostar/master/install.sh | bash
Run protostar: protostar -v

Possible Solution
I assume this is because I'm running in a container on ARM architecture (aarch64) vs amd64.

@threepwave
As a temporary solution, you can:

Set up a development environment
https://github.com/software-mansion/protostar#setting-up-environment
Run poe build
Move the dist directory to the desired location (e.g. ~/.protostar)
Add DESIRED_LOCATION/dist/protostar to PATH

Thanks, that workaround worked!


If router-link pass undefined should it render element a without href attribute

I have a menu list , one of them don't have href data
when I pass undefined to router-link , it can't show correct on screen .
https://jsfiddle.net/9r6xhqbp/19/
If I pass a undefined to router-link , should it give me a element without href and click binding ?
because I don't want click event ( I can use e.preventDefault ) , and without href attribute , because with SSR will have a weird href ....
I checked w3c , a don't have href is valid . ( I think so )
thanks .

Hi, thanks for filing this issue. You can simply use an <a> tag with v-if: https://jsfiddle.net/8xzqu3f9/
as <router-link> is only meant for in-app navigation.

@fnlctrl  thanks, your suggest is same with me .
but , now you need maintain two same value variable in template .
jsfiddle just a simple example , when in production it more complex , maybe with a submenu or  dropdown , that will be more problem with maintain period .
If give a empty string or give undefined , just return  without href and click event will not be easy understand or look better or easier maintain ?
anyway , grateful to this great framework .

Well, the current behavior is to protect developers from typos or errors, as you can see an error when passing an invalid to prop:
(found in <RouterLink>)```

For complex templates, I think it would be better to create a component to decide whether to render a `<router-link>` or a plain `<a>` tag: https://jsfiddle.net/cfc9t3oa/

@fnlctrl  I think you're right.
protect developers is also important .
I will following to do a component outside router-link let it easier maintain .
thank you .


Support status field for subscription filters

An admin is requesting for us to support the status field so issues set to certain statuses can be matched with the subscription.
Currently the data retrieved from get-create-issue-metadata-for-project does not contain the status field. Because of this, we can have the status field hardcoded in the "filters" on the client, as it is with the "event" choices of the subscription modal.

Hey there, I'd like to work on this one, please

Awesome :tada: Thanks @orlandorode97!


❓Survey: Your experience in this repo

Whether you're a frequent visitor or a relatively new visitor of the repo, please tell us about your experience with our issue support and/or PR contribution process. The survey should only take a few minutes, and you can answer for up to 3 repos:
https://www.surveymonkey.com/r/8QTQTGJ?Source=azure-sdk-for-c

Sone comments:

Why are there (almost empty) samples/ and sdk/samples/ directories?

I prefer a simple makefile to compile source for different architectures.
For small projects like this I personaly find that cmake is overkill.

There are many projects and C examples for Azure IoT Hub. I found it difficult to choose the best fitting one.

Thank you for open source projects!

Tschau
Uwe

Thanks for the feedback @uklatt !
It's useful to see here in real time. If it isn't too much trouble, can you also provide that feedback in the link above (if not done already).
Thanks!

Thank you for your interest in the experience survey. The survey has now been closed.
If you didn't get a chance to provide feedback this time, please look out for future surveys.


Split Pickles.Composition_types.Spec.ETyp.t according to backend

This PR builds upon https://github.com/MinaProtocol/mina/pull/16358, splitting the unified implementation of Pickles.Composition_types.Spec.ETyp.t according to the corresponding backends. This PR unpicks the last major use of the generalized Typ.t from snarky.

!ci-build-me


feat: How can I do more things in the router?

from dotenv import load_dotenv
from fastapi import FastAPI
from langchain import ConversationChain
from langchain.chat_models import ChatOpenAI

from lanarky import LangchainRouter

load_dotenv()
app = FastAPI()

langchain_router = LangchainRouter(
   langchain_url="/chat",
   langchain_object=ConversationChain(
      llm=ChatOpenAI(temperature=0), verbose=True
   ),
   streaming_mode=0
)
app.include_router(langchain_router)

According to this code, if I want to do more things in /chat endpoint, how can I archive it? I mean, I am not just chat, maybe before&after /chat, I need to access db, query user etc, somthing like this.
Currently, maybe we can create another endpoint chat-wrapper to wrap this endpoint and then do other things, but obviously it is not a good way.

Hi! sadly, the functionality of the LangchainRouter is a bit limited at the moment.
Currently, all endpoints created by the LangchainRouter.add_langchain_api_route() method are simple wrappers around chain.acall. To build more complex endpoints, I recommend that you use the default methods which are inherited from APIRouter.
You can refer to this implementation: https://github.com/menloparklab/chatbot-ui/blob/054936947760487b9d8fe4a6bface219b0ad530e/backend/app.py#L101-L114
You can still initialise the LangchainRouter to enable LLM caching of your choice. In the future, I will add db support to this router class.

Ok, thank you for your response! Hope more features in the future.

@ajndkr Hi, I use the code you provided above, but in my project I got error as below, could you please help to digging into what caused this error, also notice that in my code, I use starlette.responses's StreamingResponse will not cause error, but use lanarky's StreamingResponse caused this error.
> Entering new LLMChain chain...
Prompt after formatting:
System: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
Human: Hello
AI: Hello! How can I assist you today?
Human: hi
AI: Hello again! Is there anything I can help you with?
Human: hi
AI: Error communicating with OpenAI
Human: hi
AI: Error communicating with OpenAI
Human: hi
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 2.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 8.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 16.0 seconds as it raised APIConnectionError: Error communicating with OpenAI.

the code I use in my project:
from fastapi import Depends, APIRouter
from typing import Annotated
from fastapi import Request
from fastapi_utils.enums import StrEnum
from lanarky import LangchainRouter, StreamingResponse
from lanarky.routing import LLMCacheMode
from pydantic import BaseModel, constr
from sqlmodel import Session
from starlette.responses import StreamingResponse as SR

from db import models, crud_user
from db.database import get_db_session
from routes.auth import get_current_active_user
from routes.helper import get_bot, generate
from routes.types import UserMessage

router = LangchainRouter(llm_cache_mode=LLMCacheMode.IN_MEMORY)
# router = APIRouter()

class Role(StrEnum):
	SYSTEM = "system"
	ASSISTANT = "assistant"
	USER = "user"

class Message(BaseModel):
	role: constr(regex=f"^({Role.ASSISTANT}|{Role.USER}|{Role.SYSTEM})$")  # noqa: E501
	content: str

class ChatRequest(BaseModel):
	model: str
	messages: list[Message]
	max_tokens: int
	temperature: float

def create_chain(
	model: str,
	messages: list[Message],
	max_tokens: int,
	temperature: float,
):
	from langchain.chains import LLMChain
	from langchain.chat_models import ChatOpenAI
	from langchain.memory import ConversationBufferMemory
	from langchain.memory.chat_message_histories.in_memory import ChatMessageHistory
	from langchain.prompts.chat import (
		ChatPromptTemplate,
		HumanMessagePromptTemplate,
		MessagesPlaceholder,
		SystemMessagePromptTemplate,
	)

	system_prompt = next(
		(message.content for message in messages if message.role == Role.SYSTEM),
		"You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.",
	)
	chat_prompt = ChatPromptTemplate.from_messages(
		[
			SystemMessagePromptTemplate.from_template(system_prompt),
			MessagesPlaceholder(variable_name="history"),
			HumanMessagePromptTemplate.from_template("{query}"),
		]
	)

	chat_memory = ChatMessageHistory()
	for message in messages:
		if message.role == Role.USER:
			chat_memory.add_user_message(message.content)
		elif message.role == Role.ASSISTANT:
			chat_memory.add_ai_message(message.content)

	memory = ConversationBufferMemory(
		chat_memory=chat_memory, input_key="query", return_messages=True
	)

	llm = ChatOpenAI(
		model=model,
		max_tokens=max_tokens,
		temperature=temperature,
		streaming=True,
		verbose=True,
	)

	return LLMChain(
		llm=llm, prompt=chat_prompt, memory=memory, verbose=True, output_key="answer"
	)

@router.post(
	"/chat",
	summary="Langchain Chat",
	description="Chat with OpenAI's chat models using Langchain",
)
def chat(
	current_user: Annotated[models.User, Depends(get_current_active_user)],
	request: ChatRequest,
	db: Session = Depends(get_db_session),
):
	try:
		chain = create_chain(
			model=request.model,
			messages=request.messages[:-1],
			max_tokens=request.max_tokens,
			temperature=request.temperature,
		)
		crud_user.consume_once(db, current_user.email)

		# $$$$$$$$$$$$$$$$$$This will NOT cause error$$$$$$$$$$$$$$$$$$$
		# result = chain.run(request.messages[-1].content)
		# return SR(generate(result), media_type="text/event-stream")

		# $$$$$$$$$$$$$$$$$$Use below one line code as return will cause the error$$$$$$$$$$$$$$$
		return StreamingResponse.from_chain(chain, request.messages[-1].content)
	except ValueError as ve:
		return SR(generate(str(ve)), media_type="text/event-stream")
	except Exception as e:
		return SR(generate(str(e)), media_type="text/event-stream")

@KevinZhang19870314 the error is coming from openai. are you a free user? openai has different rate limits for free and paid users.

Yes, I am a free use, but you notice that in my code, I use another option will not cause this error.

Finally, I found that this is caused by chain.arun, chain.acall, chain.run method will not raise this error, why this happened?

I see. I'm not sure. Maybe a bug was introduced recently.
But if there are issues open, I'm sure the issue will be resolved soon. I'm afraid nothing can be done on this repo.

I am closing the issue for now.

Yes, I am a free user, but you notice that in my code, I use another option will not cause this error. If I not use StreamingResponse.from_chain, the error disappear.

I have the same problem, may I ask how did you solve it?

This happens mostly on apple silicon machines. Fixed it by,
bash /Applications/Python*/Install\ Certificates.command

https://stackoverflow.com/a/76519628/3297613


Update resource.h

Add an option to get the last error kept in object. (In order to be displayed by log messages.)

I feel like get_last_error() is a misnomer because the actual last error might have changed since the object was created.

Hi Charles,
Function get_last_error() is required and is correct.
Sometimes user wants to get last error that was saved by object and was taken following specific action, even though the current last error is different.
The aim of this object is to save last error at specific time, while probably the real last error is different at time of destruction of this object.

I'll agree with @sylveon here. Something like value() seems like a better name for the function.


ability to add more CURLOPT

Hi,
I need to add CURLOPT_SSL_VERIFYHOST to 0 to le curl client options, and maybe more.
Is it possible to add it or to allow Handler::init() to accept curl options ?
class Handler:
public function __construct(array $options = array(), array $curl_options= array()) {
...
$this->init($curl_options);
...
  protected function init($curl_options)
...
foreach($curl_option as $option=>$value) {
  $client->setOption( $option, $value);
}
$this->client = $client;
        $this->browser = new Browser($client);

regards

Hi,
I added this feature. Thank you.
Alexandre


Adding fixusings support

Add missing usings
Send a dictionary of key-list for ambiguous usings
Remove redundant usings

TODO:

Remove unused using statements
Sort using statements
If possible, replace RemoveRedundantUsings by something better from the framework

can we switch the bracing from k+r? apart from that, it looks good in theory to me.

Is it possible to make the response to match the old server so that we don't break backwards compatibility?
https://github.com/OmniSharp/omnisharp-server/blob/master/OmniSharp/CodeActions/FixUsingsResponse.cs

Very happy to see this PR.
It absolutely must use the same response as omnisharp v1.
Also... not sure there's much benefit from having remove usings if it's only removing dupes. We should probably delete that code and address that in a separate PR.

This code needs to be formatted using the format function in omnisharp :smile:

@sriramgd Are you planning on working on this some more? If not, I can pick it up where you left off.

@nosami I am. Will get in the updated PR by tomorrow evening. (Sorry about the break, was on vacation).

Updated, ready for second review. Seems the travis build fails due to a previous merge.

Gonna fix the build in a sec

I just tried this out. It seems to almost be working, but there are quite a few issues with it I think.
Removing using OmniSharp.Models; from a controller and then running fix usings results in 2 usings being added back
using OmniSharp.Models;
using OmniSharp;

There are also lots of ambiguous messages when there should be none (see screenshot)

Lots of namespaces in the OmniSharp project don't get imported, e.g. Xunit. I tried this on lots of different classes. I've also seen it import namespaces that are already included.

Yes, this implementation is broken. Too many false positives for the symbols. I had only tested against the unit tests which were quite trivial. Will make better tests and look at fixing this.

I am blocked on this for now. The solution I have requires being able to update the Document with a CompilationUnitSyntax before doing the ambiguity check. However it seems like document.UpdateSyntaxRoot is not available in Roslyn now (looks like it used to be a while back). I have asked a question about this on SO. In the meantime, anyone else who knows better or has other ideas could take this forward.

Actually I did get a response on SO. Was missing something obvious. Will update PR in sometime.

I have updated the implementation. This should do a better job at

Not including existing usings
Not flagging non-ambiguous symbols as ambiguous.

If this is accepted, we should replace any code which can be replaced by relevant APIs which Roslyn exposes in the future.

Just had a chance to try this again. It's definitely better, but there are still some bugs.
public class MyClass
{
    public MyClass()
    {
        var l = new List<string>();
    }
}

This code errored when trying to import usings.
Also..  does this code work with linq syntax? i.e. from s in list where...... I couldn't test because of the previous error.

Sorry, had been busy for a while.
Seems like my implementation does not work well. Please close my PR in case anyone else wants to take this on.

Closing in favour of #291


About the filter

Thanks for the reimplement of the paper!
In preprocessing/filter.py 307:
``    img = np.array(cv2.cvtColor((apply_filter_a(src_copy) + apply_filter_b(src_copy) + apply_filter_c(src_copy) +
apply_filter_d(src_copy) + apply_filter_e(src_copy) + apply_filter_f(src_copy)), cv2.COLOR_RGB2GRAY)//8)

It seems that 'apply_filter_g' is missing.

The sum of all filters is divided by 8, but there are 6 sets of filters from 'a' to 'f' ( or 7 sets if 'a' to 'g'). Maybe it should be divided by 7 ?

thanks for pointing out, I was getting errors while trying to train it. Will try to fix this as well #2 by next weekend 🤞

so may the
"img = np.array（cv2.cvtColor（（apply_filter_a（src_copy） + apply_filter_b（src_copy） + apply_filter_c（src_copy） +
apply_filter_d（src_copy） + apply_filter_e（src_copy） + apply_filter_f（src_copy）+ apply_filter_g（src_copy））， cv2.COLOR_RGB2GRAY）//7)
be right?

yes, pushing the changes... thanks for pointing this out

I have pushed the changes @Diverance @breadxiexie , you can refer to the commit 74626013


Browser/OS support

Currently the article reads like the listed browsers are supported across operation systems. My understanding is that certain browsers are only supported on certain operation systems -- otherwise people will assume that the web client would be supported on mobile browsers or Linux.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6457d204-d229-9154-28fd-ca9bbb4bddcd
Version Independent ID: f5d82846-41cb-3548-6997-ce23bd8f89c6
Content: Get clients for Microsoft Teams
Content Source: Teams/get-clients.md
Service: msteams
GitHub Login: @LolaJacobsen
Microsoft Alias: lolaj

@thomasbinder -- Thank you for your feedback. Can you provide specific wording for how you think the article should be updated? Thank you

@kenwith unfortunately I don't have the correct information. Can you find the PM?

I added this to the backlog so we can prioritize and get a PM for this. Will report back here. Closing for now since it is in the backlog system. Commenting here will re-open though.


ERROR event.go:107: etcd watcher {Watch /_coreos.com/fleet/job} returned error: unexpected end of JSON input

Hi ,
I have been able to start the etcd and fleetd_0.9.2 on the non vagrant virtual box set up on my development mac. I have not been able to execute deisctl install platform and I noticed the following in the journalctl logs. Any clues ?
Apr 22 23:17:56 localhost fleetd-0.9.2[639]: ERROR event.go:107: etcd watcher {Watch /_coreos.com/fleet/job} returned error: unexpected end of JSON input
Apr 22 23:17:57 localhost fleetd-0.9.2[639]: INFO client.go:291: Failed getting response from http://localhost:4001/: unexpected EOF
Apr 22 23:17:57 localhost fleetd-0.9.2[639]: ERROR client.go:213: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 100ms
Apr 22 23:18:41 localhost systemd[1]: Starting Generate /run/coreos/motd...
Apr 22 23:18:41 localhost systemd[1]: Started Generate /run/coreos/motd.
Apr 22 23:19:41 localhost systemd[1]: Starting Generate /run/coreos/motd...
Apr 22 23:19:41 localhost systemd[1]: Started Generate /run/coreos/motd.
Apr 22 23:20:41 localhost systemd[1]: Starting Generate /run/coreos/motd...
Apr 22 23:20:41 localhost systemd[1]: Started Generate /run/coreos/motd.
Apr 22 23:21:41 localhost systemd[1]: Starting Generate /run/coreos/motd...
Apr 22 23:21:41 localhost systemd[1]: Started Generate /run/coreos/motd.
Apr 22 23:22:41 localhost systemd[1]: Starting Generate /run/coreos/motd...
Apr 22 23:22:41 localhost systemd[1]: Started Generate /run/coreos/motd.
Apr 22 23:22:57 localhost fleetd-0.9.2[639]: ERROR event.go:107: etcd watcher {Watch /_coreos.com/fleet/job} returned error: unexpected end of JSON input
Apr 22 23:22:57 localhost fleetd-0.9.2[639]: INFO client.go:291: Failed getting response from http://localhost:4001/: unexpected EOF
Apr 22 23:22:57 localhost fleetd-0.9.2[639]: ERROR client.go:213: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 100ms
Apr 22 23:23:13 localhost systemd[1]: Starting OpenSSH per-connection server daemon (<IP_ADDRESS>:61196)...
Apr 22 23

wrong forum


Remove quay.io registry when building the image

Signed-off-by: YuChen<EMAIL_ADDRESS>

Please do the same in the release-future branch - thanks!


Performance impact of using Always Encrypted

Hi Team,
If possible, It would be really helpful to mention below points in this doc because those are frequently asked by our customer.
・Is there any risk on performance by using this feature?
・If any, is there any practice or action customer can take to mitigate it?
Thank you very much!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 21605a56-8aa8-9dbf-6743-999f204e6775
Version Independent ID: 35bbb590-6aec-4c63-9dac-cff287745d06
Content: Always Encrypted - SQL Server
Content Source: docs/relational-databases/security/encryption/always-encrypted-database-engine.md
Product: sql
Technology: security
GitHub Login: @jaszymas
Microsoft Alias: jaszymas

@yutaro113 -- Yutaro, thank you for your questions.  Please consider these resources:

SQL Server on Microsoft Q&A
DBA Stack Exchange
Stack Overflow

@jaszymas -- Jakub, please look into this potential documentation issue.

@yutaro113 Many factors can affect performance, including CPU type and load, hardware configuration (including physical vs. virtual, Hyper-Threading), the size of the queries, whether you use secure enclaves (which do improve performance, but have a limited amount of memory available), and so on.
The best way to evaluate performance impact of Always Encrypted, is to test it in your environment.