Remove Binder tooling

Fixes #98
Changes made in this Pull Request:

Remove Binder-related files

PR Checklist

[ ] Tests?
[ ] Docs?
[ ] CHANGELOG updated?
[ ] Issue raised/referenced?

Codecov Report

Merging #99 (ea20cdd) into main (dab0350) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files


[BEAM-12170] Handle duplicate metrics due to flatten unzipping.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Java Flink PortableValidatesRunner Batch

Run Java Examples on Dataflow Runner V2

Run Java PreCommit

R: @y1chi


fix: Pod install issue & listener conflict with document picker library

The pull request contains below fix -

Pod demo swift added - helps users to run 'pod install' inside the library directly
https://github.com/prscX/react-native-file-selector/issues/23

Conflict with listener
File-selector plugin is listening to document-picker file selection and throwing error causing app crash in some mobiles.

Thanks @Rinkuk1993 for the Effort and PR.

w.r.t first point: the fix is for pod installation? Could you please share the exception which you were getting.

second point: fix w.r.t conflict with listener looks good.

Thanks
</ Pranav >

For the first point -
You can check this thread https://github.com/prscX/react-native-file-selector/issues/23
https://github.com/prscX/react-native-file-selector/issues/28
Normally people are not able to do pod install inside the library due to unavailability of swift file.
They have to manually create one from Xcode(normally people don't know how to do it).
For reference - check my comment on the issue thread.
This fix will simply remove unwanted confusion.
Thanks

Is there an issue merging the changes? I am not sure about the status of this PR since there is no response

I get the following error for this PR for iOS usage:
[!] The `RNFileSelector` pod failed to validate due to 1 error:
    - ERROR | attributes: Missing required attribute `homepage`.
    - WARN  | source: The version should be included in the Git tag.
    - WARN  | description: The description is equal to the summary.

All it took to solve this issue was to change the podspec file:
  s.homepage     = "https://github.com/prscX/react-native-file-selector"

Conflict with listener
File-selector plugin is listening to document-picker file selection and throwing error causing app crash in some mobiles.

I see that someone has already added PR for the above fix due to my inactivity. Since the code is same, I have removed my code.

Pod demo swift added - helps users to run 'pod install' inside the library directly
#23

If the issue "Pod demo swift added" is also fixed, let's close this PR.


Feedback for: engine/installation/linux/centos.md

File: engine/installation/linux/centos.md
The offline package downloading address is not available right now (Access denied)
->  https://download.docker.com/linux/centos/7/x86_64/stable/Packages/

The index pages are currently missing for download.docker.com (see https://github.com/docker/docker/issues/31987)
You can download the 17.03.0 CE packages using these URLs;
https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-17.03.0.ce-1.el7.centos.x86_64.rpm
https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.0.ce-1.el7.centos.noarch.rpm
/cc @mstanleyjones @andrewhsu

Thanks @thaJeztah. Those links work for me!

A browsable index should also be available now on download.docker.com (https://github.com/docker/docker/issues/31987)

Fixed now!


Standardize strings

:koala:
Seems like we have been editing en.yml directly.
/cc @zendesk/quokka @ydrain @pdeuter
Risks

None

:+1:

maybe change the comment to # This file is generated by rake i18n:standardize. Do not edit it directly

:+1:

Why not make the comment IN CAPITAL LETTERS SO THAT IT LOOKS SCARY? :imp:

I think a test would be better, but not sure it's worth it right now.

you need to re-run it with the new header :-)

@sandlerr :haircut:

LGTM!


tl.device_assert can observe incorrect results from tl.associative_scan

In the example below we have 4 warps/128 threads performing a scan over scanning 32 elements so some values are duplicated between multiple threads. I would expect the tl.device_print to print a scan value of 1 through 32, repeated 4 times. Instead I see 1 through 128 printed once each and if I run with TRITON_DEBUG=1 the assert will fail in thread ids 32 through 127.
Note that the output is actually correct, so I guess these incorrect values are thrown away when broadcasting to the output shape. However, this is still problematic because cumsum is often used for indexing calculations and in inductor we assert that indices are within bounds.
import triton
import triton.language as tl
import torch

@triton.jit
def fn(out_ptr, in_ptr, XBLOCK: tl.constexpr, RBLOCK: tl.constexpr):
    xindex = tl.arange(0, XBLOCK)[:, None]
    rindex = tl.arange(0, RBLOCK)[None, :]
    data = tl.load(in_ptr + rindex)
    scan = tl.cumsum(data, 1)
    expected_max = tl.sum(data, 1)
    tl.device_print("scan, sum = ", scan, expected_max)
    tl.device_assert(scan <= expected_max)
    tl.store(out_ptr + xindex * RBLOCK + rindex, scan)

XBLOCK, RBLOCK = 4, 32
in0 = torch.ones(1, RBLOCK, device="cuda")
out0 = torch.empty((XBLOCK, RBLOCK), device="cuda")
fn[(1,)](out0, in0, XBLOCK, RBLOCK, num_warps=XBLOCK)

I have confirmed this fails on the latest master (08c16589573621fcb8cd5a9c3b8a0537077f876d).

It's urgent? If not, I'll take a look over the weekend

Next week would be great!

@Jokeren I can confirm the reproducer above is fixed but I'm still seeing failures when I change RBLOCK to 16. This time I see 1 to 32 within each warp.

@Jokeren I can confirm the reproducer above is fixed but I'm still seeing failures when I change RBLOCK to 16. This time I see 1 to 32 within each warp.

Sure, I'll fix it either today or tomorrow


testcases/InstallUpstreamKernel: Use grub2-set-default inplace of grubby

InstallUpstreamKernel testcase relies on grubby command to update boot
loader configuration. On hosts running certain Linux flavours which does
not support this command the test fails with command not found error:
[console-expect]#grubby --set-default /boot/vmlinuz-cat include/config/kernel.release 2> /dev/null
grubby --set-default /boot/vmlinuz-cat include/config/kernel.release 2> /dev/null
bash: grubby: command not found
Use grub2-set-default command to specify the boot order. Also switch the
order of grub2-mkconfig and grub2-set-default commands to ensure that all
installed kernels are accounted for before choosing the default one.
Signed-off-by: Sachin Sant<EMAIL_ADDRESS>

@sacsant patch looks good, But DCO complains about signature can you please have look

yeah, not sure how to fix this. Looking at the error:
Commit sha: 0f5974d, Author: Sachin Sant, Committer: Sachin Sant; Expected "Sachin Sant<EMAIL_ADDRESS>but got "Sachin Sant<EMAIL_ADDRESS>Primary email on my github account is set to<EMAIL_ADDRESS>

Okay, I have changed the email to<EMAIL_ADDRESS>in signed-off-by tag.

Okay, I have changed the email to<EMAIL_ADDRESS>in signed-off-by tag.

sure, are you looking this patch should be in now? or we  can check and fix the DCO issue having<EMAIL_ADDRESS>as user signature

Okay, I have changed the email to<EMAIL_ADDRESS>in signed-off-by tag.

sure, are you looking this patch should be in now? or we can check and fix the DCO issue having<EMAIL_ADDRESS>as user signature

Not super important for this to get in immediately. Would be good to resolve the DCO issue, prefer to use<EMAIL_ADDRESS>email.

@sacsant  not getting any clue right now as DCO recent update also limit to check only recent commit (as earlier it was checking older commit and compare user signature) and even your GitHub Profile set with<EMAIL_ADDRESS>planning to raise an issue against DCO service , looking one help can you please send a dummy PR (commit) so we can use that as reference.
May this process will take some time can we merge this request

May this process will take some time can we merge this request

Sure, let's merge this fix.

@sacsant not getting any clue right now as DCO recent update also limit to check only recent commit (as earlier it was checking older commit and compare user signature) and even your GitHub Profile set with<EMAIL_ADDRESS>planning to raise an issue against DCO service , looking one help can you please send a dummy PR (commit) so we can use that as reference.

I have created following issue
https://github.com/open-power/op-test/pull/678
It shows the DCO failure.


fix(mangademon): rewrite most of the source to account for a site remake

Checklist:

[x] Updated source's version for individual source changes
[x] Updated all sources' versions for template changes
[x] Set appropriate nsfw value
[x] Did not change id even if a source's name or language were changed
[x] Tested the modifications by running it on the simulator or a test device

Closes #753

seeing a crash when opening martial peak due to duplicate chapter identifiers. double check how you're assigning chapter ids and make sure they're unique, and make sure to test on device

I've been reading from mangademon with this version of the source for a few weeks, I have 60 from it and I haven't noticed any issues. What chapter on martial peak are you having problems with, I tested a few chapters on there and they seem to work for me.

interestingly enough, it's only happening when run on the simulator. none of the images load for me on device though. the culprit for martial peak is /chaptered.php?manga=1&chapter=3705, which I can see on the site is listed twice. this is their mistake, but you should probably filter out duplicates on our end within the source

I took a look at 3705 and it seems to work on my phone even though there were chapters with that number. I just went ahead and removed duplicates anyway though


Publish new release, that includes the SetOpaque fix

It is not possible to compile the latest release of rclone, because of the api changes.
The master branch is possible to compile because it already includes the SetOpaque fix.
For standard users this is no problem, but it makes it hard to package for distros.

I see what you mean...
I have considered making a release branch with fixes like that on to get a 1.33.1, 1.33.2, 1.33.3 which would have just the essential bug fixes in from the 1.33 release.
Is that the kind of thing you mean?
Which distro are you packaging for?

Yeah that is exactly what i mean.
I wanted to package for Solus, but i need to use a release as source, i could also use a commit as source but this is not common.

Do you have any idea when you will release a new version?

Very soon!
For your purposes I need to use one of the vendoring solutions for go otherwise the build won't be deterministic.  I've made a new issue about this #816 and will try to do this for the next release.  This will fix the issue permanently!

Looks like this bug is fixed now

@krieghof it is indeed!  Will close now


Pattern9.java

hello i created java pattern

not in java try in python

can you open i will update these


Dry Mode instead of Heat

Wondering if switching out Heat from the config with Dry will enable homekit to set it to dry. I can use Homekit to set the AC to auto, but i have to go back to the panasonic app to set it to dry.
Might also be related to the eco navi option i saw with another related inquiry here.
from the left, cool, dry, nanoex, auto
@

If I am to be honest, I completely did not understand what you would you like to achieve? If you want additional control such as nanoe, ecoNavi, etc., I add a pull request in this matter. #89

As no further comments have been received on this request, please re-open if still relevant. In the meantime several enhancements have been made to control the AC modes from v1.8.0 onwards:
New functions:

Choice of what to control Oscilate Switch: Swing, Nanoe, Eco Navi, Inside Cleaning.
Choice what value (for Swing, Nanoe, Eco Navi, Inside Cleaning) should be set with each state change made with Homekit (e.g. activation).
When the internal unit does not have a temperature sensor, the temperature of the external unit is display, and when there are no both, it is automatically depending on the mode (heating / cooling).


Add Keyce academy school

@francoisvid Please provide us with:

the school official website URL
the school street address, including city and country
an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course
a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students.

Please provide us with a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students.
You can provide us with any of the following:

A copy of instruction (a hard-copy or electronic one) given to you by the school that tells you to use an email address in this email domain
An URL of a page at the official school website that says that students use this email domain
A reference to a recognized journal where an article was published by someone from the school, with explicit affiliation to the school, and e-mail address from the email domain
A WHOIS data that link the email domain to the school somehow
A copy of official letter from the school representative saying that the school provides e-mail addresses in this domain to the students
Anything else that can prove this fact in similar way, i.e. can be verified independently.

@francoisvid You might miss my previous comment. Please react.

@francoisvid I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for your understanding!


RMSProp errors with embedding_lookup

I've been trying to train an LSTM with a word embedding and got some errors that seem related to https://github.com/tensorflow/tensorflow/issues/1117
Environment info
Operating System:
If installed from binary pip package, provide:
Linux CPU Only
0.10.0rc0
Steps to reproduce
This is the code I've been running, which should repro the issue:
import tensorflow as tf
import numpy as np
import pandas
import tensorflow as tf
from tensorflow.contrib import learn
#from preprocessing import VocabularyProcessor
VocabularyProcessor = learn.preprocessing.VocabularyProcessor

def partition_length(x_train, y_train):
  #Partition the training data by length of sequences
  x_train_dict = {}
  y_train_dict = {}
  for i in range(x_train.shape[0]):
    x = x_train[i]
    y = y_train[i]

    l = len(x)
    if l not in x_train_dict:
      x_train_dict[l] = []
      y_train_dict[l] = []
    x_train_dict[l].append(x)
    y_train_dict[l].append(y)

  x_train_np = {}
  y_train_np = {}
  for l in x_train_dict:
    x_train_np[l] = np.asarray(x_train_dict[l])
    y_train_np[l] = np.asarray(y_train_dict[l])

  return (x_train_np, y_train_np)

n_words = 0
MAX_DOCUMENT_LENGTH = 10
EMBEDDING_SIZE = 50

# Prepare training and testing data
dbpedia = learn.datasets.load_dataset('dbpedia')

x_train = pandas.DataFrame(dbpedia.train.data)[1]
y_train = dbpedia.train.target
x_test = pandas.DataFrame(dbpedia.test.data)[1]
y_test = dbpedia.test.target

print x_train.shape, y_train.shape
print x_test.shape, y_test.shape

# Process vocabulary
vocab_processor = VocabularyProcessor(10)
x_train = np.array(list(vocab_processor.fit_transform(x_train)))
x_test = np.array(list(vocab_processor.transform(x_test)))
n_words = len(vocab_processor.vocabulary_)
print('Total words: %d' % n_words)

print x_train.shape, y_train.shape

(x_train_part, y_train_part) = partition_length(x_train, y_train)

data = tf.placeholder(tf.int32, [None, None], name='data')
word_vectors = learn.ops.categorical_variable(data, n_classes=n_words,
  embedding_size=EMBEDDING_SIZE, name='words')

print('data: ', data.get_shape())
print('word_vectors: ', word_vectors.get_shape())
print('word_vectors: ', tf.shape(word_vectors))

target = tf.placeholder(tf.int32, [None], name='target')
one_hot = tf.one_hot(target, 15, 1.0, 0.0, dtype=tf.float32)

print('target: ', target.get_shape())
print('one_hot: ', one_hot.get_shape())

num_hidden = 250

_, state = tf.nn.dynamic_rnn(
    tf.nn.rnn_cell.GRUCell(num_hidden),
    word_vectors,
    dtype=tf.float32,
)

print('state: ', state.get_shape())

in_size, out_size = (num_hidden, 15)

weight = tf.Variable(tf.truncated_normal([in_size, out_size], stddev=0.01))
bias = tf.Variable(tf.constant(0.1, shape=[out_size]))

prediction = tf.nn.softmax(tf.matmul(state, weight) + bias)
#prediction = tf.contrib.losses.softmax_cross_entropy(logits, onehot_labels, weight=1.0, label_smoothing=0, scope=None)
prediction_idx = tf.argmax(prediction, 1)
one_hot_idx = tf.argmax(one_hot, 1)

print('prediction: ', prediction.get_shape())
print('prediction_idx: ', prediction_idx.get_shape())
print('one_hot_idx: ', one_hot_idx.get_shape())

mistakes = tf.not_equal(one_hot_idx, prediction_idx)
error =  tf.reduce_mean(tf.cast(mistakes, tf.float32))

cross_entropy = -tf.reduce_sum(one_hot * tf.log(prediction))

learning_rate = 0.003
optimizer = tf.train.RMSPropOptimizer(learning_rate)
optimize = optimizer.minimize(cross_entropy)

sess = tf.Session()
sess.run(tf.initialize_all_variables())

batch_size = 5
for length in x_train_part:
  x_train = x_train_part[length]
  y_train = y_train_part[length]

  print "Training on length ", length, " # elements: ", x_train.shape[0]

  error_pct = sess.run(error, {data: x_test, target: y_test})
  print('Epoch {:2d} error {:3.1f}%'.format(0, 100 * error_pct))

  for epoch in range(10):
    print('Epoch {:2d}'.format(epoch+1))
    for i in range(x_train.shape[0]/batch_size):
      sess.run(optimize, {data: x_train[i:i+batch_size], target: y_train[i:i+batch_size]})

    error_pct = sess.run(error, {data: x_test, target: y_test})
    print('Epoch {:2d} error {:3.1f}%'.format(epoch + 1, 100 * error_pct))

What have you tried?
Using any non-RMSProp Optimizer works
Logs or other output that would be helpful
This is the exception I get:
Traceback (most recent call last):
  File "lstm.py", line 131, in <module>
    sess.run(optimize, {data: x_train[i:i+batch_size], target: y_train[i:i+batch_size]})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 710, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 908, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 958, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 978, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.InvalidArgumentError: var and grad do not have the same shape[7664,50] [50,50]
         [[Node: RMSProp/update_words/words_embeddings/SparseApplyRMSProp = SparseApplyRMSProp[T=DT_FLOAT, Tindices=DT_INT32, _class=["loc:@words/words_embeddings"], use_locking=false, _device="/job:localhost/replica:0/task:0/cpu:0"](words/words_embeddings, words/words_embeddings/RMSProp, words/words_embeddings/RMSProp_1, RMSProp/learning_rate, RMSProp/decay, RMSProp/momentum, RMSProp/epsilon, gradients/words/embedding_lookup/embedding_lookup_grad/Reshape, gradients/words/embedding_lookup/embedding_lookup_grad/Reshape_1)]]
Caused by op u'RMSProp/update_words/words_embeddings/SparseApplyRMSProp', defined at:
  File "lstm.py", line 111, in <module>
    optimize = optimizer.minimize(cross_entropy)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 198, in minimize
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 313, in apply_gradients
    update_ops.append(self._apply_sparse(grad, var))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/rmsprop.py", line 122, in _apply_sparse
    use_locking=self._use_locking)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/gen_training_ops.py", line 664, in sparse_apply_rms_prop
    use_locking=use_locking, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 703, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2317, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1239, in __init__
    self._traceback = _extract_stack()

As you are asking for help using TensorFlow, please ask this question on StackOver Flow. GitHub issues are for bugs and feature enhancements only. Thank you.

@aselle this could entirely be user error, but it smells like a bug due to the fact that it is an optimizer-specific issue

I understand, but StackOverflow is probably the easiest way to get advice on finding out if you are doing something wrong or it is a bug.  If you are convinced it is a bug, please try to reduce it to a simpler test case.

@vrv this looks like a bug. It seems you wrote SparseApplyRMSProp. It seems to work with many other optimizer. I tried AtomOptimizer, MomentumOptimizer, GradientDescentOptimizer and they all worked.

I did not write SparseApplyRMSProp -- I think it was contributed by someone externally.  Let me find the commit...

From https://github.com/tensorflow/tensorflow/pull/2564
@suiyuan2009 do you think you can take a look?

let me take a look.

@kuza55 I'm sorry, check process in SparseApplyRMSProp  is wrong, I'll make a quick fix, and add additional test case.

Thanks for the quick response @suiyuan2009.!

Thanks @suiyuan2009 :)

Hi @vrv, @suiyuan2009 - a quick hello! I am still facing this problem implementing a GRU with word embedding and RMSProp. As originally noted by @kuza55, other optimizers - Adam, Adagrad, FTRL in my case - work. With RMSProp, I get the same issue. I am running TF on AWS g2.2xlarge with Python 3.4.3.
After seeing this thread today, I upgraded TensorFlow but that didn't fix the problem. Has the fix been uploaded?
Kindly advise!

what version did you upgrade to? a nighlty build? or did you build from source?

I am guessing its the nightly build as I didn't build from source. I did a sudo pip3 install -- upgrade

try running python -c "import tensorflow; print(tensorflow.__version__)".

Its 0.10.0rc0

install --upgrade doesn't fetch the nightlies, you should get the nightly
link off the main github page.
On Wed, Aug 24, 2016 at 3:20 PM, Jagannath Rajagopal <
<EMAIL_ADDRESS>wrote:

Its 0.10.0rc0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/3797#issuecomment-242226590,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAcTeZvz2K55P5PJx5dIEWXpDh6RDAyZks5qjMOtgaJpZM4JjvIk
.

Ok thanks - I'll give it a try

Right - that worked - thanks a lot :-)!

Any reliable fix on this issue yet - made part of the release? I'm also getting the "embedding_lookup" exception when using RMSprop in a LSTM network. It works fine with the Adam optimizer.


fixes #3252 support tablets w/ and w/o location in tablet location cache

Modifies the TabletLocatorCache to support tablets with and without a location.  Also modified the ScanServerSelector to support optionally waiting for scan servers when there are none instead of always falling back to tservers.  The scan severs selector falling back to the tserver was written with the assumption that tablets are always hosted.  Now that tablets are loaded onto tserers on demand it may not be desirable to load the tablets when there are no scan servers.  That is why this changes also included changes to optionally wait for scan servers when there are none.  Restarting all scan servers will be a normal administrative operations, may not want a lot of tablets to all of a sudden load on tablet servers when the scan servers are restarted.

I kicked off a fuil IT build. Some of the things in the PR may not be necessary given the changes that I am working on to make all user tables on-demand by default. I'm close to being ready to create a PR for that, I have some things I need to finish first (looking at tests, testing locally, etc.). If you want an early look at my changes, see https://github.com/apache/accumulo/compare/elasticity...dlmarion:accumulo:elasticity-ondemand-feature-changes2?expand=1.

Some of the things in the PR may not be necessary given the changes that I am working on to make all user tables on-demand by default. I'm close to being ready to create a PR for that, I have some things I need to finish first (looking at tests, testing locally, etc.). If you want an early look at my changes, see https://github.com/apache/accumulo/compare/elasticity...dlmarion:accumulo:elasticity-ondemand-feature-changes2?expand=1.

There will be conflicts.   I took a look and had some thoughts, it you post a draft PR I could make a few comments.

Full IT build completed successfully


TEST llvm-cm X86/multi_func.s FAILED

Hello, after building tflite, there is an error with llvm-cm:
(env) $                         ninja check-llvm-tools-llvm-cm
[0/1] Running llvm-cm tests
llvm-lit: /home/hrong1/llvm-src/llvm-project/llvm/utils/lit/lit/llvm/config.py:502: note: using yaml2obj: /home/hrong1/llvm-src/cmake-build/bin/yaml2obj
llvm-lit: /home/hrong1/llvm-src/llvm-project/llvm/utils/lit/lit/llvm/config.py:502: note: using llvm-cm: /home/hrong1/llvm-src/cmake-build/bin/llvm-cm
llvm-lit: /home/hrong1/llvm-src/llvm-project/llvm/utils/lit/lit/llvm/config.py:502: note: using split-file: /home/hrong1/llvm-src/cmake-build/bin/split-file
llvm-lit: /home/hrong1/llvm-src/llvm-project/llvm/utils/lit/lit/llvm/config.py:502: note: using llvm-mc: /home/hrong1/llvm-src/cmake-build/bin/llvm-mc
FAIL: llvm-cm :: X86/multi_func.s (11 of 11)
******************** TEST 'llvm-cm :: X86/multi_func.s' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 2
/home/hrong1/llvm-src/cmake-build/bin/llvm-mc -o /home/hrong1/llvm-src/cmake-build/X86/Output/multi_func.s.tmp.o --filetype=obj -triple=x86_64-unknown-linux-gnu /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s
# executed command: /home/hrong1/llvm-src/cmake-build/bin/llvm-mc -o /home/hrong1/llvm-src/cmake-build/X86/Output/multi_func.s.tmp.o --filetype=obj -triple=x86_64-unknown-linux-gnu /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s
# RUN: at line 3
/home/hrong1/llvm-src/cmake-build/bin/llvm-cm /home/hrong1/llvm-src/cmake-build/X86/Output/multi_func.s.tmp.o -csv=/home/hrong1/gematria/llvm_cm/test/X86/Inputs/multi-func.csv -granite_model=/home/hrong1/gematria/llvm_cm/test/X86/Inputs/gb-token-mit-2022_12_02.tflite -evaluator=granite | /home/hrong1/llvm-src/cmake-build/bin/FileCheck /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s
# executed command: /home/hrong1/llvm-src/cmake-build/bin/llvm-cm /home/hrong1/llvm-src/cmake-build/X86/Output/multi_func.s.tmp.o -csv=/home/hrong1/gematria/llvm_cm/test/X86/Inputs/multi-func.csv -granite_model=/home/hrong1/gematria/llvm_cm/test/X86/Inputs/gb-token-mit-2022_12_02.tflite -evaluator=granite
# .---command stderr------------
# | Unexpected node token: 'RIP'
# `-----------------------------
# executed command: /home/hrong1/llvm-src/cmake-build/bin/FileCheck /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s
# .---command stderr------------
# | /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s:8:15: error: CHECK-NEXT: expected string not found in input
# | # CHECK-NEXT: Calculated Frequency: 8.342712e+03
# |               ^
# | <stdin>:1:11: note: scanning from here
# | <reverse>:
# |           ^
# | <stdin>:2:1: note: possible intended match here
# | Calculated Frequency: 8.342695e+03
# | ^
# |
# | Input file: <stdin>
# | Check file: /home/hrong1/gematria/llvm_cm/test/X86/multi_func.s
# |
# | -dump-input=help explains the following input dump.
# |
# | Input was:
# | <<<<<<
# |           1: <reverse>:
# | next:8'0               X~ error: no match found
# |           2: Calculated Frequency: 8.342695e+03
# | next:8'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | next:8'1     ?                                   possible intended match
# |           3: <tallestBillboard>:
# | next:8'0     ~~~~~~~~~~~~~~~~~~~~~
# |           4: Calculated Frequency: 2.928508e+05
# | next:8'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |           5: <isMatch>:
# | next:8'0     ~~~~~~~~~~~~
# |           6: Calculated Frequency: 8.204262e+02
# | next:8'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |           7: <bubbleSort>:
# | next:8'0     ~~~~~~~~~~~~~~~
# |           .
# |           .
# |           .
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

********************
********************
Failed Tests (1):
  llvm-cm :: X86/multi_func.s

Testing Time: 0.59s

Total Discovered Tests: 11
  Passed: 10 (90.91%)
  Failed:  1 (9.09%)
FAILED: tools/gematria/llvm_cm/CMakeFiles/check-llvm-tools-llvm-cm /home/hrong1/llvm-src/cmake-build/tools/gematria/llvm_cm/CMakeFiles/check-llvm-tools-llvm-cm
cd /home/hrong1/llvm-src/cmake-build/tools/gematria/llvm_cm && /home/hrong1/gematria/env/bin/python3 /home/hrong1/llvm-src/cmake-build/./bin/llvm-lit -sv /home/hrong1/llvm-src/cmake-build/tools/gematria/llvm_cm

Do you have more information on the LLVM version that you built with/your environment? Everything works fine for me with tip of tree LLVM.

I pulled the latest LLVM project from https://github.com/llvm/llvm-project.git. Here is the head:
$ git log
commit 82f479ba315a417b6cd01a8c2efdc15c26689f2e (HEAD -> main, origin/main, origin/HEAD)
Author: Usama Hameed<EMAIL_ADDRESS>Date:   Mon Apr 15 19:42:45 2024 -0700

    Add asan tests for libsanitizers. (#88349)

    This patch tests LLDB integration with libsanitizers for ASan.

    rdar://111856681

Looking at the log more, it seems like this is a floating point precision issue. For example, the value given for the reverse function in your case is 8.342695e+03 instead of the expected 8.342712e+03. Given that result requires the evaluation of the GRANITE model, some FP differences depending on the platform might be expected, although I'm not too familiar with what guarantees Tensorflow/TFLite makes.
That means the functionality is working fine, there are just some minor output differences. The llvm-cm tests need to be cleaned up at some point (probably rewritten in IR rather than assembly to make them significantly more maintainable), so I'll make sure to figure out a better floating point matching strategy when doing that.

Good point! Thanks, Aiden.


Add ui button to download torrent from search result.

Currently the option to download a torrent from the search results is "buried" in the right click context menu. It would be more intuitive if a download button was shown on the ui.

Torrent search is removed.


feat: Add a fully working toolchain

Got a little bit more work to clean up the UI. It does currently require libstdc++-9-dev if your on ubuntu, as I've hard coded the
system include paths. I'll admit I'm a bit stuck on some of these compiler errors https://github.com/bazelembedded/modular_cc_toolchains/actions/runs/4591805987/jobs/8108291820#step:5:661. Although some basic compilation/runtime tests I've done locally sort of demonstrate that the toolchain works for more basic applications.

Sorry for being slow on this front :)

Regarding the compiler errors: could it be that some cxx_builtin_include_directories are still missing?
FWIW for Fuchsia clang (https://chrome-infra-packages.appspot.com/p/fuchsia/third_party/clang) I would use,
  "%package(@clang//)%/include/c++/v1/x86_64-unknown-linux-gnu/c++/v1",
  "%package(@clang//)%/include/c++/v1",
  "%package(@clang//)%/lib/clang/15.0.0/include",
  "%package(@clang//)%/lib/clang/15.0.0/share",

I wonder if Fuchsia clang + Fuchsia sysroot (https://chrome-infra-packages.appspot.com/p/fuchsia/third_party/sysroot) would work in CI.


Is there any way to show properties or special characters in the generated HTML?

Hi!
First off, incredible application, this is really, really nice!
I just had a couple quick questions:
(1) Can I show additional, ontology-defined annotation properties on the concept pages (like if my ontology has the annotation property "hasDefinition" or something)? If so, how?
(2) Is there any way to get Unicode symbols to appear? My ontology has records in German, Russian, Chinese, and Japanese, so would that be a possibility?
Thanks so much!

Can you share a sample model for testing these functionalities?


Update doobie-core, doobie-postgres to 1.0.0-RC5

About this PR
📦 Updates

org.tpolecat:doobie-core
org.tpolecat:doobie-postgres

from 1.0.0-RC4 to 1.0.0-RC5
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.tpolecat" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.tpolecat" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, version-scheme:early-semver, commit-count:1

Superseded by #325.


System.DrawingCommon is not supported on this platform

I tried to use ScottPlot on Ubuntu, but failed with the well known System.Drawing.Common is not supported for this platform error. I opened an issue at the dotnet repo and I was told that Microsoft does no support System.Drawing any longer on non-Windows.
Is there a work around this problem, so that I can run ScottPlot on Ubuntu?
Thanks, Joaquin

Hi @joacorapela, thanks for reaching out!
The core issue is that .NET 7 just dropped support for System.Drawing.Common which ScottPlot 4 depends on. ScottPlot 5 drops this dependency, but isn't ready for production use yet.
The best compromise is to modify your csproj file to target .NET6. You'll see a similar warning, but it won't be an error, and your app should still compile and function normally.
More information can be found here: https://scottplot.net/versions/

Hello @swharden. Thanks for your prompt reply. I was running .NET6 and getting an error, not a Warning:
Unhandled exception. System.TypeInitializationException: The type initializer for 'ScottPlot.Drawing.InstalledFont' threw an exception.
 ---> System.PlatformNotSupportedException: System.Drawing.Common is not supported on this platform.
   at System.Drawing.FontFamily.get_Families()
   at ScottPlot.Drawing.InstalledFont..cctor()
   --- End of inner exception stack trace ---
   at ScottPlot.Drawing.InstalledFont.Monospace()
   at ScottPlot.Renderable.Message..ctor()                                                                                at ScottPlot.Renderable.BenchmarkMessage..ctor()
   at ScottPlot.Settings..ctor()
   at ScottPlot.Plot..ctor(Int32 width, Int32 height)
   at Program.<Main>$(String[] args) in /home/rapela/tmp/testDotnet/learningCSharp/code/scripts/bayesianLinearRegression/generateRegressionDat
a/Program.cs:line 51

The problem was related to #1079 I was using
   > System.Drawing.Common      7.0.0       7.0.0

After downgrading to
   > System.Drawing.Common      5.0.2       5.0.2

My program works fine.
Thanks again for your help.

Thanks for sharing your solution!
I added this note to https://scottplot.net/faq/dependencies/?nocache so hopefully it will be easier to discover in the future.


Edge case when you update Xcode and haven't agreed to license requirements

I just updated my Xcode to 7.1 and tried to run snapshot init on my project.  The ruby code to detect the version of Xcode bailed because the xcodebuild command gives the following response:
$ xcodebuild -version -sdk
Agreeing to the Xcode/iOS license requires admin privileges, please re-run as root via sudo.

I'll submit a PR that catches the case where no version is returned so a more user friendly error can be displayed.

Thanks :+1:


RFD: Proposal to Add Keybindings for Additional JupyterCAD Commands

Currently, JupyterCAD supports keybindings for a few essential commands such as undo, redo, and removeObject. However, there are several other commands in JupyterCAD that would benefit from having dedicated keybindings to enhance UX.
Proposed Keybindings(Draft):

New Sketch: Ctrl + Shift + N
New Box: Ctrl + Shift + B
New Cylinder: Ctrl + Shift + C
New Sphere: Ctrl + Shift + S
New Cone: Ctrl + Shift + E
New Torus: Ctrl + Shift + T
Cut: Ctrl + X
Extrusion: Ctrl + Shift + E
Union: Ctrl + U
Intersection: Ctrl + I
Chamfer: Ctrl + Shift + F
Fillet: Ctrl + F
Update Axes: Ctrl + Shift + A
Update Exploded View: Ctrl + Shift + V
Update Camera Settings: Ctrl + Shift + K
Update Clip View: Ctrl + Shift + L
Export Jcad: Ctrl + Shift + X

New Box: Ctrl + Shift + B
New Cylinder: Ctrl + Shift + C
New Sphere: Ctrl + Shift + S
New Cone: Ctrl + Shift + E
New Torus: Ctrl + Shift + T

I think these may also be fine to skip

Thanks for the suggestion. I think we should keep it consistent by prefixing all shortcuts with Ctrl + Shift and should avoid collision with JupyterLab shortcuts

New Box: Ctrl + Shift + B
New Cylinder: Ctrl + Shift + C
New Sphere: Ctrl + Shift + S
New Cone: Ctrl + Shift + E
New Torus: Ctrl + Shift + T

I think these may also be fine to skip

I have no issues keeping these shortcuts


Undefined variable x

I'm hitting this line in pxla.py which doesn't seem to be tested right now:
https://github.com/google/jax/blob/505b47c1fcef45534aa0cc90509ce146f9939c15/jax/interpreters/pxla.py#L80
The variable x isn't defined anywhere.

Gooooood catch! I think I know how to write a repro, but if you've got one please share it.

My code is too complicated unfortunately :(
I'm not sure how to make a minimal repro so if you can think of a way I would say go ahead.

Here's a repro when 8 XLA devices are available (like in our favorite colab notebooks):
from jax import pmap
import numpy as onp
a = pmap(lambda x: x)(onp.ones((2, 2)))
b = pmap(pmap(lambda x: x))(a)

By the way, as a workaround you can always bounce to the host (calling onp.asarray) or to a single device (calling device_put). I'm working on a more efficient solution though.
Let me know if you need better workarounds.

Meetings!
Okay, here's something you can paste into a colab cell to fix things:
from jax.interpreters import pxla, xla
from jax.interpreters.pxla import ShardedDeviceArray, shard_arg_handlers, assign_shards_to_replicas

def shard_args(backend, devices, assignments, axis_size, tuple_args, args):
  nargs, nrep = len(args), len(devices)
  buffers = [[None] * nargs for _ in range(nrep)]
  for a, arg in enumerate(args):
    # The shard_arg_handlers allow an extensible set of types to be sharded, but
    # inline handling for ShardedDeviceArray as a special case for performance
    if type(arg) is ShardedDeviceArray:
      if nrep == len(arg.device_buffers):
        # The argument is already prepared for the right number of replicas, so
        # we just ensure that buf[r] is on devices[r] for each replica index r
        for r, buf in enumerate(arg.device_buffers):
          buffers[r][a] = buf if buf.device() == devices[r] else buf.copy_to_device(devices[r])
      else:
        # The argument is prepared for a different number of replicas, so for
        # each of our replica indices we check if there's already a buffer with
        # the correct logical assignment on the correct device, and if not just
        # copy one of them
        prev_assignments = assign_shards_to_replicas(len(arg.device_buffers), axis_size)
        for r in range(nrep):
          candidates = (buf for i, buf in enumerate(arg.device_buffers)
                        if prev_assignments[i] == assignments[r])
          for buf in candidates:
            if buf.device() == devices[r]:
              buffers[r][a] = buf
              break
          else:
            buffers[r][a] = buf.copy_to_device(devices[r])
    else:
      bufs = shard_arg_handlers[type(arg)](arg, devices, assignments, backend=backend)
      for r, buf in enumerate(bufs):
        buffers[r][a] = buf

  if tuple_args:
    buffers = [[xla.make_tuple(bufs, devices[r], backend)]
               for r, bufs in enumerate(buffers)]

  return buffers
pxla.shard_args = shard_args

I'll get this merged into master and google asap.


about training logs

hi, peize, could you provide the training log?
I don't have 8 v100 devices for training , so I need to reduce the batch size.
your training log surely helps me check the training process.
Best.

Hi~ Could you tell me your e-mail? and I will send the training log to you.

Hi~ Could you tell me your e-mail? and I will send the training log to you.

Thanks, please send it to my qq mailbox<PHONE_NUMBER>@qq.com).

Thanks, please send it to my qq mailbox<PHONE_NUMBER>@qq.com).

Hi, Can you send the log files to me too?
Here is my email<EMAIL_ADDRESS>I would appreciate that

Hi, Can you send the log files to me too?
Here is my email<EMAIL_ADDRESS>I would appreciate that

The e-mail has been cc to you.

Hello, Can you send the log files to me too?
Here is my<EMAIL_ADDRESS>I will look forward to your reply


Update _initialize.mdx

🚀 Deployed on https://62e97234dc072d49587482ad--cozy-fox-0defec.netlify.app


Dive into GCP metrics integration

I want to learn how GCP metrics integration works, particularly how metrics and dimensions from GCP are turned into documents in Elasticsearch.
Refs:

https://github.com/elastic/beats/tree/main/x-pack/metricbeat/module/gcp/metrics

I installed the GCP Load Balancing metrics integration end deployed on my cluster and agent.

After a few months, I am troubleshooting a GCP metrics problem using the loadbalancing metricset, but I can't get any metric values.
It seems there's no load balancer in use, so I don't get any value.

I found an existing load balance, so I'm picking its public IP to send some HTTP requests and get some metric values:
while true; do
    # Calculate a sleep interval in the 1-5 range.
    # Thanks to https://www.warp.dev/, I discovered the nice
    # `$RANDOM` variable. This variable contains random
    # numbers between 0 and 32767.
    sleep_interval=$(( RANDOM % 5 + 1))

    curl -i https://<public ip address>:443 --insecure

    echo "Waiting $sleep_interval seconds 🍺"
    sleep $sleep_interval
done

Now I'm getting metric values again!

Load Balancing metrics info are available at https://cloud.google.com/monitoring/api/metrics_gcp#gcp-loadbalancing

For example, here's https/total_latencies:


Use mobile horizontal scroll widget for upcoming livestreams on Wild west + Channel pages

[ ] Add to wild west - needs show all button? Or just show all?

[ ] Add to channel page

_ Add to wild west - needs show all button? Or just show all?_

@tzarebczan, the previous WW Livestream implementation tried to satisfy the need to only show 1 filter header + making it merge seamlessly with regular claims when "Show more livestream" is clicked. It was more of hack using 2 ClaimListDiscover to appear as one.
Is it ok if the 2 lists are not merged seamlessly?
The 2-list design is bad (oops) ... hard to maintain and hard to extend to support the "use horizontal scroll on mobile layout".  Separating it might be easier to handle.
Also, horizontally scrolling a huge list might not be ideal anyway, so maybe "Show more livestreams" should now route to /$/live instead? (or just /$/wildwest/live)

_ Add to wild west - needs show all button? Or just show all?_

@tzarebczan, is it ok if the livestream tiles are not seamlessly merged with the regular tiles, i.e. regular tiles will always start at new row?
Getting rid of this requirement will simplify the code a bit, and makes it easier to add horizontal scroll to the mix.

Yeah, that should be fine!

Add to channel page

Seems like Upcoming is already swipeable in Channel Page.


Reduce feature friction

GIVEN changes in the data
WHEN we need to write a new feature
THEN minimal changes in the plumbing will be necessary

Let's spend a few days exploring some options to use static files or simplify our usage of feathers.

@dankohn mentioned https://github.com/cncf/landscape loading JSON files directly
We also talked about possibly flattening into an  sqlite database.
Devstats has a multi-database merge and import process, but I believe this is only used to export json.
https://github.com/cncf/devstats/blob/master/ARCHITECTURE.md#architecture
I'd like us to think of ways we can process as much of the data as possible before hand and load static files if possible to remove the dependency on a backend. This way we can package the data with the netlify release.

Research being tracked in #115

By revising our data definition for endpoint, and loading the json directly from a bucket, it simplified the logic needed in our front end considerably.  Within the scope of the ticket description, I would say this feature friction has been reduced.


Hangs at the end when one of the SSH processes spawned by git decides to become the control master

Situation: no preexisting SSH control master.  My ~/.ssh/config contains
Host *
ForwardX11 no
ForwardAgent no
ProtocolKeepAlives 300
ServerAliveInterval 300
ControlPath ~/.ssh/control-%r@%h-%p
ControlPersist yes

Host github.com
ControlMaster auto

Output:
$ ./cloneall.py
+ AccessControl
    ControlSocket<EMAIL_ADDRESS>already exists, disabling multiplexing
    git pull exited with 0
+ Acquisition
    ControlSocket<EMAIL_ADDRESS>already exists, disabling multiplexing
    git pull exited with 0
+ AuthEncoding
+ BTrees
    ControlSocket<EMAIL_ADDRESS>already exists, disabling multiplexing
    git pull exited with 0
+ CMF-buildout
+ DateTime
+ DocumentTemplate
+ ExtensionClass
...
+ zope.winbot
+ zope.wineggbuilder
+ zope.z2release
+ zopetoolkit
+ zpt-docs
[====================] 340/340

aaaand it's waiting for something.
Process tree:
$ pstree 22392 -aup
cloneall.py,22392,mg ./cloneall.py
  ├─(git,22405)
  └─{cloneall.py},22399

Reminder about pstree's notation: () is a zombie, {} is a thread.
strace:
$ strace -p 22392 -p 22399
Process 22392 attached
Process 22399 attached
[pid 22399] read(5,  <unfinished ...>
[pid 22392] futex(0x7f7f90000c10, FUTEX_WAIT_PRIVATE, 0, NULL

What's file descriptor 5?
$ lsof -p 22392
...
cloneall. 22392   mg    5r  FIFO   0,10      0t0  502073 pipe
...

What's on the other end?
$ lsof|grep 502073
cloneall. 22392                    mg    5r     FIFO               0,10        0t0     502073 pipe
cloneall. 22392 22399              mg    5r     FIFO               0,10        0t0     502073 pipe
ssh       22519                    mg    2w     FIFO               0,10        0t0     502073 pipe

What's this ssh process?
$ ps 22519
  PID TTY      STAT   TIME COMMAND
22519 ?        Ss     0:00 ssh<EMAIL_ADDRESS>[mux]

Oh.
Killing this ssh process allows cloneall.py to terminate gracefully.

Plan: spawn an ssh process to act as a control master in the background before starting git operations.

Problem: what's the right command line to spawn an SSH control master process?
Something like ssh -fN -M -o ControlPersist=600<EMAIL_ADDRESS>perhaps?
Ok, experiments show that this works, but I probably also want -q:
$ ssh -O exit<EMAIL_ADDRESS>Control socket connect(/home/mg/.ssh/control-git@github.com-22): No such file or directory

$ ssh -fN -M -o ControlPersist=600<EMAIL_ADDRESS>
$ ssh -fN -M -o ControlPersist=600<EMAIL_ADDRESS>ControlSocket<EMAIL_ADDRESS>already exists, disabling multiplexing

$ ssh -O exit<EMAIL_ADDRESS>Exit request sent.

$ ssh -O exit<EMAIL_ADDRESS>Control socket connect(/home/mg/.ssh/control-git@github.com-22): No such file or directory

$ ssh -q -fN -M -o ControlPersist=600<EMAIL_ADDRESS>
$ ssh -q -fN -M -o ControlPersist=600<EMAIL_ADDRESS>
$ ssh -q -fN -M -o ControlPersist=600<EMAIL_ADDRESS>
$ ssh -O exit<EMAIL_ADDRESS>Exit request sent.


Correct CYGWIN LIB_DIR

Update the cygwin check to correctly set LIB_DIR for cygwin os

The issue was reported by slack https://adoptium.slack.com/archives/C5219G28G/p1704231165068949. When running locally (not explicitly set LIB_DIR) on window cygwin , the compile step failed due to missing json-simple.jar.
The PR should also fix the PR tests with github workflow  in temurin-build https://github.com/adoptium/temurin-build/actions/runs/7172315110/job/19613059851.
Locally was tested by Joe Braley.
The PR won't affect jenkins job https://ci.adoptium.net/view/Test_grinder/job/Grinder/8444/console


541 hide division name for x division reports

Resolves #541

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

10 of 10 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 96.204%

Totals

Change from base Build<PHONE_NUMBER>:
0.004%

Covered Lines:
7868

Relevant Lines:
7941

💛  - Coveralls


[S] When Service returns response without body - NPE in ServiceEngine

When Service Adapter returns clientResponse without body following exception occurs:
java.util.NoSuchElementException: No value present
	at java.util.Optional.get(Optional.java:135) ~[na:1.8.0_91]
	at com.cognifide.knotx.knot.view.service.ServiceEngine.mergeWithConfiguration(ServiceEngine.java:72) ~[knotx-example-monolith-0.6.0-SNAPSHOT-fat.jar:na]
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69) ~[knotx-example-monolith-0.6.0-SNAPSHOT-fat.jar:na]
	at rx.observers.Subscribers$5.onNext(Subscribers.java:235) ~[knotx-example-monolith-0.6.0-SNAPSHOT-fat.jar:na]

E.g. case:
Service response with status code 200 and body response when resource is available.
Otherwise service response is status code 400 with no body.

To verify.

Here you have 2 different things:

when service response has empty body, allow to display page without service body values in html (integration test added here https://github.com/Cognifide/knotx/pull/198/files)

when configuration for using service is missing, in fragment processing is:
java.util.NoSuchElementException: No value present
improvement for logging this issue will be done in another task

When I changed fourh.json from "{}" to "" I had 500 error.


Import github.com/sirupsen/logrus as is in log/log.go

Inside of log/log.go we've imported the github.com/sirupsen/logrus package under the alias log which seemed confusing. With this change we import github.com/sirupsen/logrus as is.

Ike Plugins (test-keeper)
Thank you @kwk for this contribution!

It appears that no tests have been added or updated in this PR.
Automated tests give us confidence in shipping reliable software. Please add some as part of this change.
If you are an admin or the reviewer of this PR and you are sure that no test is needed then you can use the command /ok-without-tests as a comment to make the status green.
For more information please head over to official documentation. You can find there how to configure the plugin.

Codecov Report

Merging #2258 into master will not change coverage.
The diff coverage is 65.38%.

@@           Coverage Diff           @@
##           master    #2258   +/-   ##
=======================================
  Coverage   69.34%   69.34%
=======================================
  Files         173      173
  Lines       16358    16358
=======================================
  Hits        11343    11343
  Misses       3922     3922
  Partials     1093     1093

Impacted Files
Coverage Δ

log/log.go
63.19% <65.38%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update daaa486...b65eea9. Read the comment docs.

[test]

/ok-without-tests


Implementing iDEAL and Giropay payment methods with Checkout.com

Feature Description
Adds Giropay and iDEAL payment methods with Checkout.com
Possible Implementation
Adds Giropay and iDEAL payment methods with Checkout.com
Have you spent some time checking if this feature request has been raised before?

[X] I checked and didn't find a similar issue

Have you read the Contributing Guidelines?

[X] I have read the Contributing Guidelines

Are you willing to submit a PR?
Yes, I am willing to submit a PR!

I've resolved this issue from hiring-challenge and raised PR https://github.com/juspay/hyperswitch/pull/4177 for it. Please assign this issue to me.


Fix for new BASE_URL (for EU ?)

@jdhorne
See also: https://github.com/jdhorne/pycarwings2/issues/35
This fixes the connection error which came over the EU users recently.
USA is now using another api/app, see above issue

An updated URL has been included in the pycarwings2-2.2 release on PyPi in my fork of this repo (https://github.com/filcole/pycarwings2).


RedisLockStore support for cluster mode

This error happens when using the RedisLockStore with a Redis instance in cluster mode (e.g. https://aws.amazon.com/elasticache/)

from @rgstephens:

After poking around, it looks like we would have to use a separate redis-py-clustered library because the implementation for clusters is so different.

for now i think the user that's having this issue has found a workaround, so let's postpone for now

Is there a workaround for this issue?

@wochinge Can you share the workaround you used to make the Redislockstore work in cluster mode?

My solution:
cluster_lock_store.py
import logging
import json
from typing import Text, Optional

from rasa.core.lock import TicketLock
from rasa.core.lock_store import LockStore
from rasa.utils.endpoints import EndpointConfig

from rediscluster import RedisCluster

logger = logging.getLogger(__name__)

class RedisClusterLockStore(LockStore):
    def __init__(self, endpoint_config: EndpointConfig) -> None:
        self.startup_nodes = endpoint_config.kwargs['startup_nodes']
        self.key_prefix = endpoint_config.kwargs['key_prefix']
        logger.info(f"startup nodes: {self.startup_nodes}")
        logger.info(f"key_prefix nodes: {self.key_prefix}")
        self.client = RedisCluster(startup_nodes=self.startup_nodes, decode_responses=True)
        super().__init__()

    def lock_key(self, conversation_id: Text) -> Text:
        return self.key_prefix + conversation_id

    def get_lock(self, conversation_id: Text) -> Optional[TicketLock]:
        serialised_lock = self.client.get(self.lock_key(conversation_id))
        if serialised_lock:
            return TicketLock.from_dict(json.loads(serialised_lock))

    def delete_lock(self, conversation_id: Text) -> None:
        deletion_successful = self.client.delete(self.lock_key(conversation_id))
        self._log_deletion(conversation_id, deletion_successful)

    def save_lock(self, lock: TicketLock) -> None:
        self.client.set(self.lock_key(lock.conversation_id), lock.dumps())

endpoints.yml
lock_store:
  type: 'cluster_lock_store.RedisClusterLockStore'
  startup_nodes:
    - host: localhost
      port: 7000
  key_prefix: "chatbot"

To use redis in cluster "natively" just follow this:
lock_store:
  type: redis
  db: 0
  url: your_URM
  port: your port
  key_prefix: your bot name or anything

I hope this usefull

this would be nice to be in the Rasa docs like note


metaedge_to_adjacency_matrix as a Graph class method?

Do we want to make metaedge_to_adjacency_matrix a method of the hetio.hetnet.Graph class? Suggested implementation:
    def metaedge_to_adjacency_matrix(self, *args, **kwargs):
        import hetio.matrix
        hetio.matrix.metaedge_to_adjacency_matrix(self, *args, **kwargs)

This is relevant for changes we're considering in hetmech, specially the construction of a HetMat class.

We decided that this functionality is a better fit for hetmech and that this would be non-standard for hetio's design.


Pair module

Pair module
Add new Pair module with:

Pair : collection with "first" and "second" element, as is common in some programming languages, like C++. Pretty useful, as variation of tuple.
Pair_swap : swapping of two pair. E.g. if we have two pairs: p1 {A, B} and p2 {1, 2}, then after $p1 swap "$p2" we will have p1 {1, 2} and p2 {A, B} respectively.
Pair_swap_args : method to swap fields in existing pair. E.g. if we have pair p1 {A, B}, then after $p1 swap_args we will have p1 {B, A}

with corresponding unit tests
and an addition to the package

Closing as we decided not to have pair in the library due to simplicity to create new structs or anonymous structs.


Exchange MarketSymbol to Currencies Parsing Suggestion

I use the following function to parse a MarketSymbol to its two currencies
ie STRATBTC = "STRAT", "BTC"

It requires an array of QuoteCurrencies for the exchange eg
string[] array = { "XX", "USDT", "EURS", "USD", "BTC", "Y", "ETH", "EOS", "DAI" };
as well as a SymbolFormat for the exchange eg
"{1}{0}" //Hitbtc, Binance
"{0}-{1}" //Bittrex
public static (string BaseCurrency, string QuoteCurrency) ExchangeMarketSymbolToCurrencies_V2(
    string marketSymbol, //eg  string testStr = "STRATBTC";
    string[] quoteCurrencies, //eg string[] array = { "XX", "USDT", "EURS", "USD", "BTC", "Y", "ETH", "EOS", "DAI" };
    string symbolFormat
    )
{
    if(quoteCurrencies == null) return (null, null);
    var currency1 = "";
    var currency2 = "";

    switch(symbolFormat) {
        case "{1}{0}": //eg Hitbtc, Binance
            IEnumerable<IGrouping<int, string>> groups = quoteCurrencies.OrderByDescending(x => x.Length).GroupBy(x => x.Length);
            foreach(IGrouping<int, string> grp in groups) {
                if(grp.Any(x => marketSymbol.EndsWith(x))) {
                    var idx = marketSymbol.Length - grp.Key;
                    currency1 = marketSymbol.Substring(idx);
                    currency2 = marketSymbol.Substring(0, idx);
                    return (currency1, currency2);
                }//if
            }//foreach
            break;

        case "{0}-{1}": //eg Bittrex
            var split = marketSymbol.Split('-');
            currency1 = split[0];
            currency2 = split[1];
            return (currency1, currency2);
    }//switch

    return (null, null);
}//eof

Most of this functionality is built into the ExchangeAPI class, such as symbol separator, etc. A few weird exchanges (Kraken) have non-standard symbol names and they require an internal lookup table.

Good work, can you go through and make sure the new logic works with all of the existing exchanges? Such as factoring in symbolreversal and arbitrary symbol separator. Then, we can replace the original method completely.

Oh, looks like jjxtra put a comment before I finished typing mine. I guess try seeing what you can do to improve the existing method if needed.

The problem with doing it this way is that you need to constantly update the list of currencies for each exchange.

You can either:
(1) Hard code the list of "Quote Currencies" per exchange. Quote markets rarely change. Bittrex for example only has four - { "USDT", "BTC", "ETH", "USD" }
(2) Generate a list of Distinct quote currencies once you have a list of MarketSymbols eg
List listQuoteCurrencies = listMarketSymbols.Select(x => x.QuoteCurrency).Distinct().OrderBy(s => s).ToList();

@GonzoKK, how would you propose to parse symbol "hb10"?

@vslee, It appears that "hb10" and "huobi10" should be treated differently as they are an index of 10 coins (not markets)
https://www.hbg.com/en-us/markets/hb_index/

HB10 Index is composed of 10 coins that are listed on Huobi and have the higher market value, larger scale and better circulation to reflect the overall performance of Huobi Global's market.
The sample space is all listed coins that are traded against USDT at Huobi Global.

The list of quote currencies for Huobi would be restricted to "husd", "usdt", "btc", "eth", "trx", "ht"
This means that "hb10" is NOT included and the above function would return (null, null)

Here's my method to get a list of Quote Currencies for an exchange.
For Huobi it returns "husd", "usdt", "btc", "eth", "trx", "ht"
public List<string> GetQuoteCurrencies(IEnumerable<ExchangeSharp.ExchangeMarket> GetMarketSymbolsMetadataAsync)
{
    List<string> listQuoteCurrencies = GetMarketSymbolsMetadataAsync.Select(x => x.QuoteCurrency).Distinct().OrderByDescending(s => s.Length).ThenBy(s => s).ToList();
    return listQuoteCurrencies;
}

@jjxtra, I'm gradually dipping into the code, so I'm still unfamiliar with many apects.
However it does seem that the "Symbol Separator" method (ExchangeMarketSymbolToCurrencies) is hard coded around a perfect 3/3 split btw CurrencyA and CurrencyB
What happens if quote currency is "USDT" ie 4 chars?

There are many examples of 6 char MarketSymbols where the split is 4-2 or 2-4
Others could be 4-3, 5-2, 5-3, etc

I do think that your parsing logic is better than the 3-3 splitting. However, extensive testing would need to be done with all places where this method is currently used to prevent breaking something in the process of improving things.

Yeah, parser should handle any number of characters and any split ratio.
BTW I see that Binance is hard coded with regards to quote currencies.

It is frustrating that these major exchanges do not use separators. They also can't agree on which order the symbols go in. Very poor planning and architecture. So the best we can do is hard-code the pairs.

@vslee Do you plan on tackling the alternate parsing logic?
As I mentioned, due to my unfamiliarity with the code base, I'm hesitant to make big changes on my own. However I would be more than happy to help out and do research/grunt work.
Let me know. I can help via Telegram or Discord...

I’ve reworked global to market and market to global symbols in the latest master commits. It always checks market symbol metadata now for any exchange without a market symbol separator. Let me know if it’s any better for you.

@jjxtra I'm still encountering exceptions on multiple exchanges for OnGetTickersAsync
The exchanges where OnGetTickersAsync fails include:

Bitbank
Bithumb
Cryptopia
Huobi
Bithumb
ZB.com

What symbols?

the errors vary, but for example on ZB.com

System.InvalidOperationException: Sequence contains no matching element
at System.Linq.Enumerable.First[TSource](IEnumerable1 source, Func2 predicate)
at ExchangeSharp.ExchangeZBcomAPI.d__16.MoveNext() in C:!Dev\ExchangeSharp-master(8)\ExchangeSharp\API\Exchanges\ZBcom\ExchangeZBcomAPI.cs:line 127
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()    at ExchangeSharp.ExchangeAPI.<<GetTickersAsync>b__63_0>d.MoveNext() in C:\!Dev\ExchangeSharp-master(8)\ExchangeSharp\API\Exchanges\_Base\ExchangeAPI.cs:line 592 --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at ExchangeSharp.CryptoUtility.<>c__DisplayClass84_01.<<CacheMethod>b__0>d.MoveNext() in C:\!Dev\ExchangeSharp-master(8)\ExchangeSharp\Utility\CryptoUtility.cs:line 1380 --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at ExchangeSharp.MemoryCache.d__81.MoveNext() in C:\!Dev\ExchangeSharp-master(8)\ExchangeSharp\Utility\MemoryCache.cs:line 202 --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at ExchangeSharp.CryptoUtility.d__841.MoveNext() in C:\!Dev\ExchangeSharp-master(8)\ExchangeSharp\Utility\CryptoUtility.cs:line 1378 --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at ExchangeSharp.ExchangeAPI.d__63.MoveNext() in C:!Dev\ExchangeSharp-master(8)\ExchangeSharp\API\Exchanges_Base\ExchangeAPI.cs:line 592

I'd like to write a test. Does it fail for all symbols or specific symbols?

for zb.com it fails on "xucusdt"

{"xucusdt": {
"high": "0",
"vol": "0",
"last": "1.45",
"low": "0",
"buy": "1.44",
"sell": "1.45"
}}

the reason in this case is that "xuc-usdt" is not found in symbols

i can most likely fix zb.com using my "alternate parser" logic above, however I dont want to go off at a tangent....

This works for me, latest master. I had to fix a json parse error.
ExchangeZBcomAPI api = new ExchangeZBcomAPI();
string marketSymbol = api.GlobalMarketSymbolToExchangeMarketSymbol("USDT-XUC"); // becomes xuc_usdt
var ticker = api.GetTickerAsync(marketSymbol).Sync();

Not resolved, ZB.com still fails OnGetTickersAsync

Looks like a bug on their exchange. http://api.zb.cn/data/v1/markets does not return xuc_usdt. I don't think I want to add hacks for this, so I will skip over not found elements in this call.

Here's my fix for ZB.com using the alternate parser logic...

protected override async Task<IEnumerable<KeyValuePair<string, ExchangeTicker>>> OnGetTickersAsync()
{
    //{ "hpybtc":{ "vol":"500450.0","last":"0.0000013949","sell":"0.0000013797","buy":"0.0000012977","high":"0.0000013949","low":"0.0000011892"},"tvqc":{ "vol":"2125511.1",
    var data = await MakeRequestZBcomAsync(null, "/allTicker", BaseUrl);
    List<KeyValuePair<string, ExchangeTicker>> tickers = new List<KeyValuePair<string, ExchangeTicker>>();
    var symbols = (await GetMarketSymbolsAsync()).ToArray();
    string marketSymbol;

    string SymbolFormat = "{1}{0}"; //eg "xucusdt"
    string[] QuoteCurrencies = new[] { "usdt", "btc", "qc", "zb" }; //hardcoded

    foreach(JToken token in data.Item1) {
        //for some reason when returning tickers, the api doesn't include the symbol separator like it does everywhere else so we need to convert it to the correct format
        string squashedSymbol = token.Path; //eg "xucusdt"

        var codeSwitch = "HardCoded_QuoteCurrencies";
        switch(codeSwitch) {

            case "Existing_Code": ///this crashes on "xucusdt"
                marketSymbol = symbols.First(s => s.Replace(MarketSymbolSeparator, string.Empty).Equals(token.Path));
                tickers.Add(new KeyValuePair<string, ExchangeTicker>(marketSymbol, ParseTickerV2(marketSymbol, token)));
                break;

            case "HardCoded_QuoteCurrencies":
                (string Currency1, string Currency2) pair = CryptoUtility.ExchangeMarketSymbolToCurrencies_V2(squashedSymbol, QuoteCurrencies, SymbolFormat);
                if (!string.IsNullOrEmpty(pair.Currency1) && !string.IsNullOrEmpty(pair.Currency2)) {
                    marketSymbol = pair.Currency2 + MarketSymbolSeparator + pair.Currency1;
                    tickers.Add(new KeyValuePair<string, ExchangeTicker>(marketSymbol, ParseTickerV2(marketSymbol, token)));
                }
                break;
        }//switch
    }
    return tickers;
}

I added ExchangeMarketSymbolToCurrencies_V2 to CryptoUtility.cs

public static (string Currency1, string Currency2) ExchangeMarketSymbolToCurrencies_V2(
    string marketSymbol, //eg  string testStr = "STRATBTC";
    string[] quoteCurrencies, //eg string[] array = { "XX", "USDT", "EURS", "USD", "BTC", "Y", "ETH", "EOS", "DAI" };
    string symbolFormat
    )
{
    if(quoteCurrencies == null) return (null, null);
    string currency1;
    string currency2;

    switch(symbolFormat) {
        case "{1}{0}": //eg Hitbtc, Binance, ZB.com
            IEnumerable<IGrouping<int, string>> groups = quoteCurrencies.OrderByDescending(x => x.Length).GroupBy(x => x.Length);
            foreach(IGrouping<int, string> grp in groups) {
                if(grp.Any(x => marketSymbol.EndsWith(x))) {
                    var idx = marketSymbol.Length - grp.Key;
                    currency1 = marketSymbol.Substring(idx);
                    currency2 = marketSymbol.Substring(0, idx);
                    return (currency1, currency2);
                }//if
            }//foreach
            break;

        case "{0}-{1}": //eg Bittrex
            var split = marketSymbol.Split('-');
            currency1 = split[0];
            currency2 = split[1];
            return (currency1, currency2);
    }//switch

    return (null, null);
}//eof

Result...

Here's my fix for Huobi's OnGetTickersAsync

protected async override Task<IEnumerable<KeyValuePair<string, ExchangeTicker>>> OnGetTickersAsync()
{
    List<KeyValuePair<string, ExchangeTicker>> tickers = new List<KeyValuePair<string, ExchangeTicker>>();
    string symbol;
    JToken obj = await MakeJsonRequestAsync<JToken>("/market/tickers", BaseUrl, null);

    string SymbolFormat = "{1}{0}"; //eg "xlmusdt"
    string[] QuoteCurrencies = new[] { "husd", "usdt", "btc", "eth", "trx", "ht" }; //hardcoded

    //foreach (JToken child in obj["data"])
    foreach(JToken child in obj) {
        symbol = child["symbol"].ToStringInvariant();
        (string Currency1, string Currency2) pair = CryptoUtility.ExchangeMarketSymbolToCurrencies_V2(symbol, QuoteCurrencies, SymbolFormat); //get currency pair
        if(!string.IsNullOrEmpty(pair.Currency1) && !string.IsNullOrEmpty(pair.Currency2)) { //if valid strings
            tickers.Add(new KeyValuePair<string, ExchangeTicker>(symbol, this.ParseTicker(child, symbol, null, null, "close", "amount", "vol",
                baseCurrency2: pair.Currency2, //pass baseCurrency
                quoteCurrency2:pair.Currency1  //pass quoteCurrency
                )));
        }//if
    }//foreach

    return tickers;
}

internal static ExchangeTicker ParseTicker(this ExchangeAPI api, JToken token, string marketSymbol,
    object askKey, object bidKey, object lastKey, object baseVolumeKey,
    object quoteVolumeKey = null, object timestampKey = null, TimestampType timestampType = TimestampType.None,
    object baseCurrencyKey = null, object quoteCurrencyKey = null, object idKey = null,
    string baseCurrency2 = "",
    string quoteCurrency2 = ""
    )
{
    if (token == null || !token.HasValues)
    {
        return null;
    }
    decimal last = token[lastKey].ConvertInvariant<decimal>();

    // parse out volumes, handle cases where one or both do not exist
    token.ParseVolumes(baseVolumeKey, quoteVolumeKey, last, out decimal baseCurrencyVolume, out decimal quoteCurrencyVolume);

    // pull out timestamp
    DateTime timestamp = (timestampKey == null ? CryptoUtility.UtcNow : CryptoUtility.ParseTimestamp(token[timestampKey], timestampType));

    // split apart the symbol if we have a separator, otherwise just put the symbol for base and convert symbol
    string baseCurrency;
    string quoteCurrency;
    if(!string.IsNullOrEmpty(baseCurrency2) && !string.IsNullOrEmpty(baseCurrency2)) {
        baseCurrency = baseCurrency2;
        quoteCurrency = quoteCurrency2;
    }
    else {
        if(baseCurrencyKey != null && quoteCurrencyKey != null) {
            baseCurrency = token[baseCurrencyKey].ToStringInvariant();
            quoteCurrency = token[quoteCurrencyKey].ToStringInvariant();
        }
        else if(string.IsNullOrWhiteSpace(marketSymbol)) {
            throw new ArgumentNullException(nameof(marketSymbol));
        }
        else {
            (baseCurrency, quoteCurrency) = api.ExchangeMarketSymbolToCurrencies(marketSymbol);
        }
    }//else

    // create the ticker and return it
    //JToken askValue = token[askKey];
    //JToken bidValue = token[bidKey];
    JToken askValue = (askKey != null) ? token[askKey] : null;
    JToken bidValue = (bidKey != null) ? token[bidKey] : null;
    if (askValue is JArray)
    {
        askValue = askValue[0];
    }
    if (bidValue is JArray)
    {
        bidValue = bidValue[0];
    }
    ExchangeTicker ticker = new ExchangeTicker
    {
        MarketSymbol = marketSymbol,
        //Ask = askValue.ConvertInvariant<decimal>(),
        //Bid = bidValue.ConvertInvariant<decimal>(),
        Ask = (askValue != null) ? askValue.ConvertInvariant<decimal>() : 0,
        Bid = (bidValue != null) ? bidValue.ConvertInvariant<decimal>() : 0,
        Id = (idKey == null ? null : token[idKey].ToStringInvariant()),
        Last = last,
        Volume = new ExchangeVolume
        {
            BaseCurrencyVolume = baseCurrencyVolume,
            BaseCurrency = baseCurrency,
            QuoteCurrencyVolume = quoteCurrencyVolume,
            QuoteCurrency = quoteCurrency,
            Timestamp = timestamp
        }
    };
    return ticker;
}

Result... Unfortunately Ask and Bid are 0 as Huobi is not providing them

what is that app at the bottom which is showing the results?

Its a very rough user control I wrote to browse/view Exchange Sharp results. Basically a Tree and a Grid.

Here's a log of test results by exchange. Note that Huobi and ZB.com are using the fixes above.
24 Exchanges...
Abucoins
✓ GetTickersAsync: 66 items [0,41s]
✗ GetCurrenciesAsync: ERROR Abucoins does not provide data about its currencies via the API [0,00s]
✓ GetMarketSymbolsAsync: 66 items [0,37s]
✓ GetMarketSymbolsMetadataAsync: 66 items [0,38s]
Binance
✓ GetTickersAsync: 614 items [0,00s]
✓ GetCurrenciesAsync: 202 items [0,73s]
✓ GetMarketSymbolsAsync: 614 items [0,95s]
✓ GetMarketSymbolsMetadataAsync: 614 items [0,00s]
BitBank
✗ GetTickersAsync: ERROR Length cannot be less than zero.
Parameter name: length [0,86s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 8 items [0,00s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,00s]
Bitfinex
✓ GetTickersAsync: 421 items [6,33s]
✗ GetCurrenciesAsync: ERROR Bitfinex does not provide data about its currencies via the API [0,00s]
✓ GetMarketSymbolsAsync: 421 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 421 items [0,00s]
Bithumb
✗ GetTickersAsync: ERROR Splitting Bithumb symbol 'BTC' with symbol separator '-' must result in exactly 2 pieces. [0,50s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 93 items [0,49s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,00s]
BitMEX
✗ GetTickersAsync: ERROR The method or operation is not implemented. [3,72s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 344 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 344 items [0,00s]
Bitstamp
✓ GetTickersAsync: 15 items [45,59s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 15 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 15 items [0,00s]
Bittrex
✓ GetTickersAsync: 364 items [0,00s]
✓ GetCurrenciesAsync: 304 items [0,87s]
✓ GetMarketSymbolsAsync: 364 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 364 items [0,00s]
Coinbase
✓ GetTickersAsync: 51 items [2,03s]
✓ GetCurrenciesAsync: 27 items [0,31s]
✓ GetMarketSymbolsAsync: 51 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 51 items [0,00s]
Cryptopia
✗ GetTickersAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,40s]
✗ GetCurrenciesAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,35s]
✗ GetMarketSymbolsAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,35s]
✗ GetMarketSymbolsMetadataAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,48s]
Gemini
✓ GetTickersAsync: 15 items [45,74s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 15 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 15 items [0,00s]
HitBTC
✓ GetTickersAsync: 892 items [0,00s]
✓ GetCurrenciesAsync: 564 items [2,69s]
✓ GetMarketSymbolsAsync: 892 items [2,92s]
✓ GetMarketSymbolsMetadataAsync: 892 items [0,00s]
Huobi
✓ GetTickersAsync: 535 items [2,65s]
✓ GetCurrenciesAsync: 62 items [0,62s]
✓ GetMarketSymbolsAsync: 542 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 542 items [0,00s]
Kraken
✓ GetTickersAsync: 18 items [197,04s] <<------------------------------ 3 Minutes WTF ?
✓ GetCurrenciesAsync: 29 items [0,73s]
✓ GetMarketSymbolsAsync: 83 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 94 items [0,00s]
KuCoin
✓ GetTickersAsync: 461 items [2,81s]
✗ GetCurrenciesAsync: ERROR {"success":false,"code":"404000","msg":"Not Found","retry":false,"data":null} [1,21s]
✓ GetMarketSymbolsAsync: 461 items [2,60s]
✓ GetMarketSymbolsMetadataAsync: 461 items [3,25s]
LBank
✓ GetTickersAsync: 143 items [3,22s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]WTF
✓ GetMarketSymbolsAsync: 143 items [2,63s]
✓ GetMarketSymbolsMetadataAsync: 143 items [2,37s]
Livecoin
✓ GetTickersAsync: 1021 items [3,20s
✓ GetCurrenciesAsync: 398 items [0,97s]
✓ GetMarketSymbolsAsync: 1021 items [0,95s]
✓ GetMarketSymbolsMetadataAsync: 1021 items [0,97s]
OKCoin
✓ GetTickersAsync: 33 items [0,64s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 33 items [1,37s]
✓ GetMarketSymbolsMetadataAsync: 33 items [0,00s]
OKEx
✓ GetTickersAsync: 449 items [2,09s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 449 items [13,46s]
✓ GetMarketSymbolsMetadataAsync: 449 items [0,00s]
Poloniex
✓ GetTickersAsync: 101 items [0,38s]
✓ GetCurrenciesAsync: 308 items [0,68s]
✓ GetMarketSymbolsAsync: 101 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 101 items [0,34s]
TuxExchange
✗ GetTickersAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [1,35s]
✗ GetCurrenciesAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [0,89s]
✗ GetMarketSymbolsAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [1,35s]
✗ GetMarketSymbolsMetadataAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [0,92s]
UfoDex
✓ GetTickersAsync: 20 items [0,99s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✗ GetMarketSymbolsAsync: ERROR The method or operation is not implemented. [0,00s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,00s]
Yobit
SKIPPING FOR NOW - TAKES TOO LONG
ZBcom
✓ GetTickersAsync: 187 items [2,29s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,00s]
✓ GetMarketSymbolsAsync: 185 items [0,00s]
✓ GetMarketSymbolsMetadataAsync: 185 items [0,85s]

Updated results... lotsa fixes, including Huobi, ZB.com and Bithumb.
Bitstamp (45sec) and Gemini (45 sec) very sloooooooooow.
TuxExchange's API is disabled.
Cryptopia is broken.
24 Exchanges...
Abucoins
✓ GetTickersAsync: 66 items [0,486s]
✗ GetCurrenciesAsync: ERROR Abucoins does not provide data about its currencies via the API [0,002s]
✓ GetMarketSymbolsAsync: 66 items [0,205s]
✓ GetMarketSymbolsMetadataAsync: 66 items [0,375s]
🡆 QuoteCurrencies: "BTC", "EUR", "PLN", "USD"
🡆 SampleMarkets: "ETH-USD", "MTL-BTC", "PPT-BTC", "VERI-BTC", "XEM-BTC", "XRP-BTC"
Binance
✓ GetTickersAsync: 617 items [0,000s]
✓ GetCurrenciesAsync: 202 items [0,345s]
✓ GetMarketSymbolsAsync: 617 items [0,423s]
✓ GetMarketSymbolsMetadataAsync: 617 items [0,001s]
🡆 QuoteCurrencies: "TUSD", "USDC", "USDS", "USDT", "BNB", "BTC", "ETH", "PAX", "TRX", "XRP"
🡆 SampleMarkets: "ALGOBNB", "BCHSVUSDT", "BCNETH", "ERDUSDT", "WINTRX", "ZENBNB"
BitBank
✗ GetTickersAsync: ERROR Market symbol btc_jpy is missing the global symbol separator '-' [0,869s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,001s]
✓ GetMarketSymbolsAsync: 8 items [0,000s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,001s]
Bitfinex
✓ GetTickersAsync: 421 items [6,295s]
✗ GetCurrenciesAsync: ERROR Bitfinex does not provide data about its currencies via the API [0,000s]
✓ GetMarketSymbolsAsync: 421 items [0,001s]
✓ GetMarketSymbolsMetadataAsync: 421 items [0,000s]
🡆 QuoteCurrencies: "F0:USTF0", "BTC", "DAI", "EOS", "ETH", "EUR", "GBP", "JPY", "USD", "UST", "XCH", "XLM"
🡆 SampleMarkets: "atmeth", "rcnusd", "snteth", "ustusd", "xlmeur", "xmrusd"
Bithumb
✓ GetTickersAsync: 93 items [1,837s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,000s]
✓ GetMarketSymbolsAsync: 93 items [0,480s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,000s]
BitMEX
✗ GetTickersAsync: ERROR The method or operation is not implemented. [3,107s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,001s]
✓ GetMarketSymbolsAsync: 344 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 344 items [0,000s]
🡆 QuoteCurrencies: "CNY", "ETH", "JPY", "KRW", "USD", "XBT", "XXX"
🡆 SampleMarkets: ".BVOL", ".XBTKRW", "LTCM18", "XBTM19", "XBTQ15", "XMRZ17"
Bitstamp
✓ GetTickersAsync: 15 items [46,856s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,001s]
✓ GetMarketSymbolsAsync: 15 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 15 items [0,000s]
🡆 QuoteCurrencies: "BTC", "EUR", "USD"
🡆 SampleMarkets: "bchusd", "btcusd", "ethbtc", "ltceur", "xrpbtc", "xrpeur"
Bittrex
✓ GetTickersAsync: 365 items [0,000s]
✓ GetCurrenciesAsync: 305 items [0,427s]
✓ GetMarketSymbolsAsync: 365 items [0,001s]
✓ GetMarketSymbolsMetadataAsync: 365 items [0,000s]
🡆 QuoteCurrencies: "USDT", "BTC", "ETH", "USD"
🡆 SampleMarkets: "BTC-ENG", "BTC-ETH", "BTC-SPIN", "BTC-TRX", "BTC-VIB", "ETH-BCH"
Coinbase
✓ GetTickersAsync: 51 items [2,677s]
✓ GetCurrenciesAsync: 27 items [0,277s]
✓ GetMarketSymbolsAsync: 51 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 51 items [0,000s]
🡆 QuoteCurrencies: "USDC", "BTC", "DAI", "ETH", "EUR", "GBP", "USD"
🡆 SampleMarkets: "BAT-USDC", "DNT-USDC", "ETH-GBP", "GNT-USDC", "XTZ-BTC", "ZRX-BTC"
Cryptopia
✗ GetTickersAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,527s]
✗ GetCurrenciesAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,321s]
✗ GetMarketSymbolsAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,342s]
✗ GetMarketSymbolsMetadataAsync: ERROR Unexpected character encountered while parsing value: <. Path '', line 0, position 0. [0,057s]
Gemini
✓ GetTickersAsync: 15 items [45,859s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,001s]
✓ GetMarketSymbolsAsync: 15 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 15 items [0,000s]
🡆 QuoteCurrencies: "BCH", "BTC", "ETH", "LTC", "USD"
🡆 SampleMarkets: "bchbtc", "btcusd", "ethusd", "zecbch", "zecbtc", "zeceth"
HitBTC
✓ GetTickersAsync: 892 items [2,874s]
✓ GetCurrenciesAsync: 564 items [2,101s]
✓ GetMarketSymbolsAsync: 892 items [2,532s]
✓ GetMarketSymbolsMetadataAsync: 892 items [0,000s]
🡆 QuoteCurrencies: "USDT20", "EOSDT", "EURS", "GUSD", "KRWB", "TUSD", "USDC", "BCH", "BTC", "DAI", "EOS", "ETH", "PAX", "USD"
🡆 SampleMarkets: "CBCBTC", "GSTETH", "IGNISETH", "LOCBTC", "POABTC", "XUCETH"
Huobi
✓ GetTickersAsync: 535 items [1,143s]
✓ GetCurrenciesAsync: 62 items [0,412s]
✓ GetMarketSymbolsAsync: 542 items [0,001s]
✓ GetMarketSymbolsMetadataAsync: 542 items [0,000s]
🡆 QuoteCurrencies: "husd", "usdt", "btc", "eth", "trx", "ht"
🡆 SampleMarkets: "bixeth", "gasbtc", "idtbtc", "nasbtc", "powreth", "xlmusdt"
Kraken
✓ GetTickersAsync: 83 items [1,629s]
✓ GetCurrenciesAsync: 29 items [0,425s]
✓ GetMarketSymbolsAsync: 83 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 94 items [0,000s]
🡆 QuoteCurrencies: "XETH", "XXBT", "ZCAD", "ZEUR", "ZGBP", "ZJPY", "ZUSD"
🡆 SampleMarkets: "ADACAD", "BCHEUR", "LTCUSD", "WAVESETH", "XLMEUR", "ZECXBT"
KuCoin
✓ GetTickersAsync: 462 items [1,584s]
✗ GetCurrenciesAsync: ERROR {"success":false,"code":"404000","msg":"Not Found","retry":false,"data":null} [1,159s]
✓ GetMarketSymbolsAsync: 462 items [1,931s]
✓ GetMarketSymbolsMetadataAsync: 462 items [1,193s]
🡆 QuoteCurrencies: "NUSD", "TUSD", "USDC", "USDT", "BTC", "DAI", "ETH", "KCS", "NEO", "PAX"
🡆 SampleMarkets: "BTC-USDC", "CS-ETH", "DBC-BTC", "HKN-ETH", "USDT-PAX", "VNX-BTC"
LBank
✓ GetTickersAsync: 144 items [6,286s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,001s]
✓ GetMarketSymbolsAsync: 144 items [2,587s]
✓ GetMarketSymbolsMetadataAsync: 144 items [2,440s]
🡆 QuoteCurrencies: "AUSD", "USDT", "BTC", "ETH"
🡆 SampleMarkets: "atp_usdt", "bloc_usdt", "brc_usdt", "eos_btc", "etc_btc", "vet_eth"
Livecoin
✓ GetTickersAsync: 1023 items [1,865s]
✓ GetCurrenciesAsync: 399 items [1,382s]
✓ GetMarketSymbolsAsync: 1023 items [0,996s]
✓ GetMarketSymbolsMetadataAsync: 1023 items [0,999s]
🡆 QuoteCurrencies: "DASH", "DOGE", "EURN", "EURS", "TUSD", "USDC", "USDT", "XSCR", "BCH", "BTC", "EMC", "EOS", "ETH", "EUR", "LEO", "LKE", "LTC", "PAX", "RUR", "TER", "TRX", "USD"
🡆 SampleMarkets: "CASH/BTC", "CLPC/ETH", "CRC/ETH", "HOT/ETH", "MCR/ETH", "UNRC/TRX"
OKCoin
✓ GetTickersAsync: 33 items [0,758s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,000s]
✓ GetMarketSymbolsAsync: 33 items [0,368s]
✓ GetMarketSymbolsMetadataAsync: 33 items [0,000s]
🡆 QuoteCurrencies: "EURS", "GUSD", "TUSD", "USDC", "USDT", "EUR", "PAX", "USD"
🡆 SampleMarkets: "BTC-TUSD", "ETH-EUR", "GRIN-USD", "USDK-USD", "XRP-USD", "ZEC-USD"
OKEx
✓ GetTickersAsync: 450 items [2,113s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,000s]
✓ GetMarketSymbolsAsync: 450 items [13,301s]
✓ GetMarketSymbolsMetadataAsync: 450 items [0,000s]
🡆 QuoteCurrencies: "USDK", "USDT", "BTC", "ETH", "OKB", "USD"
🡆 SampleMarkets: "BLOC-BTC", "DGB-USDT", "EGT-BTC", "FAIR-USDT", "INS-BTC", "KAN-BTC"
Poloniex
✓ GetTickersAsync: 101 items [1,098s]
✓ GetCurrenciesAsync: 308 items [1,100s]
✓ GetMarketSymbolsAsync: 101 items [0,001s]
✓ GetMarketSymbolsMetadataAsync: 101 items [0,437s]
🡆 QuoteCurrencies: "USDC", "USDT", "BTC", "ETH"
🡆 SampleMarkets: "BTC_FCT", "BTC_OMNI", "BTC_POLY", "BTC_SC", "USDC_BCHSV", "USDT_ATOM"
TuxExchange
✗ GetTickersAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [0,927s]
✗ GetCurrenciesAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [0,922s]
✗ GetMarketSymbolsAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [0,886s]
✗ GetMarketSymbolsMetadataAsync: ERROR {
"success": "0",
"error": "The API is currently disabled. Please try again later."
} [1,149s]
UfoDex
✓ GetTickersAsync: 20 items [1,049s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,004s]
✗ GetMarketSymbolsAsync: ERROR The method or operation is not implemented. [0,000s]
✗ GetMarketSymbolsMetadataAsync: ERROR The method or operation is not implemented. [0,001s]
Yobit
SKIPPING FOR NOW - TAKES TOO LONG
ZBcom
✓ GetTickersAsync: 187 items [2,403s]
✗ GetCurrenciesAsync: ERROR The method or operation is not implemented. [0,000s]
✓ GetMarketSymbolsAsync: 187 items [0,000s]
✓ GetMarketSymbolsMetadataAsync: 187 items [0,853s]
🡆 QuoteCurrencies: "usdt", "btc", "qc", "zb"
🡆 SampleMarkets: "eos_btc", "eos_usdt", "gram_btc", "slt_btc", "xmr_usdt", "zb_btc"

Bitstamp and Gemini have no work-around, no ability to get tickers in one call, no web socket tickers. Have to get tickers one at a time. No speed improvements by making the requests in parallel...


Format test results to markdown

This PR adds a new HSpec formatter to generate a markdown file representing all the test cases for some package as a tree-structured document. The current formatter is rather crude, just listing the labels of each it and describe action, but it could be easily enhanced with more information (eg. execution time, URL to source location...)
To check before merging:

[x] CHANGELOG is up to date
[x] Up to date with master

Locally generated output


remove store ability of LockedCoin

LockedCoin shouldn't be allowed to be transferred and thus shouldn't have store ability.

It would be great if we had a way to test that trying to do transfer(locked_coin) makes the verifier complain...

Added a verifier test that does this.


HOSTEDCP-1694: Automate HO Deployment Tag & Commit Updates for ROSA HCP

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes HOSTEDCP-1694
Checklist

[x] Subject and description added to both, commit and PR.
[x] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

One minor comment, otherwise looks like good starting point to me.

/test verify

@bryan-cox i will review when you push the rebase!

@stevekuznetsov - it's been rebased now


Activity closes after few scans without any crash logs

Hi,
I have used your library and I am using it for continuously scanning QR codes. It scans perfectly for few and then suddenly the activity is killed without any crash logs or anything. It doesn't have any particular pattern sometimes it closes after 1000+ or sometimes it closes at 40.

I am going to archive this repo. Unfortunately, I don't have the time to actively maintain it


eq of range problem

Feature
Expected Result
>>> range(0, 10, 3) == range(0, 11, 3)
True
>>> range(3, 1, 10) == range(5, 2, 1000)
True

Actual Result
>>>>> range(0, 10, 3) == range(0, 11, 3)
False
>>>>> range(3, 1, 10) == range(5, 2, 1000)
False

The eq function of range is invalid.
It is not enough just to compare start, stop, and step values.

If this issue is valid, can I try this issue?

Yep, go ahead!


Connection problem with twitter REST API

Everything was working fine until a few hours ago now I'm getting this: Is it only me or anyone one else?  Or is this the end of Rainbowstream. That would be mighty sad.
Thanks, I would be grateful if anyone would let me know. Cheers
"We have connection problem with twitter REST API right now :(
You currently have access to a subset of Twitter API v2 endpoints and limited v1.1 endpoints (e.g. media post, oauth) only. If you need access to this endpoint, you may need a different access level."
You can learn more here: https://developer.twitter.com/en/portal/product

same here,started yesterday.

Ok, I suppose I was hoping that it was just me and not what I already knew. Is this 'fixable' or is it really the end?   😫 😫

not fixable w/o paying the thousands of dollars a month to twitter for API access, if I understand correctly :(
we'll need to write a new app that uses bluesky instead! :)

i always thought rainbowstream would make a great rss reader. if rainbowstream does not return.. thanks to the devs for making and maintaining rainbowstream for so long. i've used it for years, everyone who sees it on my desktop asks about it.

I realize rainbowstream is gone forever. I just want to thank you for it's creation in the first place and it was a pleasure to even use it for as long as i had <3. This app was hella underrated and deserved way more attention

I realize rainbowstream is gone forever. I just want to thank you for it's creation in the first place and it was a pleasure to even use it for as long as i had <3. This app was hella underrated and deserved way more attention

Concur with your post most wholeheartedly (albeit sadly). The first thing I used to do on booting my linux box, was to open a terminal and fire up Rainbowstream.  I use Arch btw! 😋 😋

yeah, it's gone and I got the same issue. I wish we can port this client to mastodon instead

yeah, it's gone and I got the same issue. I wish we can port this client to mastodon instead

A cli client for Mastodon that can do pretty much most of what you need on Mastodon is called "Tootstream"