PLT-7064: Redo PIR purity analysis

This PR went down a bit of a rabbit-hole. I was looking at firstEffectfulTerm, and it is tricky to get right. In particular, it reasons about the evaluation order of the program. So I thought: the best way to find the first thing satisfying a predicate in a list is to just produce the whole list and then look for the thing. (It's also easier to test, since we can produce the whole evaluation order for a temrm and check to see if it makes sense.)
That led to termEvaluationOrder. Then I realised that you can use termEvaluationOrder to implement isPure (and otherwise they share a lot of the same logic!).
Finally, the use of firstEffectfulTerm itself was slightly weird: the thing we are looking for is in fact not an effectful term, since it's an occurrence of a strict variable. So I wrote something more accurate in the module where it's used.
So this amounts to a bit of a rewrite of the purity analysis, but I think the result is more general and easier to understand and test.
We also get some modest PIR and budget improvements. Most of the PIR changes seem to be the firstEffectfulTerm check just working properly.
Since we have this logic in UPLC also, I'll update UPLC to use the same strategy if we decide we like this.

Pre-submit checklist:

Branch

[x] Tests are provided (if possible)
[x] Commit sequence broadly makes sense
[x] Key commits have useful messages
[x] Changelog fragments have been written (if appropriate)
[x] Relevant tickets are mentioned in commit messages
[x] Formatting, PNG optimization, etc. are updated

PR

[ ] (For external contributions) Corresponding issue exists and is linked in the description
[x] Targeting master unless this is a cherry-pick backport
[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

That sounds counter intuitive. If you can build the whole list surely it's not harder to find the first effectful term? You are using Seq for the list which is strict and can lead to building a large list of which you only need the first few elements.

I meant it to be lazy! But you're right, Seq is strict in the structure, but not the elements (oops). I'll write some tests with some undefined terms to make sure I get it lazy enough.
I do think that phrasing a search as a generate+filter is generally nice, especially since you can use the generator for other things. For example, I think to do your suggestion here we would need to look past the first effectful term, so then we'd be rewriting anything.

stop building the list once you hit the first Unknown

The semigroup instance should do this if it's lazy enough. But I need to test that.

There are cases where this change causes more inlining, and there are also cases where it causes less inlining.

Where are the cases with less inlining? Just the ones where it was buggy before? I think those were (hopefully) rare. I think the reason this is a general win is that it's enabling more inlining of single-occurrence bindings, which is basically always a win so long as we let ourselves do it!

I made it properly lazy, which turned out to be easy with just a DList. Test shows that we can analyze a term with undefined in it just fine.


Complete TODO: pass Interval object

This PR aims at completing the TODO: pass Interval object written by @adyang1.
I'm decently confident it satisfies the requirements, but I'm not exactly sure about the testing workflow you're adopting in this project so I didn't properly tested it locally.
Hope it helps.

Thanks for your contributions! This one is particularly helpful since it has been nagging me for a while (but I never got around to completing it).
Unfortunately there aren't any unit tests yet for most of those functions -- hopefully I'll get around to them in the near future. At the moment the mu_bound_examples() and beta_bound_examples() functions in examples.py offer some sanity checks.

@adyang1 Glad to hear it's been helpful.


Enhancement/integration tests/search for exercise

Checklist

[x] Server: I added multiple integration tests (Spring) related to the features (with a high test coverage)
[x] Server: I implemented the changes with a good performance and prevented too many database calls

Motivation and Context

according to issue #2060 the test of the method getAllExercisesOnPage() is still not fully covered. To ensure a more stable functionality of this method, this pr has been made
Description

In class ProgrammingExerciseServiceIntegrationTest

Implemented searchExercises_instructor_getResultsFromOwningCourses_thatIsNotEmpty()
to test if the instructor in the group get the right search result of his/her programming exercise
Implemented searchExercisesWithProperSearchTerm()
to test if the method react correctly to different search term (search term matches the title of exercise, search term matches the prefix of the title, search term doesn't match)

In class TextExerciseIntegrationTest and ModelingExerciseIntegrationTest

implemented searchExercises_instructor_getResultsFromOwningCourses_thatIsNotEmpty()
implemented searchExercises_instructor_shouldOnlyGetResultsFromOwningCourses()
with the same reason addressed above

Test Coverage

getAllOnPageWithSize(PageableSearchDTO, User) in
TextExerciseService.java 0% in development -->  100% in this branch
ModelingExerciseResource.java 0% in development --> 100% in this branch
ProgrammingExerciseService.java 85% in development --> 100% in this branch
Current test coverage in 3 classes:
TextExerciseService.java 41% in development --> 100% in this branch

ModelingExerciseService.java 85% in dev -> 88% in this branch

ProgrammingExerciseService.java 79% in development -> 79% in this branch

getAllOnPageWithSize(PageableSearchDTO, User) in
TextExerciseService.java 0% in development -->  100% in this branch
ModelingExerciseResource.java 0% in development --> 100% in this branch
ProgrammingExerciseService.java 85% in development --> 100% in this branch
Current test coverage in 3 classes:
TextExerciseService.java 41% in development --> 100% in this branch

ModelingExerciseService.java 85% in dev -> 88% in this branch

ProgrammingExerciseService.java 79% in development -> 79% in this branch

@anditurdiu mentioned change covered in https://github.com/ls1intum/Artemis/pull/2070/commits/067a62cd838a79f09de09c1b1e43ec8238c44dca


attempt 1

Codecov Report

Merging #2190 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2190      +/-   ##
==========================================
+ Coverage   73.88%   73.88%   +<.01%
==========================================
  Files          51       51
  Lines        4185     4186       +1
==========================================
+ Hits         3092     3093       +1
  Misses       1093     1093

Impacted Files
Coverage Δ

src/data.convert.js
79.56% <100%> (+0.15%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0391272...347938f. Read the comment docs.

sorry, misclick


Temporarily switch to GitHub version of webpki.

This will allow Rustls to temporarily switch to the GitHub version as
well, while client-auth work happens in webpki & Rustls.

Never mind, this isn't useful.


Can open the Gitter chat room?

@davezuko  Gitter

Hey, thanks. I still have to configure the integrations but the room has been created: https://gitter.im/davezuko/react-redux-starter-kit


Magic.commands.wand.restore

This permission does not work.

You may also need to give Magic.commands.wand

Okay, now it works.


Convert to using the pure JS port of tiktoken

the wasm-bindings for tiktoken is causing a few issues when running in cloudrun.
We've seen errors like:

RuntimeError: memory access out of bounds
Error: unwrap_throw

which seem to crash the instance and prevent it from recovering.
Replace the wasm version with a pure JS implementation (same author)
🎩
dev up

# Grab a personal token from https://openai-proxy.shopify.io/

OPENAI_API_HOST=https://openai-proxy.shopify.ai OPENAI_API_KEY=<token from proxy> dev s

🙅  Accidentally opened it against the upstream: https://github.com/mckaywrigley/chatbot-ui/pull/1033

Tried tophatting locally - I think there are some lingering references to the old tiktoken package: https://github.com/Shopify/chatbot-ui/blob/9bb277ee446304923394bca8a8e1b3c83a360d62/components/Chat/ChatInputTokenCount.tsx#L6


Disable doclint for Java8

The project doesn't build here for Java8, with some javadoc comments failing the build because of doclint.
For example: BuildDependencyPattern triggers a 'bad use of >' for its comment line "libs-release-local:com/goldin/plugins/gradle/0.1.1/*.jar;status+=prod@gradle-plugins :: Build :: Gradle#LATEST => many-jars-build"
Simply disable doclint for now...
See http://blog.joda.org/2014/02/turning-off-doclint-in-jdk-8-javadoc.html

Thanks for looking into this @ajeans.
We've just pushed this commit, which fixes the javadoc errors. This should allow building the code with JDK8.
Please let us know what you think.

Thanks for the quick reply @eyalbe4
Feel free to close the PR then, it is a much cleaner fix.


MigrationVMStepsType step to display human readable value

This addresses the case where the step were displaying an internal value instead of human friendly string.

Oh, is the verbose description being added to the API data? We could just display that directly instead of mapping types to strings in the UI code, that'd be much better.

@mturley, I suppose we could just keep this one in the meantime it won't hurt.


Gradient computation occupies too much memories in "cnn (using while_loop) + lstm" network

System information

Have I written custom code: No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.3
Python version: 3.6
CUDA/cuDNN version: 8.0/5.1
GPU model and memory: GTX Titan X 12GB
Bazel version: No use
Exact command to reproduce: using python *.py

Describe the problem
I'm using while_loop function to build a cnn because of the scale of input tensors. And put the cnn feature into a lstm structure. The problem is that if I only do the forward computatoin it is good, but if I add the backward computation GradientDescentOptimizer().minimize(loss)) the memory is significantly insufficient .
I tried to split the two part of the model -- cnn and lstm part, and both do well with the whole computation. I think this is the fact:

Single while_loop cnn net is treated as time distributed when computing gradient. Every temporary feature vector and gradients occupy the same memory location at each time step.
When coneected with a lstm, the cnn net will be treated as many subnet in computing gradients. At every time step in backward computation, it will occupy new memory for its temporary feature vector and gradients. The number of timesteps is very large so that the memory is significantly not enough.

Here is a simplified code of my project. It will show my cnn+lstm structure:
Source code
import tensorflow as tf
import tensorflow.contrib as contrib

def vgg_m(input_layer, reuse=None):
    # input shape [batch_size, 120, 120, 5]
    with tf.variable_scope('vgg_m', reuse=reuse):
        conv_1 = tf.layers.conv2d(input_layer, 96, [3, 3], padding='same', activation=tf.nn.relu, name='conv_1')
        pool_1 = tf.layers.max_pooling2d(conv_1, 3, 2, padding='same', name='pool_1')
        norm_1 = tf.layers.batch_normalization(pool_1, name='norm_1')

        conv_2 = tf.layers.conv2d(norm_1, 256, [3, 3], padding='same', activation=tf.nn.relu, name='conv_2')
        pool_2 = tf.layers.max_pooling2d(conv_2, 3, 2, padding='same', name='pool_2')
        norm_2 = tf.layers.batch_normalization(pool_2, name='norm_2')

        conv_3 = tf.layers.conv2d(norm_2, 512, [3, 3], padding='same', activation=tf.nn.relu, name='conv_3')

        conv_4 = tf.layers.conv2d(conv_3, 512, [3, 3], padding='same', activation=tf.nn.relu, name='conv_4')

        conv_5 = tf.layers.conv2d(conv_4, 512, [3, 3], padding='same', activation=tf.nn.relu, name='conv_5')
        pool_5 = tf.layers.max_pooling2d(conv_5, 3, 2, padding='same', name='pool_5')

        flatten_6 = contrib.layers.flatten(pool_5)
        fc_6 = tf.layers.dense(flatten_6, 512, name='fc6')

    return fc_6

# [batch, time_step, image_h, image_w, image_channel
input_tensor = tf.random_normal([64, 150, 120, 120, 5], dtype=tf.float32)

# vgg_m is a cnn net whose input and output size is [batch, 120, 120, 5], [batch, 512]
# in order to create vgg_m variables for reuse later.
vgg_m(input_tensor[:, 0, :, :, :])

time_steps = 150
initial_t = tf.constant(0, dtype=tf.int32)
initial_outputs = tf.TensorArray(dtype=tf.float32, size=time_steps)

def _should_continue(t, *args):
    return t < time_steps

def _iteration(t, outputs_):
    # compute cnn feature at time t
    single_output = vgg_m(input_tensor[:, t, :, :, :], reuse=True)
    outputs_ = outputs_.write(t, single_output)
    return t+1, outputs_

_, outputs = tf.while_loop(_should_continue, _iteration, [initial_t, initial_outputs])

# transpose the batch dim and time dim to build a [batch, time_step, 512] feature and send to lstm
outputs = tf.transpose(outputs.stack(), perm=[1, 0, 2])
outputs = tf.reshape(outputs, [-1, 150, 512])

lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(512)
lstm_outputs, lstm_state = tf.nn.dynamic_rnn(
            lstm_cell,
            outputs,
            sequence_length=tf.constant(150, dtype=tf.int32, shape=[64]),
            dtype=tf.float32,
        )

# not really a "loss", just perform an loss example
loss = tf.reduce_max(tf.reduce_max(tf.reduce_max(lstm_outputs, -1), -1), -1)
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run(train_op)

Log
2017-12-01 22:44:24.228695: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-01 22:44:24.228717: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-01 22:44:24.228725: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-12-01 22:44:24.228731: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-01 22:44:24.228736: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-12-01 22:44:26.620700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX TITAN X
major: 5 minor: 2 memoryClockRate (GHz) 1.076
pciBusID 0000:4c:00.0
Total memory: 11.92GiB
Free memory: 11.81GiB
2017-12-01 22:44:26.620728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-12-01 22:44:26.620735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y
2017-12-01 22:44:26.620742: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:4c:00.0)
2017-12-01 22:44:29.102674: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.33GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available.
2017-12-01 22:44:29.102971: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.33GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available.
2017-12-01 22:44:29.103146: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 748.02MiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available.
2017-12-01 22:44:39.103321: W tensorflow/core/common_runtime/bfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 112.50MiB.  Current allocation summary follows.
2017-12-01 22:44:39.103363: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (256):   Total Chunks: 3, Chunks in use: 0 768B allocated for chunks. 20B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103372: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (512):   Total Chunks: 1, Chunks in use: 0 512B allocated for chunks. 4B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103379: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (1024):  Total Chunks: 1, Chunks in use: 0 1.0KiB allocated for chunks. 1.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103384: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (2048):  Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
...(many Chunks)
2017-12-01 22:44:39.103507: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (67108864):      Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103514: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (134217728):     Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103521: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (268435456):     Total Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:39.103528: I tensorflow/core/common_runtime/bfc_allocator.cc:660] Bin for 112.50MiB was 64.00MiB, Chunk State:
2017-12-01 22:44:39.103536: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x230a3c0000 of size 1280
2017-12-01 22:44:39.103542: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x230a3c0500 of size 256
2017-12-01 22:44:39.103547: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x230a3c0600 of size 512
...(many Chunks)
2017-12-01 22:44:39.104420: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x25cc3c8000 of size 88473600
2017-12-01 22:44:39.104425: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x25d1828000 of size 122539008
2017-12-01 22:44:39.104430: I tensorflow/core/common_runtime/bfc_allocator.cc:693]      Summary of in-use Chunks by size:
2017-12-01 22:44:39.104438: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 19 Chunks of size 256 totalling 4.8KiB
...(many Chunks)
2017-12-01 22:44:39.104599: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 245891072 totalling 234.50MiB
2017-12-01 22:44:39.104605: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 268435456 totalling 256.00MiB
2017-12-01 22:44:39.104611: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 5 Chunks of size 353894400 totalling 1.65GiB
2017-12-01 22:44:39.104617: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 405815296 totalling 387.02MiB
2017-12-01 22:44:39.104623: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 513720320 totalling 489.92MiB
2017-12-01 22:44:39.104629: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size<PHONE_NUMBER> totalling 4.00GiB
2017-12-01 22:44:39.104635: I tensorflow/core/common_runtime/bfc_allocator.cc:700] Sum Total of in-use chunks: 11.22GiB
2017-12-01 22:44:39.104644: I tensorflow/core/common_runtime/bfc_allocator.cc:702] Stats:
Limit:<PHONE_NUMBER>7
InUse:<PHONE_NUMBER>4
MaxInUse:<PHONE_NUMBER>4
NumAllocs:                     297
MaxAllocSize:<PHONE_NUMBER>

2017-12-01 22:44:39.104660: W tensorflow/core/common_runtime/bfc_allocator.cc:277] ********x************************xxxxxxxxxxx********************************************************
2017-12-01 22:44:39.104678: W tensorflow/core/framework/op_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[64,512,30,30]
2017-12-01 22:44:49.104957: W tensorflow/core/common_runtime/bfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 225.00MiB.  Current allocation summary follows.
2017-12-01 22:44:49.105020: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (256):   Total Chunks: 3, Chunks in use: 0 768B allocated for chunks. 20B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:49.105043: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (512):   Total Chunks: 1, Chunks in use: 0 512B allocated for chunks. 4B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
2017-12-01 22:44:49.105063: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin (1024):  Total Chunks: 1, Chunks in use: 0 1.0KiB allocated for chunks. 1.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.
...(similar chunks)
2017-12-01 22:44:49.107700: I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size<PHONE_NUMBER> totalling 4.00GiB
2017-12-01 22:44:49.107714: I tensorflow/core/common_runtime/bfc_allocator.cc:700] Sum Total of in-use chunks: 11.11GiB
2017-12-01 22:44:49.107731: I tensorflow/core/common_runtime/bfc_allocator.cc:702] Stats:
Limit:<PHONE_NUMBER>7
InUse:<PHONE_NUMBER>6
MaxInUse:<PHONE_NUMBER>4
NumAllocs:                     297
MaxAllocSize:<PHONE_NUMBER>

2017-12-01 22:44:49.107769: W tensorflow/core/common_runtime/bfc_allocator.cc:277] ********x************************xxxxxxxxxxx*******************************************************_
2017-12-01 22:44:49.107799: W tensorflow/core/framework/op_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[64,60,60,256]
2017-12-01 22:44:49.107877: W tensorflow/core/framework/op_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[64,512,30,30]
         [[Node: while/vgg_m/conv_4/convolution = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](while/vgg_m/conv_3/Relu, while/vgg_m/conv_4/convolution/Enter)]]

....(same thing occuring again and again util it is closed.

Is this an unknown bug
I saw a different processing with loop_state in gradients function, so is this an unknown bug with multiple loop operation(lstm contains loop as well)

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Bazel version
Exact command to reproduce

Description updated, and anyone could help?

/CC @zheng-xq @ebrevdo, can you comment?

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Nagging Awaiting TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Nagging TensorFlower: It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Nagging Assignee @ebrevdo: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

This seems not to be a bug so much as a question of how to structure architectures to reduce gradient memory usage.  Ask on StackOverflow and look at rev_block.

btw; one of the main reasons your memory is blowing up may be this line:
    single_output = vgg_m(input_tensor[:, t, :, :, :], reuse=True)

transpose input_tensor to time, major, unstack it into a TensorArray, and use .read(t) on the TensorArray.  this should significantly reduce memory usage if you backprop into input_tensor.


Teton example : signalign assertion error after printing options??

Originally reported by: Donna Calhoun (Bitbucket: donnaaboise, GitHub: donnaaboise)

This command
mpirun -n 1 teton --print-options

produces an error
...
[fclaw]    geoclaw:wave_tolerance_c 0.01
[fclaw]    geoclaw:speed_tolerance_entries_c 6
[fclaw]    geoclaw:speed_tolerance_c 1e12 1e12 1e12 1e12 1e12 1e12
[fclaw]    geoclaw:mbathy         1
[fclaw]    geoclaw:ghost_patch_pack_aux true
[fclaw]    user:example           1
[fclaw] Arguments: none
[libsc 0] Abort: Assertion '(ptrdiff_t) ptr % signalign == 0'
[libsc 0] Abort: ../../forestclaw/sc/src/sc.c:437
[libsc 0] Abort: Obtained 1 stack frames
[libsc 0] Stack 0: 0   teton                               0x000000010d94182c sc_abort_handler + 44
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

Only a problem in GeoClaw?  Or something else?

Bitbucket: https://bitbucket.org/donnaaboise/forestclaw/issue/110

No longer seems to be an issue!


Support Connection via Socket File

Package version
8.0.1
Describe the bug
I’m attempting to run an Adonis application on Uberspace. The Uberspace hosting offers Redis but connections must use a sock file. Here’s a link to their Redis guide: https://lab.uberspace.de/guide_redis/
The @adonisjs/redis package currently uses a Redis configuration file that requires an object when configuring a Redis connection. I couldn’t find a connection config key that allowed me to pass a socket path for the connection. The underlying ioredis package supports socket files for connections.
Am I missing something and are socket connections already possible with this package?
Reproduction repo
No response

I went through all options of an ioredis connection configuration object again and found the path key. That solves the issue 🙂
Here’s a simplified Redis config file for reference (config/redis.ts):
const redisConfig = defineConfig({
  connection: 'main',

  connections: {
    main: {
      path: env.get('REDIS_SOCKET_PATH'),
      host: env.get('REDIS_HOST'),
      port: env.get('REDIS_PORT'),
      password: env.get('REDIS_PASSWORD', ''),
      db: 0,
      keyPrefix: env.get('REDIS_KEY_PREFIX', ''),
      retryStrategy (times) {
        return times > 10 ? null : times * 50
      },
    },
  },
})

The configuration above allows us to use Redis via host/port during development and a Socket file in production. It seems that ioredis ignores host/port if the path option is present and not empty.

For reference, here’s the ioredis options list I mentioned
https://ioredis.readthedocs.io/en/latest/API/

Can you please create a PR in the docs to mention the usage of Socket path? I think it will be a common use-case for many

Good point. I'll submit a PR to the docs repo. Eta: by the end of the week 🙂


dbt ingestion: Incorrect work assignment of owners

Affected module
Ingestion Framework?
Describe the bug
The problem I'm facing affects 3 aspects of working with OpenMetadata:

Dbl Ingestion does not find existing users if in<EMAIL_ADDRESS>) there is '.';
Users assigned to dbl Ingestion are not displayed on the search page.
Users assigned to dbt Ingestion are not displayed in the drop-down filter by owner.

To Reproduce

A user with a similar name but replacing the '.' sign with '_' is identified without problems.

[2024-02-19T13:41:51.468+0000] {metadata.py:171} WARNING - Unable to ingest owner from DBT since no user or team was found with name daniil.kotomtsev
[2024-02-19T13:41:51.616+0000] {metadata.py:171} WARNING - Unable to ingest owner from DBT since no user or team was found with name daniil.kotomtsev
[2024-02-19T13:41:52.070+0000] {metadata.py:171} WARNING - Unable to ingest owner from DBT since no user or team was found with name daniil.kotomtsev

3.

Screenshots or steps to reproduce
Expected behavior

I expect users with '.' in<EMAIL_ADDRESS>) will be detected and linked as owners to the tables, if they are available in the system.
2-3. I expect that users linked to tables using dbt Ingestion will be displayed in the drop-down filter by owner, on the search page in the table(Object) block and on the Profile > "My Data" page.

Version:

OS: Ubuntu 22.04
Python version: -
OpenMetadata version: 1.3.0
OpenMetadata Ingestion package version: -

Additional context

I use Docker installation with  default docker-compose file.
For load data from dbt artifacts i use UI solution with load files from S3 storage.

@OnkarVO7 how are we looking for users?

@harshach we get the user entity ref by using the get_by_name API here and building the user fqn here
Let me try to reproduce this and see if calling ES here fixes it

@OnkarVO7 if the name contains "." or any entity contains "." in its name we should escape it for FQN.
@pmbrull is hte ingestion framework does the escape during the construction of FQN?

@harshach we use antlr as well to build and quote it


Add "Feels-Like" Temperature

8<------------------------ FEATURE REQUEST ------------------------------------
Missing feature
There is currently no "feels-like" temperature displayed.
Justification
Personally, that is a number I like to see. Could be nice for others as well.
Workarounds
Here's the code to calculate it. I put it in the void drawCurrentWeather() function and replaced the city name text with what the temperature feels like. Converted the code from here: https://gist.github.com/jfcarr/e68593c92c878257550d
  //Calculate "Feels-Like" Temp
  float vTemperature = currentWeather.temp;
  float vWindSpeed = currentWeather.windSpeed;
  float vRelativeHumidity = currentWeather.humidity;
  float vFeelsLike = 0.0;
  //Try Wind Chill First
  if (vTemperature <= 50 && vWindSpeed >= 3) {
    vFeelsLike = 35.74 + (0.6215*vTemperature) - 35.75*(pow(vWindSpeed,0.16)) + ((0.4275*vTemperature)*(pow(vWindSpeed,0.16)));
  } else {
    vFeelsLike = vTemperature;
  }
  //Use Heat Index if Needed
  if (vFeelsLike == vTemperature && vTemperature >= 80) {
    vFeelsLike = 0.5 * (vTemperature + 61.0 + ((vTemperature-68.0)*1.2) + (vRelativeHumidity*0.094));
    if (vRelativeHumidity < 13 && vTemperature >= 80 && vTemperature <= 112) {
      vFeelsLike = vFeelsLike - ((13-vRelativeHumidity)/4)*(sqrt((17-(fabs(vTemperature-95.)))/17));
      if (vRelativeHumidity > 85 && vTemperature >= 80 && vTemperature <= 87) {
        vFeelsLike = vFeelsLike + ((vRelativeHumidity-85)/10) * ((87-vTemperature)/5);

      }

    }
  }

8<------------------------ END FEATURE REQUEST --------------------------------

That would be a useful piece of information, I agree. However, it would be a lot more useful if it were built into the https://github.com/ThingPulse/esp8266-weather-station library that this project also relies upon. Would you mind raising a PR there?

@marcelstoer Thanks for getting back to me. That makes sense. I raised a PR for the main library. I don't know C++ so I don't know if I put it in the best place, but it works for me.

The problem with your code example is that it only works for °F and therefore it needs modifying to cope with either °C or °F can the units variable

Ah, yes, you are right - thanks @G6EJD I'll modify the code.

I modified the code and made a new pull request https://github.com/ThingPulse/esp8266-weather-station/pull/173
@G6EJD  @marcelstoer


translation lost

Type: Bug
VS Code version: Code 1.73.1<PHONE_NUMBER>f055b99068d3688932416f2346dd3b, 2022-11-09T02:22:48.959Z)
OS version: Darwin arm64 22.1.0
Modes:
Sandboxed: No

System Info

Item
Value

CPUs
Apple M1 Max (10 x 24)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
2, 4, 4

Memory (System)
32.00GB (0.36GB free)

Process Argv
--crash-reporter-id 3314e47b-b93c-4574-a628-7c01ae6f4bea

Screen Reader
no

VM
0%

Extensions (37)

Extension
Author (truncated)
Version

browse-lite
ant
0.3.0

vite
ant
0.2.5

vscode-tailwindcss
bra
0.9.1

npm-intellisense
chr
1.4.2

path-intellisense
chr
2.8.1

vscode-svgviewer
css
2.0.0

dart-code
Dar
3.52.1

flutter
Dar
3.52.0

vscode-markdownlint
Dav
0.48.1

vscode-eslint
dba
2.2.6

xml
Dot
2.5.1

es7-react-js-snippets
dsz
4.4.3

gitlens
eam
13.0.4

vscode-html-css
ecm
1.13.1

EditorConfig
Edi
0.16.4

prettier-vscode
esb
9.9.0

git-project-manager
fel
1.8.2

auto-close-tag
for
0.5.14

code-runner
for
0.11.8

copilot
Git
1.58.7236

go
gol
0.36.0

todo-tree
Gru
0.0.220

vscode-codeowners
jas
1.1.1

i18n-ally
lok
2.8.1

start-git-bash
McC
1.2.1

vscode-language-pack-zh-hans
MS-
1.73.11090913

vsliveshare
ms-
1.0.5762

vscode-react-native
msj
1.10.0

nutui-vscode-extension
nut
0.0.2

vetur
oct
0.36.1

vscode-stylelint
sty
1.2.3

vscode-mdx
uni
1.0.3

windicss-intellisense
voo
0.21.6

vscode-icons
vsc
12.0.1

volar
Vue
1.0.9

vscode-typescript-vue-plugin
Vue
1.0.9

JavaScriptSnippets
xab
1.8.0

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vstes627:30244334
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscorecescf:30445987
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
azure-dev_surveyone:30548225
pyindex848cf:30577861
nodejswelcome1cf:30587006
fc301958:30595537
282f8724:30602487
gswce1:30605430
iaj6b796:30604794
dbltrim-noruby:30604474

When I upgraded to 1.73.1 and I quit vscode completely and reopened it, still having this issue.

Maybe related:

There is no translation here, it is all the default English.
vscode.git.i18n.json

@yoyo837 can you tell me what version of the language pack you are using? You should be able to see that in the Extensions view.

@yoyo837 can you tell me what version of the language pack you are using? You should be able to see that in the Extensions view.

Yes, here it is:

Extension
version

vscode-language-pack-zh-hans
1.73.11090913

@yoyo837 are you able to install the version right before that one? If you find the language pack in the Extensions view, you should be able to select a different version.

Yes, I back to the previous version and it works fine now.

I am working to get that version without the translations unlisted from the marketplace so that the older version with the correct strings is used for 1.73.

@TylerLeonhardt Please reach out to MP team (@SaiKanth007) for getting the extension to be unlisted.

Just to enumerate workarounds... using VS Code Insiders is also a valid workaround

Looks like language version 1.73.11090913 has been withdrawn.

Yes the bad version has been deleted... however, if you have a client that has updated to that "bad version" you will not be downgraded to the "good version", unfortunately.
I would need to release a new 1.73 version which is doable but a lot of work and potentially error prone...... so instead what we will do is just wait for the next 1.74 release of VS Code which will grab a newer version of the language pack that will work with it.
You can verify that the new version of 1.74 will work by installing VS Code Insiders.

Verification for the team: install VS Code Insiders, Install a language pack, make sure Git extension is localized (for the most part)

Yes the bad version has been deleted... however, if you have a client that has updated to that "bad version" you will not be downgraded to the "good version", unfortunately.
I would need to release a new 1.73 version which is doable but a lot of work and potentially error prone...... so instead what we will do is just wait for the next 1.74 release of VS Code which will grab a newer version of the language pack that will work with it.
You can verify that the new version of 1.74 will work by installing VS Code Insiders.
I'll close this since there isn't anything left for us to do besides "business as usual" with releasing a new version projected to be at the end of the month/first week of Dec.

Great! Thank you!

oh,great

Yes, I back to the previous version and it works fine now.

Hello, how can I get the version history list?


Updated the data directory to download and check hashes for ds009 data.

@jarrodmillman
We have a lot of data, so when getting all the file hashes, I saved the dictionary to a JSON file in the data directory (pushed to this branch, so it's accessible to all) and then loaded it in data.py rather than copy-pasting all of the key-value pairs into the .py file itself. So this still meets your basic requirement that "make data" will get you all the data and "make validate" will validate the content, but I was a bit uncertain about whether introducing the external JSON file would still be legal. Also, was there anything to be done by us in the "tests" directory, other than deleting the two superfluous lines from test_data.py?
General notes for everybody else:
Don't actually try running "make data" on your computer because it will try to download all 6 GB of our data and you will cry. If you want to test it out, just make sure you've got the ds009 directory in the data directory. Then, with the data directory as your current directory, run "make validate" or "python data.py". I tried "make data" on my own computer and let it run long enough to see that it was downloading the correct .tgz file, so I'm fairly confident this should work (the tar command to decompress the file is literally the same one I used on the data originally).
I am currently only checking the hashes for the ds009 data, and not the 4 data files from class. If we end up deciding to use those for our tests, I can add to the Makefile to include downloading and checking those files too, no problem.
The only failure is coverage, which is apparently around 80%. What's not covered is the contents of if name == "main" in the data.py file.

Awesome, We should definitely follow up with Jarrod on the coverages (I haven't checked the code, so this isn't a vote to merge)

Update: Added tests. Coverage is like 90%.

@jarrodmillman @rossbar   It seems as we update the master (accept push requests) and then continue updating a open pull directory we get more and more files that says they are being changed. How do we deal with that? (This directory has 11 "changed" files, but only really 6 files that are being updated).
@janewliang It looks like it does the right thing, would you like me to grab this branch and run it on my computer?

@benjaminleroy That would be great! Yeah, try both validating the hashes on your data and also run the nosetests.
@jarrodmillman @rossbar Is there some kind of minimum coverage that we're supposed to attain? Because, for example, this branch has decreases coverage to 92.8%, but the uncovered stuff is pretty silly (stuff that gets called only when executing the file, and also a try statement that checks whether libraries for Python 2 or Python 3 should be imported).

@janewliang I was able to run the validation of the the hashes for the data and nosetests were successful. I'm afraid I don't understand it all, but it runs well I'm ok approving even with coverage decrease. Good job.

Cool, I can walk you through what I did during tomorrow's meeting.

Tests passes Travis CI and Coveralls

Changes Unknown when pulling 8a8c15c53c359e3e2006cbec56ef00df82dcf626 on janewliang:makedata into ** on berkeley-stat159:master**.


Added WhoHas contract ABI

The WHO Token is an extended ERC20 Token. It connects the social platform WhoHas to the ethereum blockchain.
Website will be public in a few hours.

hi can you post a public link to your website/platform

Sure!
whohas.io

Conflicts solved

Moved to MyEtherWallet/Ethereum-lists PR #495

Hey, @Lebski  I found three functions present in this ABI that are not present in the ABI available via etherscan.  They are 'circulatingSupply_', 'maxSupply_', 'balances'.

Hey, @Lebski I found three functions present in this ABI that are not present in the ABI available via etherscan. They are 'circulatingSupply_', 'maxSupply_', 'balances'.

Hey, @Lebski I found three functions present in this ABI that are not present in the ABI available via etherscan. They are 'circulatingSupply_', 'maxSupply_', 'balances'. Are they supposed to be present? Thanks

Hi!
@SteveMieskoski, thank you for checking.
We updated the contract and the ABIs got merged.
But it's should be fine.

moved #1673


ColSpec is broken for ZScore

MWE:
using TableTransforms

(a=rand(Int,3), b=rand(3)) |> ZScore(:b)

AssertionError: columns must hold continuous variables

assert_continuous(::NamedTuple{(:a, :b), Tuple{Vector{Int64}, Vector{Float64}}})@assertions.jl:8
applyfeat(::TableTransforms.ZScore{TableTransforms.NameSpec}, ::NamedTuple{(:a, :b), Tuple{Vector{Int64}, Vector{Float64}}}, ::Nothing)@transforms.jl:175
apply@transforms.jl:131[inlined]
Transform@interface.jl:84[inlined]
|>(::NamedTuple{(:a, :b), Tuple{Vector{Int64}, Vector{Float64}}}, ::TableTransforms.ZScore{TableTransforms.NameSpec})@operators.jl:911
top-level scope@[Local: 1](http://localhost:1234/edit?id=ad9e29a0-74c7-11ed-38ee-0ffbf894998f#)[inlined]

I think this is causing an error because the ZScore function only works on continuous variables and Int is a categorical variable.
Maybe the transform function can be use to apply the ZScore transformation only to the b column of the NamedTuple, and to remove the a column from the result.
using TableTransforms

(a=rand(Int,3), b=rand(3)) |> transform(:b, ZScore)

The root of the issue is known, the fallback method for all ColwiseTransform types is performing assertions in the input table instead of in the selection.
Do you want to give it a try and fix it @vickydeka ? Here is the relevant function to modify:
https://github.com/JuliaML/TableTransforms.jl/blob/976ce71bf7dad543d70d18120323126fad3395d3/src/transforms.jl#L172-L208

@juliohm I would definitely like to give it a try!

Fixed by #160


Update cats-effect, cats-effect-std to 3.2.6

Updates

org.typelevel:cats-effect
org.typelevel:cats-effect-std

from 3.1.1 to 3.2.6.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (3.1.1).
You might want to review and update them manually.
build.sbt

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel" } ]

labels: library-update, semver-minor, old-version-remains

Superseded by #346.


Update links in README to point at developer.nexmo.com

@adambutler Are these stable enough to link to now?

@adambutler Fixed, thanks 👍


Bug: Reconnect with Racechrono and BT SPP

I am using Racechrono with the presets for it (BL SPP)
The first connection is great and I see the data in Racechrono. After I finish my recording in Racechrono and click again on Start, Racechrono has a problem to establish the connection.
A restart of the module fixes the problem.

Thanks for the bug report
This is an annoying behavior, I have not encountered it as I have not used RaceChrono on the field with repeated sessions.
Would you have any log? Is it consistently happening?
One possibility is an issue with BT SPP event management due to incomplete closure of connection, this the callback which responds to open and close
void bt_callback(esp_spp_cb_event_t event, esp_spp_cb_param_t *param)
{
  if (event == ESP_SPP_SRV_OPEN_EVT)
  {
    log_d("BT-SPP Client Connected");
#ifdef BLEENABLED
    ble_deviceConnected = false;
#endif
    bt_deviceConnected = true;
  }
  if (event == ESP_SPP_CLOSE_EVT)
  {
    log_d("BT-SPP Client closed connection");
    bt_deviceConnected = false;
  }
}

I have not looked into different types of events or timeout mechanisms: if bt_deviceConnected is true, no new connection is established.

The bug is consistently happening.
My log is:
[I][GPSProvider.cpp:246] pollGSAGSVInfo(): Gs true
[I][BluetoothSerial.cpp:238] esp_spp_cb(): ESP_SPP_SRV_OPEN_EVT
Client Connected
[I][GPSProvider.cpp:259] pollGSAGSVInfo(): Gs false
[I][BluetoothSerial.cpp:250] esp_spp_cb(): ESP_SPP_CLOSE_EVT
Client disconnected
[I][GPSProvider.cpp:246] pollGSAGSVInfo(): Gs true
[I][GPSProvider.cpp:259] pollGSAGSVInfo(): Gs false
[I][GPSProvider.cpp:246] pollGSAGSVInfo(): Gs true
As you can see, when i start logging the bt connection is established and after the ending the connection is closed. After that the Event isn´t called again.
I tried to create a routine to restart the BT Service but this was without success.
My code for the restart is:
if (btSppStop())
{
btSppStart();
}

Thanks for confirming!
Let me reproduce locally.
Restarting BT might bring more troubles than anything, I find the BLE+BT stacks on esp32 quite unreliable when you start/stop outside of specific conditions. I would focus on understanding why the new connect event is not picked up by callback.
I assume btSppStart and btSppStop are routines you created correct? There are 2 functions bt_spp_stop and bt_spp_start in this code.

I know.
I tested it also with your code the be sure that this bug is not only on me side.

el mismo problema :(


[BUG] Error 401 on URLs (Forms and API end points)

Description
Tried accessing form submission URLs in the browser using GET and the following URLs return a 401 instead of disabling the request and redirecting back to the homepage/login page or a 404 page appropriately.
URLs with issues:
https://app.appsmith.com/api/v1/users
https://app.appsmith.com/api/v1/
https://app.appsmith.com/api/
Error: http://prntscr.com/uonvcd
Important Details

Version: Cloud
OS: Windows 10
Browser: Brave (Version 1.14.81 Chromium: 85.0.4183.102 (Official Build) (64-bit))

@mohanarpit can you verify this

According to the current configuration in SecurityConfig, this is the expected behaviour. I checked by running locally.

Closing issue because this is expected behaviour. The API server returns only JSON API responses. Hence returning HTTP responses with appropriate status codes is expected behaviour.


Fix CompositeAuth::authenticate()

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Fixed issues
#19517

👍


Make py_func accept SparseTensor

Describe the problem clearly
According to the doc, py_func accepts inp as a list of tensors (or convertible to tensor). However SparseTensor is not one of them. Could we support SparseTensor as well? Semantically there is no reason to treat SparseTensor differently.
Source Code / Logs
my_sparse_tensor = tf.SparseTensor(...)
tf.py_func(my_py_func, [my_sparse_tensor, ...], [tf.float32])

What I got:
TypeError: Tensors in list passed to 'input' of 'PyFunc' Op have types [<NOT CONVERTIBLE TO TENSOR>, ...] that are invalid.

@alextp : Any thoughts on this (since you recently touched the py_func codepaths)

One could repurpose the code which makes sparse tensors feedable / fetchable to do that.
Right now to unblock you simply pass st.value, st.indices, and st.shape to the py_func where you can make a SparseTensorValue out of them.

I'll work on this since I have wrote some code related to py_func.

@jingjingwang , what does your my_py_func look like? I think the function passed to tf.py_func is purely "numpy", so there must be a sparse object type in numpy that we can convert tf.SparseTensor to.

A SparseTensor is just three tensorflow Tensors (indices, values, and
shape). Ops which take or produce SparseTensor are just python wrappers
around tf ops which pass/return these 3 tensors.
On Thu, May 18, 2017 at 11:28 PM, Ziming Dong<EMAIL_ADDRESS>wrote:

@jingjingwang https://github.com/jingjingwang , what does your
my_py_func look like? I think the function passed to tf.py_func is purely
"numpy", so there must be a sparse object type in numpy that we can convert
tf.SparseTensor to.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/9180#issuecomment-302620030,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAATxRCQ0JkvPC57xOwPrdVx2U5Fo_Eqks5r7TaKgaJpZM4M8Xtl
.

--

Alex

@alextp , I think @jingjingwang want to pass a SparseTensor directly to a custom python function, not unpack it to three Tensors first.

Directly converting to scipy.sparse.coo_matrix would be cool, actually :)

Any progress, @suiyuan2009 ?

Here's a workaround so pyfunc accepts nested structures, feel free to make an MR so it ends up in the codebase:
def py_func_v2(func, inputs, Tout):
    """Wrapper around py_func to accept nested structured like dicts and tuples."""
    inputs = copy.copy(inputs)
    def _wrapper(*flat_inputs):
        inputs_d = tf.contrib.framework.nest.pack_sequence_as(inputs, flat_inputs)
        return func(inputs_d)
    flat_inputs = tf.contrib.framework.nest.flatten(inputs)
    return tf.py_func(func=_wrapper, inp=flat_inputs, Tout=Tout)

Hi @jingjingwang ,
tf.py_func is deprecated in TF V2. Instead, there are two options available in V2.

tf.py_function takes a python function which manipulates tf eager tensors instead of numpy arrays. It's easy to convert a tf eager tensor to an ndarray (just call tensor.numpy()) but having access to eager tensors means tf.py_functions can use accelerators such as GPUs as well as being differentiable using a gradient tape.

tf.numpy_function maintains the semantics of the deprecated tf.py_func (it is not differentiable, and manipulates numpy arrays). It drops the stateful argument making all functions stateful.


It works perfectly on runtime node but not in asar

I developed an react+Electron application with node-adodb on Windows. It works perfectly in development mode, but when packaging everything (create the .exe and other electron files), the connection.query got "Uncaught Fatal Exception"
Using node-adodb version 4.2.2
React 16.4.0
electron 2.0.5
electron-builder: 20.15.1
main.js code
ipcMain.on('read-cid', (event, obj) => {
  const connection = ADODB.open('Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\Users\\DB.mdb;')
  async function query() {
      try {
          const users = await connection.query(`SELECT top 1 id_birth, id_addr FROM IDBaseInfo  `)
          event.returnValue = users
      } catch (error) {
           event.returnValue = error
      }
  }
  query()
})

不打包成asar就运行良好，打包成asar，就会出现Uncaught Fatal Exception,请问有什么方法可以在打包成asar的情况下使用模块吗
谢谢！

Yeah, so I had this issue a few months ago and figure out the problem. Basically this. Packaged Electron projects don't play well with child_process.spawn. And also cwd() doesn't work properly in asar files, that may have also contributed. I forget.
But the workaround is the following:

In your packager settings, add a glob that matches the node-adodb package to the asar.unpack setting.
Go to line 13 of proxy.js and change it so that it matches the unpacked directory.

The way I did this was I changed
require.resolve('./adodb')
to
require.resolve('./adodb').replace('app.asar', 'app.asar.unpacked')
I also published my own npm package to make things easier for myself, you can try it out if you want, and see if it works for you. It doesn't use the latest version though so just be aware of that. Just don't forget to add the unpacked setting or it won't work.
npm i -S node-adodb-electronfork
Hope it works out for you. Cheers.

@lega0208 did I need to change code? something like
import * as ADODB from 'node-adodb-electronfork'
change to this?
import * as ADODB from '../../node-adodb-electronfork'
Thanks

@HaveF No, the path to the package should be the same. It looks like the replace function isn't working properly though, which is strange. Because it should change the "app.asar" folder name to "app.asar.unpacked". I wouldn't include it in the bundle though because if you unpack it, then there's extra overhead because it has to be read from the filesystem, so you want to only unpack the code that needs to be unpacked.
In any case, I'm on vacation on another continent at the moment so I can't help much, but iirc, Electron 3.0 includes features to address this exact problem, so I would look into that. (Plus it has a much newer version of node + chromium/V8, so less polyfills needed and better performance)
I also haven't looked into newer versions of the actual node-adodb package, nuintun may have fixed this already.
So good luck, and if you still need help in a week or so I can probably look into it.

@lega0208 Thanks for your kindness and detailed explanation. Frankly speaking, I do not understand the unpack part. I configure the electron-builder to unpack the node-adodb or your version node-adodb-electronfork, and make sure after installed the production of my app, the dir is over there. I thought bundle.js is only about the src code which exclude node_modules, maybe I am wrong. I use this webpack config, it seems not exclude node_modules dir.
I use the newest version of node-adodb(4.2.2) but failed, then find your solution but still without success.
Thank your info. I will try later.
Have a good vacation 😄

Get everything work by a trick.
I use webpack.

In the main process add this before opening the BrowserWindow

const nodeEnv = process.env.NODE_ENV;

if(nodeEnv === 'development'){
  global.ADODB = require('node-adodb')
} else {
  let libPath = path.join(app.getAppPath(), 'node_modules/node-adodb').replace('app.asar', 'app.asar.unpacked')
  global.ADODB = require(libPath)
}

In the renderer process pick the value this way:

const ADODB = require('electron').remote.getGlobal('ADODB')

Btw, new Electron version not works in the situation, I tried on 4.0.0-nightly.20181010
Cheers
REF: node.js - How can I bundle a precompiled binary with electron - Stack Overflow

Yeah, so I had this issue a few months ago and figure out the problem. Basically this. Packaged Electron projects don't play well with child_process.spawn. And also cwd() doesn't work properly in asar files, that may have also contributed. I forget.
But the workaround is the following:

In your packager settings, add a glob that matches the node-adodb package to the asar.unpack setting.
Go to line 13 of proxy.js and change it so that it matches the unpacked directory.

The way I did this was I changed require.resolve('./adodb') to require.resolve('./adodb').replace('app.asar', 'app.asar.unpacked')
I also published my own npm package to make things easier for myself, you can try it out if you want, and see if it works for you. It doesn't use the latest version though so just be aware of that. Just don't forget to add the unpacked setting or it won't work.
npm i -S node-adodb-electronfork
Hope it works out for you. Cheers.

How to use this package? Any guidance?


GLUE/SuperGLUE Tasks Improvements

Could you, please, add PR description and provide metrics (in docs or here) for new and updated models?

Could you, please, add PR description and provide metrics (in docs or here) for new and updated models?

The scores were updated in the docs

Triage for v0.16
Sberbank

[x] - Add RTE and MNLI configs
[x] - Add helper functions and config for the Choice of Plausible Alternatives task (COPA)
[ ] - BoolQ | staged for v0.17

Refactoring Library & Specific Existing Models

[x] - Revise affected NER and SQuAD pipelines and configs to accommodate them to the transformers update
[x] - Update model scores in the docs
[x] - Add new pre-trained model download paths where applicable
[x] - Bump transformers version from 2.8.0 to 4.6.0
[x] - Compatibility fixes to various tasks related to the transformers version update

New Features

[x] - Add a new task type and a corresponding pipeline: Multiple Choice

New Experimental Features

[ ] - Experimental multi-GPU support | staged for v0.17


Two Problems

Hi,
first off i really love the mod so far... But i got some two bugs (?).. whenever i have reins equipped i can't ride my pet that is with me.. this is also a brand new character i created for 1.4.0... also i just came across a door in an ocean biome that wouldn't open with the harp item...  The harp wasn't interacting with the correct door? 🤔 I will take a look at that 👍


Add mention of PyPy performance?

I came across this discussion yesterday and decided to try PyPy with TDL. I was blown away by the performance boost.  My application was easily 10x faster, which made some of my silly experiments (React-like diffing of widgets -> draw to screen) actually feasible.
Do you think it is worth adding a comment about this in the README?  Would be a great pointer for new users to immediately consider PyPy if they're having performance concerns.

Unfortunately, the speed up from PyPy is mostly just offsetting some early bad design decisions in TDL.  You'd get just as good performance from using numpy, which will be integrated in later versions of libtcod-cffi, and eventually TDL itself.
I don't really recommend PyPy to new users because it's harder to distribute the finished builds of a PyPy game.  I have it as an option for people already experienced with Python.
My current focus is on making the CPython versions run faster.

Makes sense, thanks for sharing.


Android Timestamp Retrieval Issue

I am using:
"firebase": "3.5.2",
"react-native-firestack": "^2.3.2",
Here is my code:
firebaseApp.database.ref('posts/').once('value')
.then((snapshot) => {
console.log(snapshot.val());
})
iOS returns:
{ puid: '-KVUuw6uVjl-C1YlhTJh',
timestamp:<PHONE_NUMBER>851,
title: 'the first one is not a big ',
name: 'Herve',
text: 'Bill Clinton and the rest. the first to review this information ' },
'-KVUv2CJTJjJKEtvJTtF':
{ puid: '-KVUv2CJTJjJKEtvJTtF',
timestamp:<PHONE_NUMBER>872,
title: 'the 8th of my own business and the ',
name: 'Herve',
text: 'the new one of those days ago, ' },
'-KVUuafD4WokUBgQsz53':
{ puid: '-KVUuafD4WokUBgQsz53',
timestamp:<PHONE_NUMBER>992,
title: 'hey man. he is the most ',
name: 'Herve',
text: 'hey man. he is the best way for me to ' } }
But in Android:
{ '-KVUv2CJTJjJKEtvJTtF':
{ title: 'the 8th of my own business and the ',
timestamp: 535575048,
text: 'the new one of those days ago, ',
puid: '-KVUv2CJTJjJKEtvJTtF',
name: 'Herve' },
'-KVUuw6uVjl-C1YlhTJh':
{ title: 'the first one is not a big ',
timestamp: 535546027,
text: 'Bill Clinton and the rest. the first to review this information ',
puid: '-KVUuw6uVjl-C1YlhTJh',
name: 'Herve' },
'-KVUuafD4WokUBgQsz53':
{ title: 'hey man. he is the most ',
timestamp: 535458168,
text: 'hey man. he is the best way for me to ',
puid: '-KVUuafD4WokUBgQsz53',
name: 'Herve' } }
As you can see, the timestamp returns with android is wrong, any idea why?

Perhaps a clue for you:
Sounds like the symptoms of timestamps (which are longs in java) getting cast to ints, but that should have been fixed in https://github.com/fullstackreact/react-native-firestack/pull/79

if I rename timestamp to another name like time, I have the same issue.

Hm... good question... are you using npm or master @jsappme ?

not sure what do you mean
I am using yarn with "react-native-firestack": "^2.3.2",

Ah, can you pull the latest master branch?
git clone https://github.com/fullstackreact/react-native-firestack node_modules/react-native-firestack
Or follow the instructions on the README. I haven't updated the npm module yet with #79

I am getting this error:
/Users/herve76/Desktop/react-native-firebase-starter/android/app/src/main/java/me/jsapp/MainApplication.java:30: error: constructor FirestackPackage in class FirestackPackage cannot be applied to given types;
new FirestackPackage(getApplicationContext())
^
required: no arguments
found: Context
reason: actual and formal argument lists differ in length
1 error
:app:compileDebugJavaWithJavac FAILED


Mistakes in metallity calculations

We identified two issues when computing the metallicities of substructures.

First one is a missing normalisation of the aperture-based total-gas Z.
This patch should fix it:
diff --git a/src/substructureproperties.cxx b/src/substructureproperties.cxx
index f5da189..3fd8a0f 100644
--- a/src/substructureproperties.cxx
+++ b/src/substructureproperties.cxx
@@ -5580,7 +5580,7 @@ void CalculateApertureQuantities(Options &opt, Int_t &ning, Particle *Part, Prop
             if (EncMassGas>0) pdata.aperture_vrdisp_gas[iaptindex]=EncVRDispGas/EncMassGas;
 #ifdef STARON
             pdata.aperture_SFR_gas[iaptindex]=EncSFR;
-            pdata.aperture_Z_gas[iaptindex]=EncZmetGas;
+            pdata.aperture_Z_gas[iaptindex]=EncZmetGas/EncMassGas;
             pdata.aperture_npart_gas_sf[iaptindex]=NinsideGasSF;
             pdata.aperture_npart_gas_nsf[iaptindex]=NinsideGasNSF;
             pdata.aperture_mass_gas_sf[iaptindex]=EncMassGasSF;

Basically, the value returned is \sum m_i Z_i with the division by \sum m_i at the end missing.
This is done correctly for the SFing and NSFing sub-components, however.
I haven't checked whether there are other similar problems but all the other outputs look correct when looking at our pipeline plots.

Second one is related to OpenMP and non-aperture quantities.
It seems that at least Zmet_gas and Mass_gas are not calculated correctly when running with OpenMP.
This plot shows the issue:

We'd expect Mass_gas to be Mass_gas_sf + Mass_gas_nsf.
This problem then translates into incorrect Z for the total.
If I switch off openMP then the problem is not there any more.
I think this means there is a problem with the private variables used for this calculation.
There might also be additional problems with other quantities; we haven't checked everything.

The projected aperture metallicites also seem to be weird. But I couldn't come up with a sensible patch; I don't understand well enough how this whole code section works.


hyperf 2.1 is abnormal when running in alpine docker

Describe the bug
Hyperf 2.1 is abnormal when running in alpine docker, it runs normally on the physical machine
Swoole version  4.6.6
Hyperf 2.1
WARNING check_worker_exit_status: worker#1[pid=19] abnormal exit, status=0, signal=11
A bug occurred in Swoole-v4.6.6, please report it.
The Swoole developers probably don't know about it,
and unless you report it, chances are it won't be fixed.
You can read How to report a bug doc before submitting any bug reports:
>> https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md
Please do not send bug reports in the mailing list or personal letters.
The issue page is also suitable to submit feature requests.

PHP (please complete the following information):

Version 7.3.27

SkyWalking (please complete the following information):

OS: docker  alpine:3.12
Version 4.1.2

Additional context

gdb

warning: Can't read pathname for load map: No error information.
--Type <RET> for more, q to quit, c to continue without paging--
Core was generated by `goa-checkin.async-queue.0'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000000000 in ?? ()
[Current thread is 1 (LWP 19)]
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f56e28c7d07 in ?? () from /usr/lib/libgrpc++.so.1
#2  0x00007f56e28c7bf6 in upb_array_new () from /usr/lib/libgrpc++.so.1
#3  0x00007f56e28c68ac in ?? () from /usr/lib/libgrpc++.so.1
#4  0x00007f56e28c6d1f in ?? () from /usr/lib/libgrpc++.so.1
#5  0x00007f56e28c701c in ?? () from /usr/lib/libgrpc++.so.1
#6  0x00007f56e28c71e5 in upb_decode () from /usr/lib/libgrpc++.so.1
#7  0x00007f56e2182d57 in parse_and_add_descriptor () from /usr/lib/php7/modules/protobuf.so
#8  0x00007f56e2183065 in internal_add_generated_file () from /usr/lib/php7/modules/protobuf.so
#9  0x00007f56e21830e4 in zim_InternalDescriptorPool_internalAddGeneratedFile () from /usr/lib/php7/modules/protobuf.so
#10 0x00007f56e2bafafd in sky_execute_internal (execute_data=0x7f56d8138fc0, return_value=0x7f56d5009278) at /var/local/git/skywalking/src/sky_execute.cc:122
#11 0x000055997dd76296 in execute_ex ()
#12 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138ee0) at /var/local/git/skywalking/src/sky_execute.cc:75
#13 0x000055997dd761ae in execute_ex ()
#14 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138e60) at /var/local/git/skywalking/src/sky_execute.cc:75
#15 0x000055997dd761ae in execute_ex ()
#16 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138b90) at /var/local/git/skywalking/src/sky_execute.cc:75
#17 0x000055997dd764f3 in execute_ex ()
#18 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138a10) at /var/local/git/skywalking/src/sky_execute.cc:75
--Type <RET> for more, q to quit, c to continue without paging--
#19 0x000055997dd761ae in execute_ex ()
#20 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d81387a0) at /var/local/git/skywalking/src/sky_execute.cc:75
#21 0x000055997dd764f3 in execute_ex ()
#22 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d81385a0) at /var/local/git/skywalking/src/sky_execute.cc:75
#23 0x000055997dd764f3 in execute_ex ()
#24 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138470) at /var/local/git/skywalking/src/sky_execute.cc:75
#25 0x000055997dd764f3 in execute_ex ()
#26 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d8138270) at /var/local/git/skywalking/src/sky_execute.cc:75
#27 0x000055997dd761ae in execute_ex ()
#28 0x00007f56e2baebf7 in sky_execute_ex (execute_data=0x7f56d81380c0) at /var/local/git/skywalking/src/sky_execute.cc:75
#29 0x00007f56e2edea95 in swoole::PHPCoroutine::main_func (arg=0x7f56d540b1a0) at /tmp/swoole/ext-src/swoole_coroutine.cc:733
#30 0x00007f56e2f650ed in std::function<void (void*)>::operator()(void*) const (__args#0=<optimized out>, this=0x55997fc14000) at /usr/include/c++/9.3.0/bits/std_function.h:683
#31 swoole::coroutine::Context::context_func (arg=0x55997fc14000) at /tmp/swoole/src/coroutine/context.cc:141
#32 0x00007f56e2fcd081 in make_fcontext () at /tmp/swoole/thirdparty/boost/asm/make_x86_64_sysv_elf_gas.S:64
#33 0x0000000000000000 in ?? ()
(gdb)
(gdb)
(gdb) f 1
#1  0x00007f56e28c7d07 in ?? () from /usr/lib/libgrpc++.so.1
(gdb) f 0
#0  0x0000000000000000 in ?? ()

Need to provide swoole configuration parameters and skywalking-php configuration parameters

swoole

Swoole => enabled
Author => Swoole Team<EMAIL_ADDRESS>Version => 4.6.6
Built => Apr 23 2021 08:04:41
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 1.1.1k  25 Mar 2021
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
pcre => enabled
zlib => 1.2.11
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608

skywalking

Directive => Local Value => Master Value
skywalking.enable => On => On
skywalking.version => 8 => 8
skywalking.app_code => goa-checkin => goa-checkin
skywalking.authentication => no value => no value
skywalking.grpc => skywalking-oap.skywalking:11800 => skywalking-oap.skywalking:11800
skywalking.grpc_tls_enable => Off => Off
skywalking.grpc_tls_pem_root_certs => no value => no value
skywalking.grpc_tls_pem_private_key => no value => no value
skywalking.grpc_tls_pem_cert_chain => no value => no value
skywalking.log_enable => Off => Off
skywalking.log_path => /tmp/skywalking-php.log => /tmp/skywalking-php.log
skywalking.error_handler_enable => Off => Off
skywalking.mq_max_message_length => 20480 => 20480

core.1621420066.php.19.zip

I almost used the same configuration to test the hyporef skeleton program, and no error occurred

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://hyperf.wiki
 * @contact<EMAIL_ADDRESS> * @license  https://github.com/hyperf/hyperf/blob/master/LICENSE
 */
namespace App\Controller;

class IndexController extends AbstractController
{
    public function index()
    {
        $user = $this->request->input('user', 'Hyperf');
        $method = $this->request->getMethod();

        return [
            'method' => $method,
            'message' => "Hello {$user}. ",
        ];
    }
}

Can you provide me with your controller code for testing?

Let me wrap up a minimal repetition use case

Not every time, using AB or WRK can be quickly repeated
wrk -t2 -c10 -d300s --latency  http://<IP_ADDRESS>:9501

<?php

declare(strict_types=1);
/**
 * This file is part of Hyperf.
 *
 * @link     https://www.hyperf.io
 * @document https://hyperf.wiki
 * @contact<EMAIL_ADDRESS> * @license  https://github.com/hyperf/hyperf/blob/master/LICENSE
 */
namespace App\Controller;

use Hyperf\Redis\Redis;
use Hyperf\Utils\ApplicationContext;

class IndexController extends AbstractController
{
    public function index()
    {
        $user = $this->request->input('user', 'Hyperf');
        $method = $this->request->getMethod();
        $redis = ApplicationContext::getContainer()->get(Redis::class);
        $redis->set('test1', 'test');
        return [
            'method' => $method,
            'message' => "Hello {$user}.",
        ];
    }
}

So, is it easier to make mistakes with redis?

The same is true with MySQL


Paste event on textarea

Paste event on textarea fires tbwchange event before paste content is on element.

Thanks :)


Fix an inconsistency introduced in v2.14

Inconsistency introduced by: https://github.com/openservicebrokerapi/servicebroker/pull/541
Fixes #563
Fixes #508
Signed-off-by: Doug Davis<EMAIL_ADDRESS>

@leonwanghui thanks, I updated the first comment

@mattmcneeney @pmorie @n3wscott @zrob review?

@duglin I believe the text If the response contains "state": "failed" then the Platform MUST send a deprovision request to the Service Broker to prevent an orphan being created on the Service Broker. However, while the Platform will attempt to send a deprovision request, Service Brokers MAY automatically delete any resources associated with the failed bind request on their own.  introduced by #354 into the "Polling Last Operation for Service Instances" section still remains:
https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#body-1
I understand this text should be removed as this will negatively impacts service instance update failures as reported by @leonwanghui in #508 when broker last operation for service instance endpoint responds with a 200 status with a failed body. I'd then suggest to reopen #508

@gberche-orange nice catch!! I opened https://github.com/openservicebrokerapi/servicebroker/pull/591 to try to address it


Error generating kallax.go: expected operand, got '%' (and 6 more errors)

The complete message is:
kallax: No file generated due to an occurred error:
kallax.go:881:1: expected operand, found '%' (and 6 more errors)

The lines that throws the error are:
879 Timestamps:&schemaRepositoryReferencesTimestamps{
880 JSONSchemaKey: kallax.NewJSONSchemaKey(kallax.JSONAny),
881 %!(EXTRA string="_references", "Timestamps")CreatedAt:kallax.NewJSONSchemaKey(kallax.JSONAny, "_references", "Timestamps", "CreatedAt"),
882 UpdatedAt:kallax.NewJSONSchemaKey(kallax.JSONAny, "_references", "Timestamps", "UpdatedAt"),
883 },

Main model:
type Repository struct {
	kallax.Model `table:"repositories"`
	kallax.Timestamps `kallax:",inline"`
	Endpoints []string
	Status FetchStatus
	FetchedAt time.Time
	FetchErrorAt time.Time
	LastCommitAt time.Time
	References []*Reference
}

Reference model:
type Reference struct {
	kallax.Timestamps
	Name string
	Hash SHA1
	Init SHA1
	Roots []SHA1
}

The bug apparently is in this line: https://github.com/src-d/go-kallax/blob/4a448f451c19cff86ca78eeea7abcd073bd15509/generator/template.go#L265
You are trying to print two elements and the string is only accepting one.

Fixed in #73


httpclient Response message

Httpclient post method returning an error message regarding the content of the response message, below screenshots with error message and code, please help, Thank you.

I think you need to get the Headers of your HttpContent and set the Header for Content-Type and encoding there.
varHttpContent.GetHeaders(headers) //headers: HttpHeaders
headers.Add('Content-Type', 'text/xml'); //or application/xml

I am closing this issue because it appears that it has been resolved. Please open a new issue if you believe this has not been resolved and reference the current issue.


Rename lib

So, tempted to go with Zubr for the name of the project, as mentioned.
Reasons are (in no particular order):

current contributors are all in Poland, żubr means buffalo and Żubr is a beer brand and who doesn't like beer
nice and short
namespace is available
looks good: \Zubr\str_pos()

Proceed..?

@nkkollaw  IMO that is not a good idea ;) We all use here English, both in code code and GitHub. Why don't we stick with it?

@nkkollaw I have an idea. We are fighting with sadness so maybe - PHP + Happines = Phpness or PHPappines   :D

@nkkollaw IMO that is not a good idea ;) We all use here English, both in code code and GitHub. Why don't we stick with it?

Well, a name is a name. If the library is useful, people will find it and use it. Zubr has a meaning because it's related to Poland, for English-speakers it's a made-up word like Google, Yahoo, Etc.
I think it's good enough for now. Options so far are (see #5 for details):
\Zubr\str_pos('Hello', 'What');
\Phew\str_pos('Hello', 'What'); // namespace taken by other lib (although not in the same space)
\Happy\str_pos('Hello', 'What');
\Pill\str_pos('Hello', 'What');
\Phpness \str_pos('Hello', 'What'); // too long to type IMHO?
PHPappines\str_pos('Hello', 'What'); // too long to type IMHO?

I still think it's either Zubr, or Phew even if the namespace is taken. :-)

Ok let's go with Zubr :)

Phew is cool, too IMHO. Not sure if the fact that the namespace is taken will ever be a problem, but I doubt it.

Closing this as we're going with Zubr.


[Feature] Support Google Cloud Storage IAM

What type of PR is this：

[ ] BugFix
[x] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Problem Summary(Required) ：

Now, we support Google Cloud Storage. But we do not support access key and secret key in gs:// anymore, because this method is not allowed in GCS SDK, AKSK existed reason is to be compatible with s3 SDK.

Parameter Name
Default value
Description

gcp.gcs.use_compute_engine_service_account
false
Like AWS instance profile

gcp.gcs.service_account_email
""
One of the values in json key file

gcp.gcs.service_account_private_key_id
""
One of the values in json key file

gcp.gcs.service_account_private_key
""
One of the values in json key file

gcp.gcs.impersonation_service_account
""
The service account name which you want to assume(Like AWS assume role)

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[x] This pr will affect users' behaviors
[x] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[ ] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

@Mergify backport branch-3.0


does not appear

I get the map visible and everything but not the photolayer and any of the pics, Can anyone help me see what I'm doing wrong?
`    `

This is how my JSON looks

No Title??? ===> Array
(
[photos] => Array
(
[0] => Array
(
[title] => 2019-07-27 21:12:23
[url] => https://www.dfdsfdsfd.dfd/photos/userUpload/
[thumbnail] => thumb_20190727_211223.jpg
[picture] => 20190727_211223.jpg
[coordinates] => 39.764262277778, -84.189678194444
[lat] => 39.764262277778
[lon] => -84.189678194444
)


iOS asks for push notification confirmation on app start

The push notification confirmation dialog shows when the app starts. Not when calling init/registration of the push notification.
I would love to control when the permission dialog is shown. Is there any way to do that?
Is the dialog being shown because of local notifications and remote notifications permissions are bundled? See this link:
http://www.tagwith.com/question_791213_ios-8-cordova-app-requests-push-notification-permissions-immediately-on-first/
This is an issue in the old PushPlugin:
https://github.com/phonegap-build/PushPlugin/issues/378

@benmcmaster this was reported by another user awhile ago but I never could reproduce it and went away for him. Can you give me some more detail on the reproduction scenario? If you are not calling PushNotifcation.init() on "deviceready" you shouldn't see the request for push permission dialog.

Hey @macdonst ! Yeah, I removed the call to PushNotifcation.init() completely from my code, and I am still getting the permission dialog on app startup. It's like something else is running when the plugin code is included...
I am also using Phonegap Build.

@macdonst is there anything else I can give you to help recreate the issue?

@benmcmaster Nah, I just did a factory reset on one of my devices and I'll be trying out exactly this issue on a fresh device tonight.

@benmcmaster I can't reproduce this issue. I have a project setup so that Push.init() is not called until I click on a button. Take a look at this video:
https://dl.dropboxusercontent.com/u/887989/pushplugin/PushInit.mov
I star the app, no push permission dialog. Then you can see the flash on the button where I tap it. Immediately the push permission dialog shows up then the app receives a registration ID.
If there is anyway you can give me a reproduction scenario for this one I'd appreciate it.

@macdonst Hmm. Did you build that with Phonegap Build?
Is that code in a github repo? I could grab it, and build it on my end...

@benmcmaster No, I didn't build it with PGB.
Here is a link to the index.js I used. It is the same as my sample in the repo but it changes where Push.init() is called.
https://dl.dropboxusercontent.com/u/887989/pushplugin/index.js
the only other thing you'll need to do is create an element with and id of ack to be able to trigger the init method.

@macdonst I just took the example included in this repo, and your new index.js. I built your code with PGB. I am still getting the permission dialog on app start. I even commented out all of the registration code in index.js just in case. I still get the permission dialog on app start. Somehow the constructor of the PushNotification class is getting called on app startup without calling init().
Using phonegap version cli-5.2.0

@macdonst There is a line of code at the top the plugin push.js code inside of the .ipa that PGB compiles:
cordova.define("phonegap-plugin-push.PushNotification", function(require, exports, module) { /* global cordova:false */
I have never built a plugin before, and I have limited iOS experience. I wonder if that code is somehow calling the constructor...

@macdonst One more update: Looks like it's not a JS problem. I pulled the init code out of the constructor in push.js and created an PushNotification.prototype.init function and added some console logs just to make sure. I pushed my version up to my npm account and pointed PGB to grab the plugin from there.
I am still getting the permission confirmation dialog without any of the init JS code being called at all. It must be something on the iOS side of things... What could PGB be doing different?

We still have the same issue (as described in https://github.com/phonegap/phonegap-plugin-push/issues/49)...

@benmcmaster No, don't worry about that module line. What version of iOS are you using?
@martinfilliau are you using PGB? What version of iOS?

@macdonst I have noticed this bug since iOS 8 came out. Currently I am using iOS 9.

@benmcmaster @martinfilliau can anyone put together a small project that demonstrates it with a repo scenario? I can't seem to reproduce the problem. I believe it is a problem I just can't figure out what variable to tweak in order to reproduce it. Maybe it is another plugin that you guys are using that confuses things. If I can get some more detail I'd love to get this fixed.

@macdonst I literally just took the example project from this repo and built it with PGB. There were no other plugins in the project... I will try and see if there is anything else I can isolate...

Hi @macdonst — I think it's from Phonegap Build. I'm using the build service as well. Could you kindly try to compile your repo with Phonegap build at https://build.phonegap.com/

Hi Everyone, I can confirm that this is only an issue when building a project by means of PhonegapBuild.  When I open up the resulting iOS project in XCode and run it to device (or Archive and distribute from there) the confirmation dialog only appears if and when the "PushNotification.init" call takes place.
However as mentioned by @benmcmaster when I upload and build through PGB the confirmation dialog appears as soon as the app starts up the first time. Even if I remove all of the plugin referencing / calling code completely but just leave the plugin element in the config.xml file.

It's now working with release 1.4.2 built on PGB. I don't get the confirmation message on app startup on iOS 9. Can anyone else confirm this? Anyone have an iOS 8 test device?

For anyone stumbling upon this thread searching for this very same issue, in later version of iOS (8 onwards I believe) it is also necessary to get permission to set badges. So if you make an attempt to set a badge using a different plugin before calling push.init() then the permission dialog will be shown.

Closing, inactivity. Also, we are up to iOS 11.3 now so supporting 8.x is not longer required.


Updates from Julien

I need to rollback the commit for RxJava and Portfolio and then finish the Audit with Rx.
Then I can start looking at the doc

I've reverted the portfolio rx commit


Support for rendering custom attributes.

h('input', {props:{"data-id": "test-data-id", "id": "test-input"}})

outputs only
<input id="test-input">

data-id attribute is not rendering. http://requirebin.com/?gist=72ff6f4487069d2239cf Sometime users want to custom attributes on dom nodes.
Related issue: https://github.com/paldepind/snabbdom/issues/10

Yes. You are correct. This is because the property module sets properties on DOM nodes and not HTML attributes. @yelouafi has written an attribute module that I've just merged and published.
I haven't tested it myself but it should be possible to add data attributes like this:
h('input', {attrs: {"data-id": "test-data-id"}, props: {id: "test-input"}})

Also check out the documentation.
Let me know if this solves your issue :)

Sorry http://requirebin.com/?gist=72ff6f4487069d2239cf  working now.


code changes to get to compilation stage

runnable code draft is complete and compilation bugs can be resolved

since this is develop branch, just merging to facilitate assistance from others


comment

It's very important to write the basics of the library that you used in your program.

Adding some comments in the program that is much important for beginners to understand what you want to do in that program.

Thanks but these comments add no real value.


about pretrain weight

Hello, could you please release your pre training weights on the LLVIP dataset?

Thank you for your reply!
@.***
From: silee
Date: 2024-04-23 09:40
To: sejong-rcv/INSANet
CC: frreason; Author
Subject: Re: [sejong-rcv/INSANet] about pretrain weight (Issue #1)
Thank you for your interest in our research.
Unfortunately, we only organize our code for KAIST datasets and due to other environment dependencies reasons, we no longer have access to the experiment code about the LLVIP dataset. We apologize for any inconvenience.
To alleviate any inconvenience, we have attached a link containing the files we configured for training on the LLVIP dataset and our detection results file (INSANet_LLVIP_test_det.txt). Feel free to ask about related to reimplementation. LLVIP
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


Javascript is painful to work with

I am making a lot of use of the custom javascript in just about every area it is possible to use it. It's been very challenging to work with for a few reasons:

[ ] no apparent way to console.log
[ ] no good/helpful error handling
[ ] comment support is intermittent at best. I have had some builds fail silently because a comment was in the javascript, or on a separate line in the javascript as opposed to tacked on to the end. Or sometimes, i get an error in the javascript that comes down to a comment, but there is helpful information to figure this out other than just removing lines until it stops breaking.
[ ] syntax is picky. sometimes it's upset about semicolons existing, sometimes not. Sometimes it seems ok with es6, sometimes not, but either way, it fails awkwardly, or doesn't fail at all

@mistersender
All functions you can find in the node global should be available by default, so you should be able to use console.log like normal.
When you have a multi line function don't use > to declare multiple lines because > declares a really long string without including the line breaks. Instead you should use | to declare multiple lines. For more about it you can look at this stack overflow issue
The paragraph above should explain why your js wasn't running correctly when you added in comments.
After looking through the current code, this app does not support babel transpiling. At the point where we take the string you pass in (aka pre_build, post_build, etc.) and convert it into a real JS function we don't modify the code at all. So if the version of node you're running supports that part of es6 then it will work. If it doesn't support it then it will error out. I'm currently working on a branch for the 1.0.0 release that will enable the support for transpiling if a babel config exists in the .babelrc or package.json.


error while running mp_face_mesh demo python file using the github repo

runfile('F:/projects/media_pipe/mediapipe-master/test.py', wdir='F:/projects/media_pipe/mediapipe-master')
Reloaded modules: mediapipe
Traceback (most recent call last):
File "F:\projects\media_pipe\mediapipe-master\test.py", line 12, in
from mediapipe.python.solutions import drawing_utils as mp_drawing
File "F:\projects\media_pipe\mediapipe-master\mediapipe\python_init_.py", line 17, in
from mediapipe.python._framework_bindings import resource_util
ModuleNotFoundError: No module named 'mediapipe.python._framework_bindings'

i am running it on windows 10

How do you install mediapipe? Can you show the details of your python site-packages/mediapipe/python dir?

Also, I realize that some users launch the python runtime under the mediapipe repo, which might cause this issue.
If you clone the repo, please leave your local mediapipe repo directory and launch your python runtime under some other directory such as the root directory. Otherwise, python will read the mediapipe source directory and can't find the binary binding library.

okay got it . Thanks for the response

@prasadkevin did running from another directory fix this for you? Running this on RPI4 and the error persists.

did running from another directory fix this for you? Running this on RPI4 and the error persists.

@prasadkevin did running from another directory fix this for you? Running this on RPI4 and the error persists.

I also have this problem.

did running from another directory fix this for you? Running this on RPI4 and the error persists.

@prasadkevin did running from another directory fix this for you? Running this on RPI4 and the error persists.

I also have this problem.

Please refer to:
https://github.com/google/mediapipe/issues/1346#issuecomment-852391743
and https://pypi.org/project/mediapipe-rpi4/
I believe this at least avoid the above errors.

d runni

actually,I follow the step. However, I still got a problem.
Traceback (most recent call last):
File "/home/pi/Desktop/hands.py", line 2, in
import mediapipe as mp
File "/usr/local/lib/python3.7/dist-packages/mediapipe/init.py", line 16, in
from mediapipe.python import *
File "/usr/local/lib/python3.7/dist-packages/mediapipe/python/init.py", line 17, in
from mediapipe.python._framework_bindings import resource_util
ModuleNotFoundError: No module named 'mediapipe.python._framework_bindings'

d runni

actually,I follow the step. However, I still got a problem. Traceback (most recent call last): File "/home/pi/Desktop/hands.py", line 2, in import mediapipe as mp File "/usr/local/lib/python3.7/dist-packages/mediapipe/init.py", line 16, in from mediapipe.python import * File "/usr/local/lib/python3.7/dist-packages/mediapipe/python/init.py", line 17, in from mediapipe.python._framework_bindings import resource_util ModuleNotFoundError: No module named 'mediapipe.python._framework_bindings'

Did you figure it out the problem? I am also stuck in this error.


Remove bloat from api and frontend

Used big base images and installed many unused packages.
These changes make it lighter and faster to work with.

some minor beautification of the code

looks good


Data Model schema VS DTO

Data Model VS DTO
Summary
Do we need to maintain both a data model and a DTO or can we combine?
Current Behaviour
Currently each endpoint has a data model defined as a schema and the DTO. These are two separate files that needs to be both maintained and updated everytime there is a change to the data model.
Expected Behaviour
We wonder if it is possible to merge the two and maintain a single file for each data model.

I found the following post that ask exactly this question:
https://stackoverflow.com/questions/65769384/models-vs-dto-in-nestjs

More information:

https://en.wikipedia.org/wiki/Data_access_object
https://en.wikipedia.org/wiki/Data_transfer_object
https://en.wikipedia.org/wiki/Single-responsibility_principle

https://docs.nestjs.com/controllers
https://docs.nestjs.com/techniques/validation

https://medium.com/globant/crud-application-using-nestjs-and-mongodb-99a0756adb76

This is a good list of links to read. My perspective on this REST and models is admittedly slightly different (I favor a single model rather than a model and a DTO model to DRY (Don't Repeat Yourself), but I also wouldn't want to diverge from the accepted patterns of nest.js. Time to read more...

Closing for lack of activity.


Create FitnessOrderedGeneticAlgorithm

It's a modified version of the StandardGeneticAlgorithm where it sorts instances with higher fitness value and than performs elitims on top n and mate them. It is computationally costly because uses sorting but, it should converge much faster.

Thank you for this addition.


Enhancement - label on Resource Object

Request to add a label field to the Resource Object and display that on the swagger-ui if present instead of the path field.
Reason - I use a library that generates the swagger spec from service at runtime with endpoint of /swagger.  I have a top level service that generates a Resource Listing from a registry of active services.  In the swagger-ui, each service shows up with a label of "swagger" (i.e., the path).  So I get a multi-service list of .../registration/swagger, .../identity/swagger, etc. which just displays the last node in the path ("swagger").

Screen shot with mod to swagger-ui.js to display label (temporary hack on older version).

That's actually already resolved in Swagger 2.0 as there are no longer resources but tags. You can assign a path to zero or more tags, and it will appear in all of them in the UI. If there are no tags, the ui will put them under the 'default' group. Hope this helps.

Looks like I'll have to update my library to generate a 2.0 spec as opposed to 1.2.  Go figure, submit a request and generate work for myself.  Cheers.

The best kind of issues ;-)

I feel your pain but 1.2 is a done deal so making enhancements to it doesn't make sense when its successor is already out.

Definitely puts a crimp in my procrastination efforts. ;-)  Btw, cool product guys.  We're starting the full roll out to our dev staff this next week.

And you're releasing your cool library as a FOSS?!

Of course.  I prototype in Go in a Java dev shop, so most of my stuff goes FOSS.  Just need to improve my lack of documentation skills.

Documentation is for weak people.

I have the basics done to produce a 2.0 doc on a single service.  Have some work to do on complex types in the definitions section as well as the security portion.  The schema object is a beast.  I have a few ideas to aggregate multiple services into a single doc, but can already see some complications.  I'll use the forum as I hit roadblocks.  One thing I see is name collision in the definitions section.  One idea to help would be a namespace attribute in the schema.  The thing driving the single page of services is that we're pursuing micro services and feel a single inventory will promote reuse.
I am getting the error icon on the page, but since I'm behind the firewall, the debugger redirect doesn't work.  I paste the json in the Swagger Editor and it says it's a valid doc.  No biggie right now.
If you want samples for the spec (see a lot of TODO), let me know.

Sure thing, would be happy to help where possible. Feel free to join us on IRC as well for a more interactive conversation if you want.


added go-http example

Description of changes:
I added a simple Golang example using the net/http core package and zip method.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for the contribution.


Add ORA_NCHAR_LITERAL_REPLACE setting to "Common Problems"

Our development team recently encountered an issue with characters outside of ISO/IEC 8859 not being inserted into an Oracle Database correctly via the Oracle connector for Logic Apps. They would instead be inserted as upside-down question marks (i.e. "¿"), even if the column in question could support the provided characters (e.g. a column of type NCLOB with a national character set of AL16UTF16).
Thanks to help from Microsoft Support, we resolved this by setting a system-wide environment variable ORA_NCHAR_LITERAL_REPLACE to "TRUE" on the on-premises data gateway where the Oracle client was installed. This setting (described as a one-off in this Oracle DB documentation on "Literals") was the only mention I could find of this environment variable. From what I can tell, the problem stemmed from our normal character encoding of WE8ISO8859P1 being used for all character literals during INSERT, even though our national character encoding of AL16UTF16 supported a wider range of characters.
This issue appears to me as a common scenario for legacy databases that started with a restricted character set before expanding to a larger character set using NCHAR, NVARCHAR2 or NCLOB. Given that the solution involves updating the On-premises data gateway and the "Common Problems" section covers issues related to this gateway, it looks to me like the most appropriate place for this to be described.
Is this problem a good candidate for the "Common Problems" section? Is it better suited being described elsewhere?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f7f70535-1e83-54f0-4bc8-b80f5919f118
Version Independent ID: 57f5c3a0-7a24-ce56-d84c-542d85ffe6a9
Content: Connect to Oracle Database - Azure Logic Apps
Content Source: articles/connectors/connectors-create-api-oracledatabase.md
Service: logic-apps
GitHub Login: @ecfan
Microsoft Alias: estfan

@TomChapple Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

@TomChapple: We've added this request to our work backlog. Thank you!
@MayankBargali-MSFT: #please-close