Add GPU support and improve performance for tf.diag and tf.diag_part

This PR tries to settle the issue #13491 and makes some contribution for tf.diag and tf.diag_part:

Add GPU support.
Using a simple transfer trick makes code clear and easy to be parallel.
Rewrite the CPU code using the recent rewrite of matrix_band_part as a template, which getting rid of the Eigen generator mechanism, remove the restriction which the input rank is at most 3.

Implementation of transfer trick can be described as follows.
According to the tf.diag op definition, output[i1,..., ik, i1,..., ik] = input[i1,..., ik].
Let the rank of input is [s1,..., sk], then any offset of input's pointer can be represented by coordinate [i1,..., ik], where index = i1*(s2*...*sk) + i2*(s3*...*sk) +... + ik.
Let new_index is the offset of output's pointer with coordinate [i1,..., ik, i1,..., ik], then we have
new_index = i1*(s2*...sk*s1*...*sk) + i2*(s3*...*sk*s1*...*sk) +... + \
            ik*(s1*...*sk) + i1*(s2*...*sk) + i2*(s3*...*sk) +... + ik
          = (i1*(s2*...*sk) + i2*(s3*...*sk) +... + ik) * (1 + s1*...*sk)
          = index * (1 + s1*...*sk)

Let size = s1*...*sk, we finally have new_index = index * (1 + size), which is the transfer function we use below.
This trick make our implementations clear and easy to be parallel.
If there is anything I need to modify, please let me know. Thank you for your review.

Can one of the admins verify this patch?

@tensorflow-jenkins test this please

I am not sure whether I need to modify the BUILD script, because I add two files namely diag_op.h and diag_op_gpu.cu.cc.

@cwhipkey
I guess I could optimize it by multi-thread shard, should I do it for in this PR?

@cwhipkey
Is that good to go? Thank you very much~

Sorry for the delay, I'll review it in detail today.
One question - do you have benchmarks that show the difference in performance of old vs new on CPU?  (there are some other micro-benchmark examples, like in core/kernels/matmul_op_test.cc - with the microbenchmarks, you then build the binary and run the test binary with args like   --benchmarks=all --benchmark_min_time=20 or something).

@cwhipkey Thank you for your reviews.

I've made changes according to your comments.
I've add shard function for CPU version in this PR, which gets rid of trouble of a new PR.
I've add the benchmark in core/kernel/diag_op_test.cc, and compare with the old in performance.

In general, current CPU version can speed 10 times, and GPU version can speed more than 100 times for vector of length 512.
Here are the results:
Old CPU Version (96f334b0d6ec61f5037b8ddd1860a9aa7874efef)
$bazel test :diag_op_test --test_arg=--benchmarks=all --test_output=all
...
==================== Test output for //tensorflow/core/kernels:diag_op_test:
2017-10-18 07:12:01.963569: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Running main() from test_main.cc
Benchmark                      Time(ns) Iterations
--------------------------------------------------
BM_Diag_16_DT_INT32_cpu           25606      28894       0.6M items/s
BM_Diag_16_DT_FLOAT_cpu           23701      29621       0.7M items/s
BM_Diag_16_DT_COMPLEX64_cpu       22794      30482       0.7M items/s
BM_Diag_128_DT_INT32_cpu         236919       3006       0.5M items/s
BM_Diag_128_DT_FLOAT_cpu         225901       3035       0.6M items/s
BM_Diag_128_DT_COMPLEX64_cpu     248124       2874       0.5M items/s
BM_Diag_512_DT_INT32_cpu        3368587        208       0.2M items/s
BM_Diag_512_DT_FLOAT_cpu        3242726        215       0.2M items/s
BM_Diag_512_DT_COMPLEX64_cpu    3517025        199       0.1M items/s
================================================================================
//tensorflow/core/kernels:diag_op_test                                   PASSED in 9.3s

Executed 1 out of 1 test: 1 test passes.

Current CPU&GPU Version (7d5e1cee1aa3cf6124c4dd0cee225eeba63be8d2)
Running main() from test_main.cc
Benchmark                      Time(ns) Iterations
--------------------------------------------------
BM_Diag_16_DT_INT32_cpu           18760      34370       0.9M items/s
BM_Diag_16_DT_FLOAT_cpu           19241      34903       0.8M items/s
BM_Diag_16_DT_COMPLEX64_cpu       19791      33423       0.8M items/s
BM_Diag_16_DT_INT32_gpu           34986      20144       0.5M items/s
BM_Diag_16_DT_FLOAT_gpu           37343      19353       0.4M items/s
BM_Diag_16_DT_COMPLEX64_gpu       35488      19930       0.5M items/s
BM_Diag_128_DT_INT32_cpu          22539      29165       5.7M items/s
BM_Diag_128_DT_FLOAT_cpu          26046      31457       4.9M items/s
BM_Diag_128_DT_COMPLEX64_cpu      29816      20297       4.3M items/s
BM_Diag_128_DT_INT32_gpu          36148      19113       3.5M items/s
BM_Diag_128_DT_FLOAT_gpu          34664      19109       3.7M items/s
BM_Diag_128_DT_COMPLEX64_gpu      35264      19279       3.6M items/s
BM_Diag_512_DT_INT32_cpu          74927       9439       6.8M items/s
BM_Diag_512_DT_FLOAT_cpu          73087       9120       7.0M items/s
BM_Diag_512_DT_COMPLEX64_cpu      86292       7751       5.9M items/s
BM_Diag_512_DT_INT32_gpu          35550      17597       14.4M items/s
BM_Diag_512_DT_FLOAT_gpu          34587      19479       14.8M items/s
BM_Diag_512_DT_COMPLEX64_gpu      37200      19071       13.8M items/s
================================================================================
//tensorflow/core/kernels:diag_op_test                          (cached) PASSED in 20.7s

Executed 0 out of 1 test: 1 test passes.

System information of test environment

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Server release 7.2 (RedHat)
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.3.0
Python version: Python 3.6.1 :: Anaconda 4.4.0 (64-bit)
Bazel version (if compiling from source): 0.6.1
CUDA/cuDNN version: 8.0/5.1.3
GPU model and memory: Tesla M40/12G
Exact command to reproduce:

Can one of the admins verify this patch?

Sorry for that I just close this PR accidentally...

@cwhipkey Thank you for your reviews.

I've made changes according to your comments.
I've add shard function for CPU version in this PR, which gets rid of trouble of a new PR.
I've add the benchmark in core/kernel/diag_op_test.cc, and compare with the old in performance.

In general, current CPU version can speed 10 times, and GPU version can speed more than 100 times for vector of length 512.
Here are the results:
Old CPU Version (96f334b0d6ec61f5037b8ddd1860a9aa7874efef)
$bazel test :diag_op_test --test_arg=--benchmarks=all --test_output=all
...
==================== Test output for //tensorflow/core/kernels:diag_op_test:
2017-10-18 07:12:01.963569: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Running main() from test_main.cc
Benchmark                      Time(ns) Iterations
--------------------------------------------------
BM_Diag_16_DT_INT32_cpu           25606      28894       0.6M items/s
BM_Diag_16_DT_FLOAT_cpu           23701      29621       0.7M items/s
BM_Diag_16_DT_COMPLEX64_cpu       22794      30482       0.7M items/s
BM_Diag_128_DT_INT32_cpu         236919       3006       0.5M items/s
BM_Diag_128_DT_FLOAT_cpu         225901       3035       0.6M items/s
BM_Diag_128_DT_COMPLEX64_cpu     248124       2874       0.5M items/s
BM_Diag_512_DT_INT32_cpu        3368587        208       0.2M items/s
BM_Diag_512_DT_FLOAT_cpu        3242726        215       0.2M items/s
BM_Diag_512_DT_COMPLEX64_cpu    3517025        199       0.1M items/s
================================================================================
//tensorflow/core/kernels:diag_op_test                                   PASSED in 9.3s

Executed 1 out of 1 test: 1 test passes.

Current CPU&GPU Version (36e9764583f856a00fb3573daaea74812c8edf47)
Running main() from test_main.cc
Benchmark                      Time(ns) Iterations
--------------------------------------------------
BM_Diag_16_DT_INT32_cpu           18760      34370       0.9M items/s
BM_Diag_16_DT_FLOAT_cpu           19241      34903       0.8M items/s
BM_Diag_16_DT_COMPLEX64_cpu       19791      33423       0.8M items/s
BM_Diag_16_DT_INT32_gpu           34986      20144       0.5M items/s
BM_Diag_16_DT_FLOAT_gpu           37343      19353       0.4M items/s
BM_Diag_16_DT_COMPLEX64_gpu       35488      19930       0.5M items/s
BM_Diag_128_DT_INT32_cpu          22539      29165       5.7M items/s
BM_Diag_128_DT_FLOAT_cpu          26046      31457       4.9M items/s
BM_Diag_128_DT_COMPLEX64_cpu      29816      20297       4.3M items/s
BM_Diag_128_DT_INT32_gpu          36148      19113       3.5M items/s
BM_Diag_128_DT_FLOAT_gpu          34664      19109       3.7M items/s
BM_Diag_128_DT_COMPLEX64_gpu      35264      19279       3.6M items/s
BM_Diag_512_DT_INT32_cpu          74927       9439       6.8M items/s
BM_Diag_512_DT_FLOAT_cpu          73087       9120       7.0M items/s
BM_Diag_512_DT_COMPLEX64_cpu      86292       7751       5.9M items/s
BM_Diag_512_DT_INT32_gpu          35550      17597       14.4M items/s
BM_Diag_512_DT_FLOAT_gpu          34587      19479       14.8M items/s
BM_Diag_512_DT_COMPLEX64_gpu      37200      19071       13.8M items/s
================================================================================
//tensorflow/core/kernels:diag_op_test                          (cached) PASSED in 20.7s

Executed 0 out of 1 test: 1 test passes.

System information of test environment

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Server release 7.2 (RedHat)
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.3.0
Python version: Python 3.6.1 :: Anaconda 4.4.0 (64-bit)
Bazel version (if compiling from source): 0.6.1
CUDA/cuDNN version: 8.0/5.1.3
GPU model and memory: Tesla M40/12G
Exact command to reproduce:

(note for mergers - there are a couple small items to be changed before merging.  I may have picked the wrong 'approval' option in the review dialog, I'm not positive).

Thanks @cwhipkey for the review. This PR has been updated according to your comments. Please take a look.

jenkins, test this please

Seems that the Ubuntu CC is stalling?

@cwhipkey Seems that the Ubuntu CC test is stalling?

jenkins, test this please

@cwhipkey
Seems that the Ubuntu CC is still not running. Is there something wrong with the check program?

@cwhipkey @vrv Thank you for your attention.
The test failure of Ubuntu contrib above seems not be related with this PR according to the log. So is there anything I can do to avoid it like merging latest master branch?
Here is the tail part of the log:
//tensorflow/contrib/data/python/kernel_tests:sloppy_transformation_dataset_op_test TIMEOUT in 352.5s
  /tmpfs/tmp/bazel/execroot/org_tensorflow/bazel-out/linux_gnu_x86-py3-opt/testlogs/tensorflow/contrib/data/python/kernel_tests/sloppy_transformation_dataset_op_test/test.log

Executed 399 out of 399 tests: 398 tests pass and 1 fails locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

It's my pleasure to make some contributions for tf~


CHEBI:139655, please update structure

Hi Adnan,
The current structure is not consistent with the usual way to depict CoA compounds. The attached compound is. I hope you will use it.
Cheers,
Kristian
CHEBI_139655.txt

structure updated.

structure updated.


minecraft bug crash

Ubuntu
Openjdk 16.0.1 2021-04-20
OpenJDK Runtime Environment Microsoft-22300 (build 16.0.1+9)
OpenJDK 64-Bit Server VM Microsoft-22300 (build 16.0.1+9, mixed mode)
Start Minecraft
Logs is in attached file
hs_err_pid12238.log
I've not more information

Hi @Lolothepro! This looks like an LWJGL bug. Can you please submit this issue over at https://bugs.mojang.com/projects/MC/issues/? The Mojang team will take the first look. Thanks!

They have reported it as MC-237569 (private report), but with different non LWJGL-related?) JVM crashes attached.

So I guess I have to wait for the bug to be fixed, right?
any idea how long before Mojang fixes the bug?

It is difficult to tell when the developers will fix this bug due to the large number of bug reports.
Could you please in the meantime try updating your graphics card drivers and let us know if that helped.
In case that does not help you can also try contacting Community Support.


Clarification regarding QAOA implementation

The original qaoa algorithm introduced in the paper by E Farhi uses the same beta parameter for all the qubits in mixer Hamiltonian propagator , same gamma for all the terms in cost Hamiltonian propagator.

ham = xacc.getObservable('pauli', '-5.0 - 0.5 Z0 - 1.0 Z2 + 0.5 Z3 + 1.0 Z0 Z1 + 2.0 Z0 Z2 + 0.5 Z1 Z2 + 2.5 Z2 Z3')
nbParamsPerStep = 7 + 4

from this code block from the qaoa_example.py I think we are using different betas for each qubit in the mixer and different gammas for each term in cost Hamiltonian. Am i right?
if yes , does this provide any advantage over the original approach?

Hi @aswanthkrishna,
Yes, you're right.
The QAOA implementation in XACC used an extended parameterization scheme whereby each operator in both the cost and mixer Hamiltonians has its own angle.
Since the goal of the ansatz circuit is to produce a quantum state that optimizes the cost function, we have some degree of flexibility in choosing how to parametrize a QAOA problem.
If we choose a smaller set of parameters, where the optimization landscape has a lower dimension at the expense of higher circuit depth. If we choose a larger set, we can explore a wider set of quantum states with lower circuit depth, of course, at the expense of higher optimization dimension.
In the implementation, we opted for the extended scheme which is a superset of the standard scheme (bundle a group of betas and gammas per step into one parameter) for maximum flexibility and extensibility.

Thank you for the quick reply @tnguyen-ornl
typically wont this make the optimisation landscape complex and introduce more local minimas? in my experiment the value to which qaoa converges is highly sensitive to the starting point. Can you suggest some ways that i can solve this?

I don't have a concrete answer to the local minima problem.
I totally agree with you about the sensitivity to initial parameters. In fact, this has been studied quite extensively. There are some heuristic initialization strategies that were shown to be much better than random initialization.
For example, see arXiv:1812.01041

Thanks for the reference. Also how will i go about bundling the parameters as u suggested with the current implementation? I want to compare the performance of both approaches. thank you for your patience.

This is a Python example that shows how we can bundle params and also apply a simple initialization strategy:
import xacc,sys, numpy as np

# == Demonstrated custom QAOA API ==

# Get access to the desired QPU
qpu = xacc.getAccelerator('qpp')

# Construct the Hamiltonian as an XACC PauliOperator
# This Hamiltonian corresponds to the QUBO problem:
# y = -5x_1 -3x_2 -8x_3 -6x_4 + 4x_1x_2 + 8x_1x_3 + 2x_2x_3 + 10x_3x_4
ham = xacc.getObservable('pauli', '-5.0 - 0.5 Z0 - 1.0 Z2 + 0.5 Z3 + 1.0 Z0 Z1 + 2.0 Z0 Z2 + 0.5 Z1 Z2 + 2.5 Z2 Z3')

# We need 4 qubits
buffer = xacc.qalloc(4)

# The number of steps (often referred to as 'p' parameter):
# alternating layers of mixer and cost Hamiltonian exponential.
nbSteps = 8

# Example: using vanilla QAOA (nbParams =  2 * nbSteps)
nbTotalParams = 2 * nbSteps

# Init params:
# Example: using an *Annealing* initial params:
# i.e. gamma = (1 - s(t)); beta = s(t)
# ramping down gamma and ramping up beta
s = lambda step : float(step/nbSteps)
initParams = []
for i in range(nbSteps):
    # Convention: beta then gamma
    initParams.append(s(i))
    initParams.append(1.0 - s(i))
# Get an optimizer/stepper
opt = xacc.getOptimizer('nlopt', { 'initial-parameters': initParams })

# Create the QAOA algorithm
qaoa = xacc.getAlgorithm('QAOA', {
                        'accelerator': qpu,
                        'observable': ham,
                        'optimizer': opt,
                        'steps': nbSteps,
                        })

# Define optimization function:
def qaoa_vanilla(x):
    assert len(x) == nbTotalParams
    # NOTE: here, we bundle betas and gammas per step into one beta and one gamma
    # i.e. the vanila QAOA.
    # Convention: betas then gammas
    NUM_BETAS_PER_STEP = 7
    NUM_GAMMAS_PER_STEP = 4
    flattenedParams = []
    for i in range(nbSteps):
        flattenedParams += (NUM_BETAS_PER_STEP * [x[2*i]])
        flattenedParams += (NUM_GAMMAS_PER_STEP * [x[2*i + 1]])

    assert len(flattenedParams) == nbSteps * (NUM_GAMMAS_PER_STEP + NUM_BETAS_PER_STEP)
    energy = qaoa.execute(buffer, flattenedParams)
    print("Energy = ", energy[0])
    return energy[0]

enery, params = opt.optimize(qaoa_vanilla, nbTotalParams)
print('Min QUBO value = ', enery)

Thank you!


Added config get and set message

This provides an API for setting configuration items that are common to all nodes, and getting their current configuration state from a node. It is a simple, fixed-size format that is easier to implement and debug than GetSet. This proposal uses byte-aligned, constant-size fields, and groups together important and universal settings.
Reasons why I think this will be easier to use:

GetSet has padding bits
GetSet uses the variable-length name field. Strings are a bit awkward to work with here, and aren't required.
GetSet's Value and NumericalValue fields are variable-length; especially for handling string format.
It is unclear from its spec how GetSet's set vs get, and expected acknowledgement format work.

Overall, managing GetSet's alignment has nested levels of bit-level, variable-length alignment, which is tricky to implement and debug, given the simple nature of its task. This applies to both serializing and deserializing.
I am open to ideas on data type IDs, fields to add, etc. A notable downside of this proposal is that the format is not very efficient due to byte-level packing, including of fields that have 2 bits. Given this message is expected to be sent infrequently, and is still compact (less than a full legacy frame), I think this is a worthwhile tradeoff.

I have a better understanding of services now; this PR is not necessary.


CMS-348: Add CSV export feature

Jira Ticket
CMS-348
Description

The CSVs require some info we weren't pulling from strapi, and we don't need that info anywhere else so I didn't want to two new FK tables and a junction table just for that. I tried a few approaches and this was the cleanest/simplest one:

Added a jsonb field to the Park model called "managementAreas" which stores an array like this:

[
  {
    "mgmtArea": {
      "strapiId": 123,
      "name": "North Okanagan"
    },
    "section": {
      "strapiId": 57,
      "name": "Okanagan"
    }
  }
]

The details get fetched in the normal syncData script, and the CSV export just prints out the names from that array.
In particular, I'd like someone to look at the Sequelize query in the Export endpoint, because was guessing at some of the sub-query/join stuff. I've been testing it since I imported the real Strapi data and it looks okay, but let me know if you notice anything strange.
Other stuff:

Updated the eslint config to stop showing errors in the seeders/migrations/.adminJs folders with auto-generated files.
Refactored the Express routes slightly so they're all sub-routes of /api/ and we don't need to add /api/ and the base path to every new route we define.
Tweaked the fluid width of the flash message so it looks better with the long message on the Export page

Rebased and resolved conflicts. The requirements changed again this morning so I'll push another update and then merge it 🫠


[vm] Races from dart:io's exit skipping Dart_Cleanup

There are new test failures on [dart2js] Implement @pragma('dart2js:resource-identifier')...[cfe] Add MapPatternRestEntry.
The tests
lib/isolate/regress_34752_test/0 Crash (expected Pass)

are failing on configurations
vm-msan-linux-release-x64

--- Command "vm" (took 06.000337s):
DART_CONFIGURATION=ReleaseMSANX64 out/ReleaseMSANX64/dart --no-enable-fast-object-copy --sound-null-safety -Dtest_runner.configuration=vm-msan-linux-release-x64 --ignore-unrecognized-flags --packages=/b/s/w/ir/.dart_tool/package_config.json /b/s/w/ir/tests/lib/isolate/regress_34752_test.dart

exit code:
-6

stdout:
ok

stderr:
==8917==WARNING: MemorySanitizer: use-of-uninitialized-value
==8917==WARNING: invalid path to external symbolizer!
==8917==WARNING: Failed to use and restart external symbolizer!
    #0 0x55be9c95069f  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x31d669f)
    #1 0x55be9c9500b6  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x31d60b6)
    #2 0x55be9c950f83  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x31d6f83)
    #3 0x55be9c0dcbe2  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x2962be2)
    #4 0x55be9c0de68f  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x296468f)
    #5 0x55be9c0df61d  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x296561d)
    #6 0x55be9c0e0701  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x2966701)
    #7 0x55be9c0aa0b6  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x29300b6)
    #8 0x55be9c0aae51  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x2930e51)
    #9 0x55be9bf5142b  (/b/s/w/ir/out/ReleaseMSANX64/dart+0x27d742b)
    #10 0x7fee780fc6da  (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da) (BuildId: 1f06001733b9be9478b105faf0dac6bdf36c85de)
    #11 0x7fee7766561e  (/lib/x86_64-linux-gnu/libc.so.6+0x12161e) (BuildId: f7307432a8b162377e77a182b6cc2e53d771ec4b)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/b/s/w/ir/out/ReleaseMSANX64/dart+0x31d669f)
Exiting

--- Re-run this test:

log
The test tests message passing between isolates.
There is a single commit in the range that deals with sending messages: 7c0a6946684c1721925377be944c596a92633436
I don't see an obvious uninitialized value in that CL though.
@rmacnak-google could you take a look if your CL caused this? If not feel free to re-assign this issue back to me, and I'll do the bisecting of the commit range locally.

If I run the whole lib/isolates suite I get a few flakes like:
==3028127==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55ef35986373 in dart::CompilerPass::flags() const ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.h:142:34
    #1 0x55ef35986373 in dart::CompilerPass::Run(dart::CompilerPassState*) const ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:207:8
    #2 0x55ef35986717 in dart::CompilerPass::RunInliningPipeline(dart::CompilerPass::PipelineMode, dart::CompilerPassState*) ../../out/ReleaseMSANX64/../../runtime/vm/compiler\
/compiler_pass.cc:284:3
    #3 0x55ef358afa7e in dart::CallSiteInliner::TryInliningImpl(dart::Function const&, dart::Array const&, dart::InlinedCallData*, bool) ../../out/ReleaseMSANX64/../../runtime\
/vm/compiler/backend/inliner.cc:1501:13
    #4 0x55ef35894b6c in dart::CallSiteInliner::TryInlining(dart::Function const&, dart::Array const&, dart::InlinedCallData*, bool) ../../out/ReleaseMSANX64/../../runtime/vm/\
compiler/backend/inliner.cc:1178:26
    #5 0x55ef358b73ac in dart::CallSiteInliner::InlineStaticCalls() ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1805:11
    #6 0x55ef358a147b in dart::CallSiteInliner::InlineCalls() ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1130:30
    #7 0x55ef358a0894 in dart::FlowGraphInliner::Inline() ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:2714:11
    #8 0x55ef35987616 in dart::CompilerPass_Inlining::DoBody(dart::CompilerPassState*) const ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:421:1
    #9 0x55ef35985f58 in dart::CompilerPass::Run(dart::CompilerPassState*) const ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:229:18
    #10 0x55ef35986c1f in dart::CompilerPass::RunPipeline(dart::CompilerPass::PipelineMode, dart::CompilerPassState*) ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compil\
er_pass.cc:342:3
    #11 0x55ef35112bc2 in dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*) ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:568:22
    #12 0x55ef3511466f in dart::CompileFunctionHelper(dart::CompilationPipeline*, dart::Function const&, bool, long) ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/com\
piler.cc:714:46
    #13 0x55ef351155fd in dart::Compiler::CompileOptimizedFunction(dart::Thread*, dart::Function const&, long) ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.\
cc:899:10
    #14 0x55ef351166e1 in dart::BackgroundCompiler::Run() ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:1139:7
    #15 0x55ef350e0096 in dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) ../../out/ReleaseMSANX64/../../runtime/vm/thread_pool.cc:158:15
    #16 0x55ef350e0e31 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseMSANX64/../../runtime/vm/thread_pool.cc:330:9
    #17 0x55ef34f8740b in dart::ThreadStart(void*) ../../out/ReleaseMSANX64/../../runtime/vm/os_thread_linux.cc:154:5
    #18 0x7fd20615cfd3 in start_thread nptl/pthread_create.c:442:8
    #19 0x7fd2061dd66b in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

So maybe my CL is related because it added a prefer-inline pragma that causes the first invocation of the inlining pipeline to occur sooner.
CompilerPass::flags_ is globally initialized though, so I don't understand how MSAN could believe it is not initialized.

Hm, the background compiler is racing with the global destructors:
Thread 9 (Thread 3337147.3338425 (DartWorker)):
#0  0x000056198b2708e1 in PoisonMemory () at ../staging/llvm_build/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/msan/msan_poisoning.cpp:251
#1  0x000056198b21d236 in __sanitizer_dtor_callback_fields () at ../staging/llvm_build/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/msan/msan_interceptors.cpp:963
#2  0x000056198b8b4f75 in dart::compiler::RuntimeEntry::~RuntimeEntry (this=<optimized out>) at ../../runtime/vm/compiler/runtime_api.h:235
#3  dart::compiler::RuntimeEntry::~RuntimeEntry (this=0x56198def05f0 <dart::kNoSuchMethodFromPrologueRuntimeEntry>) at ../../runtime/vm/compiler/runtime_api.h:223
#4  dart::RuntimeEntry::~RuntimeEntry (this=0x56198def05f0 <dart::kNoSuchMethodFromPrologueRuntimeEntry>) at ../../runtime/vm/runtime_entry.h:31
#5  0x000056198b21d9ec in MSanCxaAtExitWrapper(void*) () at ../staging/llvm_build/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/msan/msan_interceptors.cpp:1180
#6  0x00007f72cf2a6435 in __run_exit_handlers (status=0, listp=0x7f72cf43a820 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at ./stdlib/exit.c:113
#7  0x00007f72cf2a65aa in __GI_exit (status=<optimized out>) at ./stdlib/exit.c:143
#8  0x000056198b2b3ef9 in dart::bin::Platform::Exit (exit_code=0) at ../../out/ReleaseMSANX64/../../runtime/bin/platform_linux.cc:172
#9  0x000056198b2b5100 in dart::bin::Builtin_Process_Exit (args=<optimized out>) at ../../out/ReleaseMSANX64/../../runtime/bin/process.cc:266
#10 0x000056198b6ec76e in dart::NativeEntry::AutoScopeNativeCallWrapperNoStackCheck (args=0x7f72c21f9070, func=0x56198b2b5080 <dart::bin::Builtin_Process_Exit(_Dart_NativeArguments*)>) at ../../out/ReleaseMSANX64/../../runtime/vm/native_entry.cc:217
#11 0x00007f72cd782cc4 in ?? ()
#12 0x000071f000004600 in ?? ()
#13 0x0000000000000001 in ?? ()
#14 0x00007f72c21f90c0 in ?? ()
#15 0x00007f72c21f90b8 in ?? ()
#16 0x00007f72c21f90c0 in ?? ()
#17 0x00007f72c2753921 in ?? ()
#18 0x00007f72ce88ddf1 in ?? ()
#19 0x00007f72c21f90e0 in ?? ()
#20 0x00007f72cbcae581 in ?? ()
#21 0x00007f72ce888081 in ?? ()
#22 0x0000000000000000 in ?? ()

Thread 3 (Thread 3337147.3337333 (DartWorker)):
#0  dart::CompilerPass::Run (this=0x56198def8bf8 <dart::compiler_pass_ApplyClassIds>, state=0x7f72cb5f79b8) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:207
#1  0x000056198c27a718 in dart::CompilerPass::RunInliningPipeline (mode=<optimized out>, pass_state=0x7f72cb5f79b8) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:284
#2  0x000056198c1a3a7f in dart::CallSiteInliner::TryInliningImpl (this=this@entry=0x7f72cb5f8350, function=..., argument_names=..., call_data=0x7f72cb5f8078, stricter_heuristic=false) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1501
#3  0x000056198c188b6d in dart::CallSiteInliner::TryInlining (this=this@entry=0x7f72cb5f8350, function=..., argument_names=..., call_data=call_data@entry=0x7f72cb5f8078, stricter_heuristic=false) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1178
#4  0x000056198c1ab3ad in dart::CallSiteInliner::InlineStaticCalls (this=this@entry=0x7f72cb5f8350) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1805
#5  0x000056198c19547c in dart::CallSiteInliner::InlineCalls (this=this@entry=0x7f72cb5f8350) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:1130
#6  0x000056198c194895 in dart::FlowGraphInliner::Inline (this=0x7f72cb5f8480) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/backend/inliner.cc:2714
#7  0x000056198c27b617 in dart::CompilerPass_Inlining::DoBody (this=<optimized out>, state=0x7f72cb5f8a70) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:421
#8  0x000056198c279f59 in dart::CompilerPass::Run (this=0x56198def8bc8 <dart::compiler_pass_Inlining>, state=state@entry=0x7f72cb5f8a70) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:229
#9  0x000056198c27ac20 in dart::CompilerPass::RunPipeline (mode=<optimized out>, pass_state=0x7f72cb5f8a70) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/compiler_pass.cc:342
#10 0x000056198ba06bc3 in dart::CompileParsedFunctionHelper::Compile (this=this@entry=0x7f72cb5f9550, pipeline=pipeline@entry=0x7f72cb5f9b80) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:568
#11 0x000056198ba08670 in dart::CompileFunctionHelper (pipeline=0x7f72cb5f9b80, function=..., optimized=true, osr_id=osr_id@entry=-1) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:714
#12 0x000056198ba095fe in dart::Compiler::CompileOptimizedFunction (thread=thread@entry=0x71f000021400, function=..., osr_id=osr_id@entry=-1) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:899
#13 0x000056198ba0a6e2 in dart::BackgroundCompiler::Run (this=0x708000004980) at ../../out/ReleaseMSANX64/../../runtime/vm/compiler/jit/compiler.cc:1139
#14 0x000056198b9d4097 in dart::ThreadPool::WorkerLoop (this=this@entry=0x711000000140, worker=worker@entry=0x7030000042f0) at ../../out/ReleaseMSANX64/../../runtime/vm/thread_pool.cc:158
#15 0x000056198b9d4e32 in dart::ThreadPool::Worker::Main (args=123351460758256) at ../../out/ReleaseMSANX64/../../runtime/vm/thread_pool.cc:330
#16 0x000056198b87b40c in dart::ThreadStart (data_ptr=<optimized out>) at ../../out/ReleaseMSANX64/../../runtime/vm/os_thread_linux.cc:154
#17 0x00007f72cf2f0fd4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#18 0x00007f72cf3708d0 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

Fixing this means either making dart:io's exit go through the whole VM shutdown before calling libc's exit or telling MSAN we don't care about global destructors.

Thanks for the investigation @rmacnak-google !
https://github.com/dart-lang/sdk/blob/main/runtime/bin/process.cc#L260-L267

or telling MSAN we don't care about global destructors

Couldn't we get undefined behavior on exit that way. (If we would leak some memory that would be harmless if we exit anyway, but undefined behavior might potentially not be harmless.)

making dart:io's exit go through the whole VM shutdown

That seems to be the sane way to do it. I presume we do VM shutdown as well if we reach the end of the main function. I would be kind of surprised if we have different behavior on versus reaching the end of the main function. E.g. users writing
void main(){
  if (/*...*/) {
    exit(2);
  }
  // exits with 0
}

would be rather surprised if exit(2) is much quicker and has different semantics.
Do we have any idea how much slower the whole VM shutdown is?

I'm more worried about it getting stuck than being slow. The VM shutdown needs to wait for every isolate to respond to an OOB kill message, which might never happen if an isolate is blocked in C code.

I stepped through what happens at exit and there are several things getting destroyed that could provoke crashes in other Dart threads: dart::field_offsets_table, dart::Log::noop_log_, dart::bin::dfe_.

There are at least two races:

exit triggering global destructors still being used by Dart or background compiler threads. Affects both MSAN and uninstrumented builds
exit triggering LSAN between new VirtualMemory and LSAN_REGISTER_ROOT_REGION. Should only affect LSAN.

Related: skipping Dart_Cleanup also triggers thread leak reports from TSAN (https://github.com/dart-lang/sdk/issues/41811)


Add blocks for inverese of log2 and log10

In Modelica.Blocks.Math there are blocks of all trigonometric functions including their inverse functions, e.g. Cos and Acos. In the same manner Log and Exp are implemented. Unfortunately, for Log2and  Log10 there do not exist inverse blocks.
Possible names of the missing blocks are

Power2
Power10

Where do you see Log2?

You're right. Log2 shall be added as well.

Would it be sufficient with a general Power block instead of a Power2, Power10 block?

Yes, this also serves the purpose and is more flexible. I am favor of implementing Power.

I don't think we should add Log2 - but I am in favor of generalizing Log and Exp (or adding Power).
One possibility would be to add a parameter base (default e to Log or 10 to Log10); another option would be to add a new block.
Similarly we should have base for Power/Exp-block. There are some minor numerical issues (i.e. whether log(e)=1 or if that will add an offset).
Having the base as an input might be possible, but seems excessive.

Adding the base is more flexible approach. I am in favor of providing a generalized Power and Log function.
I would also not go for a base input, but for a base parameter (for example, with default Modelica.Constants.e).

@christiankral Are you going to provide a pull request for this issue? Otherwise I will assign it to the library officer.

OK I will provide a pull request.

Resolved by d23413c2625bfc44c86a9a5a27192a1bb79c6767.


Let users watch for mouse-move events

Hey @ThatOtherPerson, we'd like to work with you to get this in, but since Eve 0.3 isn't really fast enough to receive a continuous slew of mouse move events, we'll need to do a more limited version. To start with, we'd like to bar this feature behind an #html/listener/mouse-position tag (just like hover is 1 2). The handler will bubble from the target and will only dispatch to Eve if at least one of the elements in the hierarchy has that listener. Additionally, it will only attach event information about the element(s) with that listener to keep volume down.
Once we've done that we'll do some qualitative testing to see if we also need to throttle the events.
Are you interested in spearheading this?

Yes, sounds good!

@joshuafcole I've tried implementing your suggestions, let me know if there's anything more I should fix.

@joshuafcole Thanks for the feedback! The PR has been updated.

Sorry for the slow response, between writing an eve-mode for emacs, putting out some fires in 0.2, and preparing for the holidays my time has been fairly well accounted for. I should be much more available after the 4th!

No problem!

Hey @ThatOtherPerson,
Since we're approaching end of life for 0.3, I'm going to close this PR. If you happen to free up after 0.4 is released, ping me and I'd be happy to help you port your work!


Feature/copy update pricingrules

3scale copy service and 3scale update service commands add pricing rules to the list of entities to copy/update.
Needs new release of 3scale-api-ruby with the following PR merged
https://github.com/3scale/3scale-api-ruby/pull/47

ready for review @mikz @miguelsorianod

@mikz all required changes applied

One nitpick regarding the git history. I think it is a good practice to commit code and tests in the same commit. The tests testing that code should be in the same commit because it is easier to bisect and quite obvious in the history that the tests belong to some code and they are green.


[AWS Budgets Dashboard] : Create a daily graph comparing the projected budget amounts (AWS Forecated amount) with the actual daily cumulative costs and the AWS budgets.

Hi,
[Feature 1] : I check the forecasted amount values in the AWS console (AWS Budgets section) almost daily to identify any discrepancies with my budget or forecast. This feature would allow everyone on a daily basis to monitor deviations within their departments that may not qualify as obvious cost anomalies or might not even be on the radar yet, such as new workloads. It would provide everyone with greater assurance early in the month that the projected amounts closely align with the budget.
[Feature 2 ] : On top of that, and I take the great example of this video from the FinOps foundation https://www.youtube.com/watch?v=XgbbK-GEnPY&ab_channel=FinOpsFoundation 28:38min, as "future outlook" is it feasible to include a line or bar chart representing the future forecast for the upcoming month and the next quarter until the end of the year?
Thanks,
Regards,

Great feedback. We will definitely look into this.


Stereo flag is separated from the structure and each time you click on 'Layout' button zoom decreases

Steps to Reproduce

Open Ketcher
Add structure ketcher (83).zip with stereobonds on canvas
Start pressing 'Layout' button

Actual behavior
Stereo flag is separated from the structure and each time you click on the layout, the zoom decreases
Expected behavior
The structure moves to the center of the canvas when you click on the layout and remains in the center of the canvas with each subsequent click. The stereo flag stays next to the structure.
Screenshots
https://github.com/epam/ketcher/assets/62464241/4da9e6cd-8053-41cf-9402-3f1710978b4f
Desktop (please complete the following information):

OS: Windows 10
Browser Chrome
Version 112.0.5615.138 (Official Build) (64-bit)

Ketcher version
Version 2.13.0-rc.2

Tested. Bug is fixed.
https://github.com/epam/ketcher/assets/62464241/9d0a72f1-6e5b-4cbd-acda-8691ce584570


(feat) referenceTime and formats for am calendar

Backported referenceTime and formats optional params support

Any particular reason this pull request hasn't been accepted?
I did exactly this same work yesterday and was going to submit a pull request, then saw this open one (oops). It's straight forward and would be a very useful addition.

Thanks for reminding me about this PR, @arimus
Probably I had to promote it more actively =)

Please see my next comment:
https://github.com/urish/angular2-moment/pull/78#issuecomment-251622047

@irsick can you please update the PR to the latest source code? I would love it to be a part of the upcoming 1.1.0 release. Thanks!

Sure, no problem

Awesome, much appreciated!

Done

Thank you! What is the reasoning behind changing the order of the arguments?
In general, I think it is better to stay consistent with moment.js

The reason is pure practical. My team and I work a lot with dates and data conversions in our big Angular 2 project. The situation when you need to change referenceTime happens hardly ever (only once in my practice), while  output formats needs to be customized almost every time when calendar pipe is used.
I found out that having
{{myDate | amCalendar:{sameDay:'[Same Day at] h:mm A'} }} is much more natural and intuitive than
'{{myDate | amCalendar:null:{sameDay:'[Same Day at] h:mm A'} }}'
So you don't need to know the meaning of referenceTime just to customize output formats.

Thanks for the explanation @irsick!
@maggiepint Do you know the reasoning behind the order of calendar() method arguments?
i.e. why referenceTime comes before format?

Talked to Matt about this one, and basically, because we're dumb? I agree that the order you're proposing makes more sense.
I think we could probably do some type checking and make the parameter order work either way, but I'm not eager to change it outright. At this point it's breaking, and a lot of people use it.

@maggiepint thank you for your response! You aren't dumb, that's for sure :)
Is making the first argument optional (using typechecking) a feasible option?
I believe we can implement it in angular2-moment, but I would prefer to keep to implementation closely tied to what moment does...

Logged in moment/moment#3658
I may try to get to this one for the LOLZ.

Awesome @maggiepint! What are the LOLZ? (funny gifs come into mind, but it probably has another meaning in this context)
@irsick would you like to implement a check as maggie suggested to make the parameter order either way, so we stay compatible with moment?

I'd say that the reason Moment has them in a silly order is just history. referenceTime was supported first and then formats was added more recently. The easiest way to do the reverse-compat was to tack the new arguments on at the end. Probably would have been wiser to flip the order then and do type-checking for reverse compat, but we are not wise.
I agree the order is ungood. And having the first object be a Moment causes lots of confusion on its own. Moment's API doesn't use option objects, so it would be weird to add them now, but that would actually be my preferred way of passing in referenceTime.

Updated PR to support both orders of arguments.
Even if I personally agree with object argument proposed by @icambron for the sake of pipe syntax simplicity I made it based on type check by @maggiepint.

Thank you @irsick for following up with all the changes, and everyone on this thread for the valuable discussion.

Released as part of 1.1.0


⚠ Samsung has degraded performance

In f8a748b, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2098 ms

Resolved: Samsung performance has improved in 9d00163.


[REVIEW]

Resume link
https://drive.google.com/file/d/1ka2fW7cYkNVz1ind7uUI-NTn6ZXG7uad/view?usp=sharing

Reviewed on video, video to come out in a few weeks
Looks good 👍  some thoughts

needs more spacing between sections
skills section SQL and SQl needs to be consistent
VSCode and Jupter should be separate bullet points like the others, it is the only one with a ,
projects to have GitHub links


Seems like there's an import cycle

This package seems to import itself?
$ go test stream_test.go
go: downloading github.com/stretchr/testify v1.3.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/davecgh/go-spew v1.1.1
# command-line-arguments
package command-line-arguments (test)
	imports github.com/json-iterator/go
	imports github.com/json-iterator/go: import cycle not allowed
FAIL	command-line-arguments [setup failed]
FAIL

$ ./test.sh
package github.com/json-iterator/go
	imports github.com/json-iterator/go: import cycle not allowed

As I suspected, this was due to not running in the correct GOPATH. Closing.


[Documentation]

updated some sentences in more likable way

Hello @AniketShuklaG!
Let me close this PR, because I don't see the changed parts any longer present in the documentation, making this PR outdated.
Please create new PRs for other open issues!


[ENHANCEMENT] file name with .model.dart

I'm using Hive Object Converter to introduce Hive notations into models
Would be nice to automate the process and have Json to Dart let me decide the output pattern so that the other plugin can find the generated files

@Miamoto-Musashi Hi 👋
Thanks for suggestion. I don't use Hive and don't know how it's important. If you should have option to put enhancement filename it's not same as change file name manually?

Hi @iamarnas,
I'm using jsonc generator as far as I know the only attribute I can play with is __className, to make the other plugin find the models I should use __className: Address.model or similar but then I will find that class name everywhere on the code so it would be nice to have Address class generated in address.model.dart file.
Hive is quite popular in the community bye my knowledge.
I was also looking to contribute but not so confident with node

Hi @Miamoto-Musashi
By using the jsonc file is a quick fix. I will add this suggestion to the list and try to integrate it as soon as possible and all suggestions how to be possible make it simple as possible for users are welcome. Right now I work with commands subscriptions, for example it will be possible to build jsonc file models from the right mouse button context menu.

Hi @Miamoto-Musashi
I added file name enhancement to the latest version. To test it you just need to separate your class name with a dot and after the dot, everything will be added as an enhancement name. Example: className.enhancementName. Can you test and see if it works as you expected?

yes working, thank you.
very minor enhancement is the lint option, using dart linter the generated code violates sort_constructors_first rule

@Miamoto-Musashi

yes working, thank you.

Then now I can update documentation?

very minor enhancement is the lint option, using dart linter the generated code violates sort_constructors_first rule

It is a quick fix that i can add to the settings option with the next version.

probably __className should become __fileName since it is really driving the file name just for clarity

probably __className should become __fileName since it is really driving the file name just for clarity

It not really driving the only file name. The __className is a base class name that holds all JSON generated context and is used for a file name for the map name and for syntax. I thought a lot about how I would describe this key to users and become that __className most used and most familiar to beginners.

probably we can close this one?

probably we can close this one?

@Miamoto-Musashi Sure, if you not have any issues with this we can close this. I have waited until you test it properly.


How to make a flutter package with the built librust.a or .so for iOS/Android?

Now I can use v2.0.0-dev.28 to build running iOS/Android app.
I've checked https://cjycode.com/flutter_rust_bridge/manual/integrate/library/creating-libraries/dart-only and https://cjycode.com/flutter_rust_bridge/manual/integrate/library/creating-libraries/flutter-wrapper, but still can't get it to work.
I wonder how to turn it into a flutter package with the built librust.a or .so that other developers can use it without any rust related settings.

Hi, could you please point out specific errors such that I can have a look?

Hi, could you please point out specific errors / issues such that I can have a look?

How can I create flutter package with the flutter_rust_bridge? Is there any documentation on that?

Close since inactivity. Feel free to reopen if having any issues!


Fix/scr info expose error msg

Description
Part of SCR fixes -

Reduce verbose message / stack trace information emitted by graphql endpoints

Type of Change

chore: Routine tasks, maintenance, or tooling changes

How to Test
This changes do not impact any existing features. It is just changing the error output when exceptions are caught in graphql resolvers.

Test all flows
'hard code'  error exceptions and test using graphql playround to see the reduced output

Checklist
Please check all the boxes that apply to this pull request using "x":

[x ]  I have tested the changes locally and verified that they work as expected.
[ ]  I have added or updated the necessary documentation (README, API docs, etc.).
[ ]  I have added appropriate unit tests or functional tests for the changes made.
[ ]  I have followed the project's coding conventions and style guidelines.
[ ]  I have rebased my branch onto the latest commit of the main branch.
[ ]  I have squashed or reorganized my commits into logical units.
[ ]  I have added any necessary dependencies or packages to the project's build configuration.
[x ]  I have performed a self-review of my own code.
[ ]  I have read, understood and agree to the Developer Certificate of Origin below, which this project utilises.

Screenshots (if applicable)
[If the changes involve visual modifications, include screenshots or GIFs that demonstrate the changes.]
Additional Notes
[Add any additional information or context that might be relevant to reviewers.]

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
   have the right to submit it under the open source license
   indicated in the file; or

(b) The contribution is based upon previous work that, to the best
   of my knowledge, is covered under an appropriate open source
   license and I have the right under that license to submit that
   work with modifications, whether created in whole or in part
   by me, under the same open source license (unless I am
   permitted to submit under a different license), as indicated
   in the file; or

(c) The contribution was provided directly to me by some other
   person who certified (a), (b) or (c) and I have not modified
   it.

(d) I understand and agree that this project and the contribution
   are public and that a record of the contribution (including all
   personal information I submit with it, including my sign-off) is
   maintained indefinitely and may be redistributed consistent with
   this project or the open source license(s) involved.

Frontend Testing Passed:

Smoke Test For Docker Passed
Smoke Test Report for Docker.zip


Installation failure

hi , some questions had been found when i used this  command。'pip install -e .'

I guess it's here

Hello, I'm also experiencing the same issue. Have you found a quick fix or any temporary workaround that you could share?

Same issue ☝

has been resolved ，好用爱用。

Closed as a non-active issue, and it seems like it has been resolved.


Implement setClob and getClob

This change is

Reviewed 1 of 1 files at r1.
Review status: 1 of 4 files reviewed at latest revision, 1 unresolved discussion.

java/jdbc/src/main/java/io/vitess/jdbc/VitessPreparedStatement.java, line 757 at r2 (raw file):
Previously, tirsen wrote…
Tests added.

We can throw same exception with appropriate error message like 'SQL Feature Not Supported for clob size greater than<PHONE_NUMBER>.

Comments from Reviewable

java/jdbc/src/main/java/io/vitess/jdbc/VitessPreparedStatement.java, line 757 at r2 (raw file):
Previously, harshit-gangal wrote…
We can throw same exception with appropriate error message like 'SQL Feature Not Supported for clob size greater than<PHONE_NUMBER>.

Done.

Comments from Reviewable

Reviewed 1 of 1 files at r2.
Review status: 2 of 4 files reviewed at latest revision, 1 unresolved discussion.

java/jdbc/src/main/java/io/vitess/jdbc/VitessPreparedStatement.java, line 757 at r2 (raw file):
Previously, tirsen (Jon Tirsen) wrote…
Done.

Did you miss to push the changes?

Comments from Reviewable

java/jdbc/src/main/java/io/vitess/jdbc/VitessPreparedStatement.java, line 757 at r2 (raw file):
Previously, harshit-gangal wrote…
Did you miss to push the changes?

I did. Pushed!

Comments from Reviewable

Reviewed 2 of 2 files at r3, 1 of 1 files at r4.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


ER_ACCESS_DENIED_ERROR after several hours of idle or minimal usage

Environment
Knex version: 0.15.2
Database + version: MySQL 5.7.22 (on RDS)
OS: 64bit Amazon Linux/4.5.3
knex configuration:
{
    client: 'mysql',
    connection: {
        host: '...',
        port: 3306,
        user: '...',
        database: '...',
        charset: 'utf8mb4',
        dateStrings: true,
    },
    pool: {
        min: 0,
        max: 25,
        acquireConnectionTimeout: 60000
    }
}

Bug (or our own misuse)?
After an indeterminate (but usually, 11 hours or more) period of little or no usage, we start to see connections refused, with each call throwing ER_ACCESS_DENIED_ERROR: Access denied for '...'@'an-ip-address'. It isn't after the first request comes through, but typically an hour or two after activity starts picking up.
Some finer details about how we use knex:

We use Koa, and connections are attached to the context in a middleware, using getConnection(read or write db name) (declaration below).
mysqlConfiguration is a static function that returns the knex config above.
We have a write db and read replica that we have separate configurations for.
Restarting our node server fixes the issue.

const connections = {};

export function getConnection(connectionName) {
    if (!connections[connectionName]) {
        const connectionConfig = Environment.mysqlConfiguration[connectionName];

        const connection: knex = knex({
            ...connectionConfig,
            asyncStackTraces: true
        });

        connections[connectionName] = connection;
    }

    return connections[connectionName];
}

My guess is that whoever wrote this initially wanted to hold onto the connection instance from the first module resolution and have that persist for the lifetime of the application. We were using a very old version of Knex that didn't have these issues until I recently upgraded us from 0.9 to 0.15.2. My other guess is that RDS has killed some stale connections in the first instance's pool, and they're not getting pruned.
Should we not be doing this, and instead call knex(connectionConfig) for a fresh knex instance on each request?

You should use always the same knex instance which is initialized just once when node process is started (or multiple knex instances if you need to connect to multiple databases).
The configuration you pasted in the original post does not seem correct. It doesn't describe which database client you are using and pool setup should be done in upper level than connection details. However I don't think that it is the problem in your case... except for that pool setup which might mess up something.
Try to print out some debug information about how many connections there are in pool etc. Check out tarn.js documentation, how to get data about pool internals. Pool can be found from knex instance from knex.client.pool.

Also closing currently as cannot be reproduced :(

No worries, let me know if you are able to see that there indeed is situations, where RDS closes connections in a way that knex doesn't notice. Also would be good to make sure that you have latest version of mysql driver installed.

Sure thing. I may honestly opt for instance restarting until I have the bandwidth to get to the bottom of this. If it is of any use, before it starts happening, we see:
Connection Error: Error: Connection lost: The server closed the connection.
I see that warning comes from the validate function in the Tarn config. Since it's happening in validate, that would mean that the ER_ACCESS_DENIED_ERROR is occurring after acquiring new connections, not because of stale ones, right?

Since it's happening in validate, that would mean that the ER_ACCESS_DENIED_ERROR is occurring after acquiring new connections, not because of stale ones, right?

Yes, sounds like old connections are disposed just fine, but new ones cannot be created anymore...  but that error is coming from database server so I have no idea why it could stop accepting old credentials or maybe that error code can mean something else too.
You can also monitor connection count etc. from RDS while figuring it out. Other old trick has been to add debug printouts to nearby the places where errors are thrown directly to a code inside node_modules, for example nearby this /var/app/current/node_modules/knex/lib/util/make-knex.js:40:19.

@elhigu I wanted to mention that I see this in the mysql logs right before this happens. Additionally, it's not all connections in the pool that fail, just some.
2018-12-08T01:32:19.402149Z 2360156 [Note] Access denied for user 'socialnative'@'REDACTED_2' (using password: YES)
2018-12-08T01:32:19.404563Z 2360157 [Note] Access denied for user 'socialnative'@'REDACTED_2' (using password: YES)
2018-12-08T01:32:34.824238Z 2358993 [Note] Aborted connection 2358993 to db: 'DB_NAME_REDACTED' user: 'socialnative' host: 'REDACTED_2' (Got an error reading communication packets)
2018-12-08T01:32:34.824380Z 2360027 [Note] Aborted connection 2360027 to db: 'DB_NAME_REDACTED' user: 'socialnative' host: 'REDACTED_2' (Got an error reading communication packets)
2018-12-08T01:37:41.369732Z 2360694 [Warning] IP address 'REDACTED_1' could not be resolved: Name or service not known

It looks like an issue where DNS resolution fails (which isn't extraordinary in my opinion) but the pool isn't discarding the connection mysql aborted. It continues to try to run queries on it and gets back ER_ACCESS_DENIED_ERROR and that bubbles up to our application. I also tweaked our max connections to 15, 25, and 50 and still saw this.
I also ought to remark that I saw this after a hefty knex upgrade. We began seeing this issue after going from 0.9 to 0.15.2, which makes me think it's a regression -- it's just too coincidental to me not to be.
Finally, I'm sorry if I'm being a pest here. :/

Does all errors go away if you replace REDACTED_1 / REDACTED_2 with real IP instead of DNS name or if you add IP:s for those hosts to be resolved locally by application server i.e. in /etc/hosts?
That sounds like a bug somewhere if connections are not discarded from pool in case of DNS error... can that be reproduced by just trying to connect to some invalid DNS name? If that can be reproduced, it will be easy to debug where the problem lies.

They are real IPs, I think MySQL does the reverse lookup regardless if skip_name_resolve is false.
So far, I've been getting around this by terminating the process and restarting it, but I think it can be reproduced by throttling the mysql host's dns server.
I can't really edit /etc/hosts because I'm spinning up new hosts pretty often.
I think I can also work around by setting skip_name_resolve to true. I'm fine without having that check.

I can't really edit /etc/hosts because I'm spinning up new hosts pretty often.

I meant to just try it out if it that actually fixes the problem, it could be caused by that DNS server problem... but if it is because of reverse DNS resolution I'm not sure if /etc/hosts is even used for that.
Sounds pretty far fetched that reverse lookup would be causing the problem, mysql should do it only once per IP address https://dev.mysql.com/doc/refman/8.0/en/host-cache.html

... I think it can be reproduced by throttling the mysql host's dns server

Anyways let me know if you are able to reproduce it. Would be interesting to see if mysql driver doesn't notify about broken connection  in that scenario and knex pooling really breaks.

Thanks for the guidance, I really appreciate it. I'll try to reproduce this and will report back here. Something absolutely strange I'm seeing is that our host_cache table is empty?? That's probably why it's doing reverse lookups up all the time. Will be a fun little adventure figuring that one out!


Problem with infinity symbol

Renders as:
"D:/example//Library/Godspeed You! Black Emperor/F#A# \x1e/"

(from F♯ A♯ ∞, leaving this note so i can copy/paste the infinity easily)

"taglib_set_strings_unicode(false)" uses latin-1 encoding, which doesn't contain the infinity symbol.

i don't think there's any way around this, unfortunately. just going to copy those files manually.

this likely applies to other alphabets as well. yikes.

I fixed it! the crux of the c++ taglib bindings issue wasn't linking, it was a "typedef int BOOL" statement in minwindef.h. I commented it out, in an act of sheer and desperate hubris.

turns out the minwindef conflict was because i was linking/including both the c and c++ bindings. classic.


dequeue or dequeue_many will fail immediately after close the queue

After close the queue, subsequent dequeue and dequeue_many operations that won't block waiting for more elements to be enqueued and will fail immediately.

Can one of the admins verify this patch?

Jenkins, test this please.


Python client

Don't pay too much attention to this just yet - things are still very broken, and subject to complete change

🐍

I'm excited to see some activity on this. Looking forward to seeing some example scripts and then I'll try to play around as soon as I have some time.

Let's close this to keep all our open PRs on a single page!

Let's close this to keep all our open PRs on a single page!

I don't see what you are referring to, the python client never got created/merged, can you please direct me to it?

Indeed, we would also like to know where to look at progress on this.


Amplify build failed , There was an issue connecting to your repo provider , is showing even after reconfiguring with github repo

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

App Id
demayssxwivxw
AWS Region
us-east-1
Amplify Hosting feature
Build settings
Describe the bug
So my amplify frontend is connected to a github repo . And it has been going well till now. Right now we are getting build failure , and it's showing the error There was an issue connecting to your repo provider, click "Reconnect repository" in General Settings, and then try your build again. I have tried resolving it and reconnecting it and other solutions, but still build failure.
Unable to get the updates live
Expected behavior
Should be able to build the amplify project , as it was working fine till now
Reproduction steps
1.Build the app
2. After provisioning, build error
Build Settings
No response
Log output

# Put your logs below this line

2022-12-27T13:17:11.197Z [INFO]: # Cloning repository: git@github.com:eslamstech/eslamsplatformv1.git
2022-12-27T13:17:13.437Z [INFO]:
2022-12-27T13:17:13.439Z [INFO]: Cloning into 'eslamsplatformv1'...
2022-12-27T13:17:13.440Z [INFO]: # Checking for Git submodules at: /codebuild/output/src865559917/src/eslamsplatformv1/.gitmodules
2022-12-27T13:17:13.451Z [INFO]: # Retrieving environment cache...
2022-12-27T13:17:13.496Z [WARNING]: ! Unable to write cache: {"code":"ERR_BAD_REQUEST","message":"Request failed with status code 404"})}
2022-12-27T13:17:13.496Z [INFO]: ---- Setting Up SSM Secrets ----
2022-12-27T13:17:13.496Z [INFO]: SSM params {"Path":"/amplify/demayssxwivxw/main/","WithDecryption":true}
2022-12-27T13:17:13.549Z [INFO]: # Defaulting to Node version 16
2022-12-27T13:17:23.874Z [INFO]: # Node version 16 is available for installation
2022-12-27T13:17:24.034Z [INFO]: # Installing Node version 16
2022-12-27T13:17:33.507Z [INFO]: # Now using Node version 16
2022-12-27T13:17:34.493Z [WARNING]: Cannot find module<EMAIL_ADDRESS>                                    Require stack:
                                    - /snapshot/AWSMobilePioneerExecute/out/index.js
2022-12-27T13:17:34.493Z [INFO]: # Patching NPM package '@aws-amplify/cli' to 10.5.2...
2022-12-27T13:17:41.944Z [INFO]: # Done patching NPM package '@aws-amplify/cli' to version 10.5.2
2022-12-27T13:17:42.029Z [INFO]: # Retrieving cache...
2022-12-27T13:17:42.068Z [INFO]: # Retrieved cache
2022-12-27T13:17:52.236Z [INFO]: ## Starting Backend Build
                                 # Starting phase: build
2022-12-27T13:17:54.468Z [INFO]: Amplify AppID found: demayssxwivxw. Amplify App name is: eSlamsPlatformv1
2022-12-27T13:17:54.518Z [INFO]: Backend environment main found in Amplify Console app: eSlamsPlatformv1
2022-12-27T13:17:55.710Z [WARNING]: ✖ There was an error initializing your environment.
2022-12-27T13:17:55.728Z [INFO]: 🛑 Failed to pull the backend.
2022-12-27T13:17:55.728Z [INFO]: Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
2022-12-27T13:17:55.730Z [INFO]:
2022-12-27T13:17:55.730Z [INFO]: Session Identifier: 6afdcb1a-85ac-45a8-be2a-b162d8cecf62
2022-12-27T13:17:55.819Z [ERROR]: !!! Build failed
2022-12-27T13:17:55.820Z [ERROR]: !!! Non-Zero Exit Code detected
2022-12-27T13:17:55.820Z [INFO]: # Starting environment caching...
2022-12-27T13:17:55.820Z [INFO]: # Environment caching completed
Terminating logging...

Additional information
No response

Hi @roshomon420 👋 , thanks for reaching out to us and apologies for the delayed response on this thread.
Are you still running into these backend build errors? If yes, can you please set the AMPLIFY_ENABLE_DEBUG_OUTPUT environment variable to true for your Amplify app demayssxwivxw ? It should print a stack trace in the logs which could be helpful in debugging the error.
You can refer to this documentation to set an environment variable in the Amplify Console: https://docs.aws.amazon.com/amplify/latest/userguide/environment-variables.html#setting-env-vars


feat(home): add cookies dialog

closes #188

This popup should appear on all pages not only on home. User shouldn't be able to access his profile without accepting cookies policy.


Mysterious "'http'" printed for unknown reason

ArcRest or ArcRestHelper
arcrest.manageorg._portals.Portal.featureServers()
Version or date of download
Either of:

"master" branch of https://github.com/Esri/ArcREST.git
"ArcREST_Package 3.5.6" from https://pypi.python.org/pypi/ArcREST_Package

Bug or Enhancement
🐛 Bug: When running my program, the message 'http' appears in the output with no apparent source.
Repo Steps or Enhancement details
Found when running the "Fetch your folders" example from the Getting Started section of README.md, found after applying bug fix from #281.

I opened PR #283 with a fix for this issue.


Bug in ARC Length Calculator not getting answer

Describe the bug
After giving input in Arc Length Calculator it is showing all values are required to obtain answer
Screenshots

Hi, I want to work on this issue I will start working on it as soon as I get assigned!! I am a part of GSSoC'21.
Kindly assign this issue to me!
Discord Username : Abhijeet Sinha(P)
Discord Tag: #4018


onMonthChange event - backwards or forwards?

Hello,
Is there a handy way to indicate whether we are paginating forwards or backwards when the onMonthChange() event is triggered?
Thanks,
Jon

In "onMonthChange" function you have the instance to check currentMonth:
onMonthChange: (selectedDates, dateStr, instance) => {
console.log(instance.currentMonth)
}


Support for External Sensor for Optimus Jobs

Currently Optimus Supports sensors for job dependencies which are within and the outside the project but they are managed by the same Optimus Server.  It would be helpful if Optimus supports job sensors which are managed in a different Optimus, as with in an organisation there will be many deployments, checking for data availability may not always guarantee completeness & correctness of data which is guaranteed through Optimus dependencies.
Expectation :
The sensor provides checks for the status of the jobs b/w the input window.
Configuration :
dependencies :
 job :
 type : external
 project :
 host :
 start_time : // start time of the data that the job depends on
 end_time : // end time of the data that the job depends on.

The Optimus Server which accepts the requests based on its window, schedule configuration checks for all the jobs which outputs the data for the given window
This has the challenge of breaking the dependencies when job name changes.

Just wanted to mention that we design this looking from a perspective for supporting generic user-defined sensors as well like an HTTP sensor waiting for 200.

yeah makes sense

http sensor config proposal at job:
dependencies :
 - sensor :
     type : http
     param :
      method : POST
      url : https://optimus-host:80/serve/
      headers:
       - Content-type : application/json
       - Authentication : Token
      body : '{example: {"param-name-1": "value-1", "param-name-2": "value-2"}}'

Config looks good. Do we have a clear segregation between what is a pre-hook and what is a sensor?

I want to simplify the sensor config as normally http sensors are just HTTP GET type and wait for success response.
Even airflow supports only GET method for HTTP sensor. Please check the source code here
updated proposal for http sensor :
dependencies :
 - http :
     request-params:
       key-1 : value-1
       key-2 : value-2
     url : https://optimus-host:80/serve/
     headers:
        Content-type : application/json
        Authentication : Token

corresponding at go struct will be :
type JobDependency struct {
	JobName string `yaml:"job"`
	Type    string `yaml:"type,omitempty"`
	HttpSensor  Http `yaml:"http,omitempty"`
}

type Http struct {
	RequestParams map[string]string `yaml:"request-params"`
	URL     string `yaml:"url"`
	Headers map[string]string `yaml:"headers"`
}

we are going configure sensor under dependencies section. I suppose, pre-hook dependencies are defined at plugin level under pluginInfo

proposal-2 :
put extenal-dependencies at root
external-dependencies :
 http :
    -
     Name : http-sensor-1
     request-params:
       key-1 : value-1
       key-2 : value-2
     url : https://optimus-host:80/serve/1/
     headers:
        Content-type : application/json
        Authentication : Token-1
    -
     Name : http-sensor-2
     request-params:
       key-1 : value-3
       key-2 : value-4
     url : https://optimus-host:80/serve/2/
     headers:
        Content-type : application/json
        Authentication : Token-2

corresponding proton changes:

https://github.com/odpf/proton/pull/100

corresponding optimus changes:

https://github.com/odpf/optimus/pull/150

JSON Sample Structure which will be stored at DB :
{
   "HTTPDependencies":[
      {
         "Name":"http_sample_dependency_1",
         "RequestParams":{
            "request-param-1":"value-1"
         },
         "URL":"http://sample/dependency/1",
         "Headers":{
            "Content-Type":"application/json"
         }
      },
      {
         "Name":"http_sample_dependency_2",
         "RequestParams":{
            "request-param-2":"value-2"
         },
         "URL":"http://sample/dependency/2",
         "Headers":{
            "Content-Type":"application/json"
         }
      }
   ]
}

tested with the defined struct here
tested with future dependency here

The new HTTP schema looks good. Any reason to extract external dependencies to root?
Also, should we care about the response code? What if user wants to treat 200, 201 response code as a success parameter of the sensor or even 404?

Any reason to extract external dependencies to root?

decouple new external dependencies from existing job dependency section as dependency resolution is not needed for sensors

Should we care about the response code? What if user wants to treat 200, 201 response code as a success parameter of the sensor or even 404?

No, We shouldn't.

few more reasons why we pulled out external dependencies out

We just want users to allow only one dependency to be configured in an array element, if we keep supporting http & other dependencies in the same place, users can configure all or few dependencies in order to avoid that we need to rely on type param, which is a duplication for user and the logic behind the scenes will follow the same. Keeping these tradeoffs and the one which @siddhanta-rath mentioned I believe it is better to pull out external dependencies.

On the http sensor, for now we can just go with supporting 200+ as sensor success, later we can consider providing more flexibility if needed to the users.

We just want users to allow only one dependency to be configured in an array element, if we keep supporting http & other dependencies in the same place, users can configure all or few dependencies

A job can have an external dependency and an internal dependency right? Why can't we configure both? Also, I think type param can be removed and we can easily figure out what to do with the dependency based on what the user has provided. Correct me if I am wrong.

dependency resolution is not needed for sensors

This is an internal technical thing that we can sort out without affecting users? They don't have to worry about one more root config?

How can we avoid users configuring multiple dependencies of various types without type field in a single array element? @kushsharma
Ideally I would expect a single root element for dependenices by having a structure where the array of job dependencies parallel to array of http and GCS dependencies, it is just that the change isn't backward compatible.

@kushsharma I think need here is across two Optimus deployments and not within the same Optimus deployment and two projects within that. But I think we can achieve that also without root level and specifying extra params/cinfigs for across deployments one.

@ravisuhag yes, I just gave a generic example, in case of optimus-to-optimus this could be from top of my head
dependencies:
  - job: sample_internal_job
  - job: cross_project/sample_external_job
  - http:
      name : sensorName
      query:
        project: p1
        job: j1
      url : https://optimus-host:80/path-for-status-check

we wanted to group a specific type dependencies at yaml, so it will be intuitive for user to where to put what?
yaml file will also look much cleaner.
problem with older approach:
User can give input like this :
dependencies:
 - job: job_name_1
 - job: job_name_2
 - http :
    name : sensorName_1
    request-params:
      key-1 : value-1
      key-2 : value-2
    url : https://optimus-host:80/serve/1
    headers:
      Content-type : application/json
      Authentication : Token
 - job: job_name_3
 - job: job_name_4
 - http :
    name : sensorName_2
    request-params:
      key-1 : value-1
      key-2 : value-2
    url : https://optimus-host:80/serve/2
    headers:
      Content-type : application/json
      Authentication : Token
 - job: job_name_5
 - job: job_name_6
 - gcs:
    path: gcs://path
    service-account: account_details
 - http :
    name : sensorName_3
    request-params:
      key-1 : value-1
      key-2 : value-2
    url : https://optimus-host:80/serve/3
    headers:
      Content-type : application/json
      Authentication : Token

With new approach, we are grouping http dependencies under http and
will group gcs dependencies under gcs tag like below :
external-dependencies :
 http :
    -
     Name : http_sensor_1
     request-params:
       key-1 : value_1
       key-2 : value_2
     url : https://httpbin.org/get
     headers:
        Content-type : application/json
        Authentication : Token_1
    -
     Name : http_sensor_2
     request-params:
       key-1 : value_3
       key-2 : value_4
     url : https://httpbin.org/get
     headers:
        Content-type : application/json
        Authentication : Token_2

gcs :
    -
     path: gcs://path_1
     service-account: account_details_1
    -
     path: gcs://path
     service-account: account_details_2

@sravankorumilli correct me if i am wrong

@siddhanta-rath I think the learning curve will increase this creating multiple nodes of dependencies and two places whee we define them.
I think all dependencies grouped together will be better. We can think about what is the best way to structure them.

Okay, lets put all the dependencies under single root dependencies and will move back to eariler proposal-1 as mentioned on the previous comment:
https://github.com/odpf/optimus/issues/138#issuecomment-1034889824

aligned on that @kushsharma @ravisuhag , we will move everything under dependencies  thank you for the feedback. Somehow in our discussions we are misaligned that if we go with that option we will break the compatibility.

we are not getting rid of type at the moment will keep it for now, will remove it as part of seperate refactorying exercise but we don't need that any longer.

Nice. Can we also rename request-params to just query or queries or params? Request is kind of implicit here.

Sure. Lets change it to params


Inputs outputs wrapper

Tracking issue
https://github.com/flyteorg/flyte/issues/4654
Why are the changes needed?
Allow flytekit to pass additional information in outputs that can be propagated to the UI/flyteadmin and used in subsequent tasks..
What changes were proposed in this pull request?

How was this patch tested?

Setup process
Screenshots
Check all the applicable boxes

[ ] I updated the documentation accordingly.
[ ] All new and existing tests passed.
[ ] All commits are signed-off.

Related PRs

Docs link

i think this PR needs a few rounds of testing

I don't think it's complete, I agree... I only tested inputs/outputs for an execution... if you have scenarios in mind for testing, that would help a lot..


added Tririga Buildings flows

Added Flows along with OM Package to main branch. Should have merged this one first- my mistake


runtypes returns integer out of range - for number size greater than int (2^32)

I am getting this error: "integer out of range"
for validating a property as a number for this number for example:
6628412528.
it is bigger than 2^32 - 4 kb:
2147483647
but way smaller than Number.MAX_SAFE_INTEGER:
9007199254740991
Please support the value of MAX_SAFE_INTEGER,
or for now please help me find a work around for processing those numbers
Thanks!
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

Hi, I can't reproduce the error. Could you elaborate on it as some example code or repo? I dare say our codebase have no chance to emit such error text, as there's no built-in runtype called Integer or some.

correct, sorry my bad it returned from the db layer.
closing, thanks!


DO NOT MERGE - Update gsheets.md

Google Sheets updates
Merge status: Do not merge this PR. Waiting on PM to give merge go ahead as they're still working through things.
What are you changing in this pull request and why?

Checklist

[ ] Review the Content style guide so my content adheres to these guidelines.
[ ] For docs versioning, review how to version a whole page and version a block of content.
[ ] Add a checklist item for anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

Adding or removing pages (delete if not applicable):

[ ] Add/remove page in website/sidebars.js
[ ] Provide a unique filename for new pages
[ ] Add an entry for deleted pages in website/vercel.json
[ ] Run link testing locally with npm run build to update the links that point to deleted pages

@mirnawong1  this can be released first thing tomorrow!


Font Awesome fixed width icons not rendering

Example of source
Icon 1: icon:magic[]
Icon 2: icon:magic[fw]

In the above example only the first icon is generated. The second one, with fw (fixed width) set, displays no icon at all.
This doesn't look to be an intentional behavior. If fixed width cannot be applied in PDF rendering the flag should be ignored.

I've proposed a more complete solution. You can test it using the following:
:icons: font

a icon:gitlab[fw] b
a icon:bolt[fw] b

a and b should be aligned.


Add API to report all descriptors matching a tag(s)

i.e. the tags 'ecp' + 'idp' would list all ECP enabled IdP, detail could be kept light, enough descriptive information to display a basic record to an end user and allow client code to appropriately reference the full record in metadata.

No longer required, see MDQP impl.
Further API design would be undertaken as a future package of work


[Podspec] Make Core subspec use a Ruby string array for exclude_files

For consistency (and maybe correctness?) use an array instead of a string with spaces in it for the exclude_paths value in the Podspec.

@facebook-github-bot shipit


Provide markup examples and illustrations within elementSpec

Adding this from closed issue in music-encoding/music-encoding, cf. https://github.com/music-encoding/music-encoding/issues/222

As mentioned in music-encoding/music-encoding#222 and obvious from the guidelines repo being separated from the schema development it is questionable whether to include code examples in the schema definition, nevertheless the guidelines and/or specs should provide examples and potentially even more than in the text-flow.
Maybe the best way would be to run a query of all examples in the guidelines as well as over all https://github.com/music-encoding/sample-encodings
and display the results at the end of the respective element spec


Tileset-Editor: Added alternative tool for drawing a rectangular shape.

Ex:
Current tool:

New tool with snap enabled

New tool with free design

Hum, i like this feature, but i'm not sure if having 2 ways of doing the same thing is good, ot ar least i'm not sure if having 2 buttons exposed is good. @akien-mga @reduz I'd like the opinion of a UX expert. Maybe the rectangle tool could have this behavior by default and fill the entire rect if holding a button...
In any case code is good

Actually having two tools is superfluous, I merged the behavior of both (with a click without dragging you get the old behavior).
Let me know if it's ok :)

thank you very much!


Unity Readme Example Fixes

The readme examples for Unity has a couple issues as far as I can tell:
https://github.com/labstreaminglayer/liblsl-Csharp/blob/master/README-Unity.md
I am using Unity 2020.3.10f1

LSLInput calls LSL.resolve_stream around line 19 but it can not find it.  It should be LSL.LSL.resolve_stream as Unity seems to be getting confused with the namespace and class being the same (which probably should changed).

LSLOutput references liblsl.channel_format_t.cf_float32 but it can not find it.  If you remove liblsl from the start then it works.

This is more of a best practices than an actual error but LSLOutput calls FixedUpdate and has a comment that says it is called once per frame. This is not true. FixedUpdate gets called at a fixed interval regardless of framerate. Typically 0.02ms.  Perhaps that is ideal as framerate can fluctuate, but it is also dangerous as it could make the framerate behavior erratically if too much processing is put within FixedUpdate. Most likely this should be changed to Update or if it is intentional then the comment should be updated to reflect FixedUpdate's behavior.

LSLInput calls LSL.resolve_stream around line 19 but it can not find it. It should be LSL.LSL.resolve_stream as Unity seems to be getting confused with the namespace and class being the same (which probably should changed).

Yes, the namespace changed at some point from the poorly named LSL to the even more poorly named LSL.LSL. This didn't get reflected in this README.

LSLOutput references liblsl.channel_format_t.cf_float32 but it can not find it. If you remove liblsl from the start then it works.

I don't know how the liblsl namespace ever got in there.

This is more of a best practices than an actual error but LSLOutput calls FixedUpdate and has a comment that says it is called once per frame. This is not true. FixedUpdate gets called at a fixed interval regardless of framerate. Typically 0.02ms. Perhaps that is ideal as framerate can fluctuate, but it is also dangerous as it could make the framerate behavior erratically if too much processing is put within FixedUpdate. Most likely this should be changed to Update or if it is intentional then the comment should be updated to reflect FixedUpdate's behavior.

Thank you for this insight.
I will take this opportunity to once again express my concern that this C# wrapper is not meeting proper standards for style and technique according to Microsoft's guidelines. I am currently (and slowly) developing a new wrapper(https://github.com/Diademics-Pty-Ltd/LSL-dotnet) which I believe is more idiomatic, takes advantage of the latest C# advancements, adheres to Microsoft's naming and style conventions, and has way better names for the static method namespaces. If you care to take a look, I'd be interested in your feedback.

I will take this opportunity to once again express my concern that this C# wrapper is not meeting proper standards for style and technique according to Microsoft's guidelines.

PRs welcome!
The discussion in #18 seemed like it was going somewhere.
I don't mind drastically changing the API of liblsl-Csharp. We can increment the major version. I think Unity users will be the only ones who are annoyed (little benefit to them compared to native C# users).
My main motivation is that I don't want to spend eternity answering questions about namespace issues, so if there's a better long-lasting API then I'm all for it.
Just note that it shouldn't be targeting "only windows". It also has to work in e.g. Unity on iOS and other Mono targets.


BT HFP: Add AT+NREC=0 command for disabling AG echo cancellation. (IDFGH-1385)

For testing HFP Client in the case where audio is routed via HCI, it is practical to route the incoming audio from the AG to the outgoing audio via a buffer. However, if the AG has echo cancellation (EC) enabled, the EC will suppress the audio from the HF.
This PR adds an API to use the AT+NREC command for disabling echo cancellation in the AG.
This is probably also needed if the HF has it's own echo canceller.

Please ignore the .gitmodules change. It has already been submitted as a separate PR.
It is a bit tedious to remove from the branch :-/

@nifomorph Thanks for the contribution and sorry for the slow turnaround. We have looked into this PR. Thanks.

@nifomorph Thanks for the contribution. And the PR have been merged into master.
Refer to commit 75578300f1e11d79d496434f7c394a76b20bb22b & 9f145ff165099f32b92b7ee022a1debc5f0a583d(merged commit)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 All committers have signed the CLA.

Thanks for contribution.


camel-jira extension:more polish up

may be better if you give the pr job the time to verify the pr

may be better if you give the pr job the time to verify the pr

@lburgazzoli , sure for next time

Yes, there is no hurry. Letting the CI pass is the minimal requirement, giving others reasonable time frame to give feedback would be even better.


Is there anyway to work the dev items? When used in creative I get the “not worthy” msg when attempting to use any of the items.

Not really. These items are restricted to IC developers only.


When starts, the executables suddenly shutdown.

No errors, it simply boot up and disappear.
There are some logs that i can read?

did you follow the instruction in README(aka prepare a legacy d2 v1.13c)?

I have a D2 + LOD installation on the same machine.
But i have version 1.14d ( actually it runs on battle.net ).
Is the version 1.13c the only one which works?

yeah, I think only 1.13a-1.13c is supported.
actually I can try to support v1.10 to v.12, but not for 1.14, coz not all required function addresses are revealed in 1.14.

Thank you, got it!


Rewrite Readme in example directory

The table was mostly useless, incomplete and with dead links, so I re-wrote it (and introduced some categories)
Closes #471

While going through the examples, I noticed a few things:
inputs.rs - does not render different kinds of input. Is that intentional?
framework_benchmark - blazing slow (i have to wait seconds for the rows to be inserted). is that normal?
order.rs - purpose unclear despite comment
rsx_compile_fail.rs - confusing name, comment and purpose
svg.rs - doesn't appear to work
svg_basic.rs - doesn't appear to work

Also, the core_reference folder contains a bunch of duplicates, and cannot be run. While the documentation states that this is intentional, this makes it seem kinda useless, both for users learning Dioxus and for testability. I propose to get rid of this and move any useful examples out of there. This will also ensure that they compile during tests. Thoughts on this?

Thank you so much ❤
This has just been one of those things that felt like a chore but brought down the overall polish.

inputs.rs is a messy file and was added a long time ago to make sure all of the possible input types are covered. It really needs to be a test but I don't know how to test the webview properly
framework_benchmark is significantly faster in release mode, but still slower than its WASM counterpart. The overhead of the webview is relatively high and we're pretty naive with serialization/deserialization on that boundary.
order.rs should removed. It exists for a similar purpose to inputs.rs as an adhoc test
I'm not sure what's up the svg examples. There's been some changes to the interpreter recently that might've broken them. They definitely worked for the 0.2.4 release.

The core reference folder should really be merged into the reference mdbook, especially since a number of them are meant not to compile.


PDP cache (redis) is not updated for group operations

When a user is assigned to a new group, all cache entries related to this user should be forcibly expired, or else a user calling an endpoint will be subject to the old set of policies for that "hot" endpoint.
Steps to reproduce

create a user with default "permit" policy for an endpoint
use the endpoint; User should have access to it
assign the user to an existing group that denies access to the endpoint
retrieve a new token for the user (this may be an issue of its own)
use the endpoint again; User should not have access to it
verify that user is still able to access the endpoint

Steps to circumvent

login to redis-cli
issue del PDP;[userid];[http method];[endpoint]

Ticket analisys meeting (May 29th): To check.

The described scenario is not reproducible nowadays, because there is a limitation that makes group/profile unchangeable after user creation, so it`s impossible switch from one group to another and test a possible cache failure.


Repo sync for protected CLA branch

The pull request is created from main637848793749402376 to main to fix git push error for protected CLA branch

@opbld15 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Improve CloudFront cache hit ratio

Issue #, if available:

https://github.com/aws-solutions/serverless-image-handler/issues/304
Description of changes:

There's no need to include in the cache key the origin and accept http headers if AutoWebP is disabled.
This improves the cache hit ratio if AutoWebP is not used.
Checklist

[x] :wave: I have run the unit tests, and all unit tests have passed.
[ ] :warning: This pull request might incur a breaking change.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@guidev thanks for your contribution. Could you please elaborate the reason why

There's no need to include in the cache key the origin and accept http headers if AutoWebP is disabled.

but when AutoWebP is enabled the cache key needs the origin and accept header?
Also, could you please add unit tests to cover the changes?

Correct me if I am wrong.
ServerlessImageHandler solution code, aims to create, via CDK, a "final distributable template" that will be configured later by using CfnParameter(s) (e.g. autoWebPParameter).
The code provided by this PR is making decisions at CDK runtime based on a value that is unknow during cdk runtime.
In detail:
props.autoWebP is defined on serverless-image-stack.ts as autoWebPParameter.valueAsString, which in turn is finally translated on the generated template (json flavor) as "Ref": "AutoWebPParameter".
It won't contain the "Yes" or "No" value at CDK runtime.
On the other hand, "origin" has nothing to do with WebP's automatic conversion.
--
I am sending another PR based on cloudformation's "Fn::If" intrinsic function, that will be evaluated by cloudformation.

@fvsnippets You're right

The code provided by this PR is making decisions at CDK synthesis time, based on a value that is unknow during cdk runtime.

@fvsnippets You're right, let's delete this pull request


docs(rfc): Independent compute release flow

Related to https://github.com/neondatabase/cloud/issues/11698

Should we adjust the compute pool logic for the proposed release changes? Probably not, but asking just in case

For now, we shouldn't. With this v1 release flow it's expected that we will start all new computes in the pool with a new version, while we still re-utilize old pre-created ones, if there is a shortage of new computes. Later, yes, we would need to teach pools to maintain pools with both versions if we want kinda canary deployments

especially important once we do slow rollout strategies where pools will need to maintain pre-warmed computes for both old and new image for the duration of the rollout

I didn't want this to be part of v1 because it requires a lot of changes on the control plane side, but I briefly mentioned that in the Further work section https://github.com/neondatabase/neon/pull/8881/files#diff-39df61b534dc5f736661bd5f139140b573a0f35c32524866e0339e425b9f22e4R299

extension stuff

Yeah, I mentioned that too, but waiting for the Anastasia's input here, as I don't know this part well enough yet https://github.com/neondatabase/neon/pull/8881#discussion_r1746008089. I only know that we need some metadata in cplane and compute to make remote extensions working, but don't know how exactly it's produced


disabling yarn gpg in Travis to fix hanging

disabling yarn gpg in Travis to fix windows hanging
as per  https://travis-ci.community/t/build-doesnt-finish-after-completing-tests/288/9
Oct '18
@AviVahl
Checked it without my before_install. Still got stuck. It appears the yarn installation (and pgp verification) occurs anyway, because my repo contains a yarn.lock.
The repo is setup using yarn workspaces, so I have no proper matching alternative in npm land yet. :frowning:
EDIT:
tried setting env variable YARN_GPG=no (via [Travis's] configuration) to force no gpg validation, but the container appears to be stuck on: Worker information
I’ve used secure env tokens (via Travis config- “hide from log”), which seems to be broken with Windows containers. Trying regular env var to see if [the] behavior is different.
EDIT:
Yes! It was the gpg that install.sh of yarn runs. Build now passes, after setting the YARN_GPG=no environment variable.

and as per : https://travis-ci.community/t/build-doesnt-finish-after-completing-tests/288/13
Oct '18
josh (@joshk) Josh Kalderimis
Travis CI Staff (Head of Product at @travis-ci)
Hi @chrisdothtml
Could you please try adding YARN_GPG=no to your .travis.yml config to see if this helps?

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: luxcium sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.