Remove proxy when subcluster removed

When you remove the subcluster from the vdb, its pods will get deleted after the subcluster is removed. The client proxy becomes useless and must be removed too together with the configMap.
Also, when a subcluster size is scaled down to 0(e.g. stop_subcluster), we should scale down proxy size to 0 as well. And this should be done after stop_subcluster(all connections are draining down).
TODO: Update e2e test case

We don't need to manually delete config maps or deployments in the operator. We should let subcluster sts own proxy deployment, and proxy deployment own config map. So when the subcluster is deleted, the proxy pod and its config map will be auto deleted.

Creating, updating and deleting k8s objects is generally done in obj_reconciler.go. This can be simpler, in obj_reconciler, just before a subcluster's statefulset is deleted, delete its client proxy. Also in obj_reconciler, there is a place where the sts is updated, all you have to do is to check the sts size and if it is zero, set the client proxy deployment size to zero too, and update it.
Let me know if you have any confusion.

For the scale down to 0 part, I think I found the right place to update the client proxy size to 0. But for remove subcluster part, is it under method "checkForDeletedSubcluster"?
Also, do we need to worry about the order of the reconcile call? For example, when I removed the subcluster sts in vdb, the dbremovesubcluster_reconciler and obj_reconciler will be called at same time, or in different order?

We don't need to manually delete config maps or deployments in the operator. We should let subcluster sts own proxy deployment, and proxy deployment own config map. So when the subcluster is deleted, the proxy pod and its config map will be auto deleted.

Based on the existing structure, I see there is proxy lives inside the subcluster already in verticadb_types.go. So if the subcluster is deleted, does that mean the proxy pod will be deleted as well? Do we need to add logic for this part?

Creating, updating and deleting k8s objects is generally done in obj_reconciler.go. This can be simpler, in obj_reconciler, just before a subcluster's statefulset is deleted, delete its client proxy. Also in obj_reconciler, there is a place where the sts is updated, all you have to do is to check the sts size and if it is zero, set the client proxy deployment size to zero too, and update it.
Let me know if you have any confusion.

For the scale down to 0 part, I think I found the right place to update the client proxy size to 0. But for remove subcluster part, is it under method "checkForDeletedSubcluster"? Also, do we need to worry about the order of the reconcile call? For example, when I removed the subcluster sts in vdb, the dbremovesubcluster_reconciler and obj_reconciler will be called at same time, or in different order?

Both scale down and deployment should occur in obj_reconciler. The reconcilers are never run at the same time, the order is what you see in verticadb_controller. But if you look closely in obj_reconciler, you will see we never delete a sts or scale it down before remove_subcluster/remove_node.
Let's talk tomorrow, I will explain it.

Adding Qin's e2e test commits in https://github.com/vertica/vertica-kubernetes/pull/980/commits/c5b4b9cb2eabd5cc9bda4b5213c8fd7073da9e22

You can take it from here. please add more unit tests.

@cchen-vertica, having the sts own the deployment has a limitation which is proxy pods can't trigger the controller, so if a proxy pod/deployment is deleted, as they aren't owned by vdb, nothing will happen on the operator, so if there is any reconciler that depends on the proxy, it won't be run. Example: if you delete proxy pod, a new one will be created but the routing label won't be added because it won't trigger the controller.
I decided to explicitly delete the proxy dep and configmap when a subcluster is removed.

@LiboYu2 Can you also review this PR?


Adding provenanceInformation

Each NormalisedEvent should have an additional field called provenanceInformation which contains:

feedUrl: the URL of the feed the event came from
publisherName: the name of the publisher of the feed the event came from
derivedFrom: a list of URLs containing the id of the original event, plus the id of the original event's parent if applicable. If no suitable (globally unique) id can be found, append the value of identifier to the URL of the feed.

Rather than doing a SQL query across 3 tables in the pipes, we may want to consider de-normalising some of the database tables so publisher name and feed url are in their own columns in the raw_data table.
We need to consider whether we do this in the normalisation pipes, or add it at the end when the RDPE feed is generated.

 Normalisations

Currently the raw data downloader stores the id field from the top level of the RDPE feed (in the data_id db field). These are usually short strings or occasionally UUIDs; I haven't seen one example with a URI yet. So it's obviously not the value of id or @id from the data at least most of the time. In some (most?) cases (eg) the id on the RDPE level is not even present in any field in the data.
For derivedFrom we need the value of id or @id if present in data.
If we want to add derivedFrom outside of the pipes, we need to store the correct id (pulled from the data) in the database at the time of raw_data storage, so we can look it up for parents. If we add derivedFrom inside the pipes, this shouldn't be a problem.
(Though we may still want to revisit whether we bother storing the RDPE id at the top level in the database, becuase I don't think it's useful for anything..)

If helpful:

RPDE id is just useful for record-level synchronisation between systems when harvesting. It simply identifies records in the same feed, to allow processing of updates. It has no wider relevance.
Everything within data is part of the linked data structure of the opportunity data. @id/id are optional for data publishing, and mandatory when Open Booking API is used.

That said if you wanted to pin-point a particular data item in a feed (such as to help a data publisher debug a validation issue), you would need both the RPDE page URL and the RPDE id.
See https://github.com/openactive/conformance-services/issues/14#issuecomment-647491665 for more info

Okay, now i understand the RDPE id better, thanks.
@thill-odi The wiki examples are ambiguous. Do you want us to use the id from the RDPE feed (glued to the feed url) or the id/@id value from the data for the provenance information, in the case where these are different?

Hi, @rhiaro: the former (id from the RPDE feed) in preference.

Can I get us to to take a step back here and think about other options?
As I understand it, the user story of this feature is so that if your looking at a piece of normalised data that is excellent or bad in some way, you want to be able to trace it back to the raw data it come from so you can debug?
Is that correct?
If so, how about not adding anything extra to the data, but instead, adding a small end point to this app, something like /normalised-data/ID/info, were ID is the id field in the piece of normalised data? We store links between database records, so at this point it would be easy to return a blob of JSON with the one or two raw data items it came from, the actual content of those raw items and more info on them.
Just to double check with @rhiaro: The id in the normalised data will always be there and will always be unique? (From how the database works I think so)

 Add provenanceInformation block

@odscjames: The API endpoint idea is a good one, providing that it's workable. The point is indeed understanding/debugging data items, so another endpoint with the full representation is indeed preferable to te restricted view in provenanceInformation.


Memory allocation bug - 250k parameters runs ok, 25k parameters crashes.

Memory allocation seems pretty arbitrary. I am able to train a network with 250k parameters no problem, and another with 25k parameters hangs (for a looong time) and eventually crashes. They are tested on the same data. Here are the two implementations:
The 25k parameter model:
def dense_gradient_model(X_train, y_train, epochs=5000):
    n_samples, n_timesteps, n_feat = X_train.shape
    main_input = Input(shape=(n_timesteps, n_feat), name='main_input')
    lstms_1 = [Bidirectional(LSTM(2, return_sequences=True))(main_input) for _ in range(20)]
    lstms_2 = [Bidirectional(LSTM(2, return_sequences=True))(lstm_1) for lstm_1 in lstms_1]
    merge_l = merge(lstms_2, mode='concat')
    outputs = TimeDistributed(Dense(7))(merge_l)
    model = Model(input=main_input, output=outputs)
    model.compile(optimizer='rmsprop', loss='mse')
    history = model.fit(X_train, y_train, nb_epoch=epochs,
        validation_split=0.1)
    return history, model

And here is the 250k parameter model:
def dense_gradient_model(X_train, y_train, epochs=5000):
    n_samples, n_timesteps, n_feat = X_train.shape
    main_input = Input(shape=(n_timesteps, n_feat), name='main_input')
    lstm_1 = Bidirectional(LSTM(80, return_sequences=True))(main_input)
    lstm_2 = Bidirectional(LSTM(80, return_sequences=True))(lstm_1)
    outputs = TimeDistributed(Dense(7))(lstm_2)
    model = Model(input=main_input, output=outputs)
    model.compile(optimizer='rmsprop', loss='mse')
    history = model.fit(X_train, y_train, nb_epoch=epochs,
        validation_split=0.1)
    return history, model

The smaller model gives the following error:
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=core-avx2 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/home/hristo/mlenv/local/lib/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -I/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof -L/usr/lib -fvisibility=hidden -o /home/hristo/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmpD8_C2b/268803646968696c28ad22885423a5e1.so /home/hristo/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmpD8_C2b/mod.cpp -lpython2.7
ERROR (theano.gof.cmodule): [Errno 12] Cannot allocate memory
ERROR (theano.gof.opt): Optimization failure due to: constant_folding
ERROR (theano.gof.opt): node: MakeVector{dtype='int64'}(TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4})
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/opt.py", line 1922, in process_node
    replacements = lopt.transform(node)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/tensor/opt.py", line 6387, in constant_folding
    no_recycling=[], impl=impl)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/op.py", line 924, in make_thunk
    no_recycling)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/op.py", line 828, in make_c_thunk
    output_storage=node_output_storage)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1190, in make_thunk
    keep_lock=keep_lock)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1131, in __compile__
    keep_lock=keep_lock)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1589, in cthunk_factory
    key=key, lnk=self, keep_lock=keep_lock)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 1155, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1492, in compile_cmodule
    preargs=preargs)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 2301, in compile_str
    p_out = output_subprocess_Popen(cmd)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/misc/windows.py", line 77, in output_subprocess_Popen
    p = subprocess_Popen(command, **params)
  File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/misc/windows.py", line 43, in subprocess_Popen
    proc = subprocess.Popen(command, startupinfo=startupinfo, **params)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1223, in _execute_child
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

[x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

[x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

The small model memory error, the problem is that the os can't make new
process. Do you have other stuff running on the CPU at the same time? Can
you compile before loading your dataset if the dataset is big?
The problem is that CPU memory is missing.
On Mon, Nov 21, 2016 at 8:51 AM, Hristo Buyukliev<EMAIL_ADDRESS>wrote:

Memory allocation seems pretty arbitrary. I am able to train a network
with 250k parameters no problem, and another with 25k parameters hangs (for
a looong time) and eventually crashes. They are tested on the same data.
Here are the two implementations:
The 25k parameter model:
def dense_gradient_model(X_train, y_train, epochs=5000):
n_samples, n_timesteps, n_feat = X_train.shape
main_input = Input(shape=(n_timesteps, n_feat), name='main_input')
lstms_1 = [Bidirectional(LSTM(2, return_sequences=True))(main_input) for _ in range(20)]
lstms_2 = [Bidirectional(LSTM(2, return_sequences=True))(lstm_1) for lstm_1 in lstms_1]
merge_l = merge(lstms_2, mode='concat')
outputs = TimeDistributed(Dense(7))(merge_l)
model = Model(input=main_input, output=outputs)
model.compile(optimizer='rmsprop', loss='mse')
history = model.fit(X_train, y_train, nb_epoch=epochs,
validation_split=0.1)
return history, model
And here is the 250k parameter model:
def dense_gradient_model(X_train, y_train, epochs=5000):
n_samples, n_timesteps, n_feat = X_train.shape
main_input = Input(shape=(n_timesteps, n_feat), name='main_input')
lstm_1 = Bidirectional(LSTM(80, return_sequences=True))(main_input)
lstm_2 = Bidirectional(LSTM(80, return_sequences=True))(lstm_1)
outputs = TimeDistributed(Dense(7))(lstm_2)
model = Model(input=main_input, output=outputs)
model.compile(optimizer='rmsprop', loss='mse')
history = model.fit(X_train, y_train, nb_epoch=epochs,
validation_split=0.1)
return history, model
The smaller model gives the following error:
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=core-avx2 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/home/hristo/mlenv/local/lib/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -I/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof -L/usr/lib -fvisibility=hidden -o /home/hristo/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmpD8_C2b/268803646968696c28ad22885423a5e1.so /home/hristo/.theano/compiledir_Linux-3.19--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmpD8_C2b/mod.cpp -lpy
thon2.7
ERROR (theano.gof.cmodule): [Errno 12] Cannot allocate memory
ERROR (theano.gof.opt): Optimization failure due to: constant_folding
ERROR (theano.gof.opt): node: MakeVector{dtype='int64'}(TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4}, TensorConstant{4})
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/opt.py", line 1922, in process_node
replacements = lopt.transform(node)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/tensor/opt.py", line 6387, in constant_folding
no_recycling=[], impl=impl)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/op.py", line 924, in make_thunk
no_recycling)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/op.py", line 828, in make_c_thunk
output_storage=node_output_storage)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1190, in make_thunk
keep_lock=keep_lock)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1131, in compile
keep_lock=keep_lock)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1589, in cthunk_factory
key=key, lnk=self, keep_lock=keep_lock)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 1155, in module_from_key
module = lnk.compile_cmodule(location)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cc.py", line 1492, in compile_cmodule
preargs=preargs)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/gof/cmodule.py", line 2301, in compile_str
p_out = output_subprocess_Popen(cmd)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/misc/windows.py", line 77, in output_subprocess_Popen
p = subprocess_Popen(command, **params)
File "/home/hristo/mlenv/local/lib/python2.7/site-packages/theano/misc/windows.py", line 43, in subprocess_Popen
proc = subprocess.Popen(command, startupinfo=startupinfo, **params)
File "/usr/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1223, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

Check that you are up-to-date with the master branch of Keras. You can
update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
If running on Theano, check that you are up-to-date with the master
branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
Provide a link to a GitHub Gist of a Python script that can reproduce
your issue (or just copy the script here if it is short).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4453, or mute the thread
https://github.com/notifications/unsubscribe-auth/AALC-yz0AIpMTYNypezyeS9HayCRbJauks5rAaHrgaJpZM4K4OQv
.

The bug seems pretty consistent when I make the layers with list comprehension. I did not have anything else running (maybe chrome, but I replicated the bug multiple times). It seems that memory used scales not only with parameters, but with number of layers. However, I've built bigger nets before, so that shouldn't be that big of a deal. Another thing I noticed: the error message is repeated multiple times. Is python perhaps trying to parallelize layer creation or something?

The multiple error printing are probably due to Theano adding extra error
information. We don't parallelize the compilation of Theano function, so
that isn't the cause.
The total number of parameter is approx the number of layers * the layers
size. This is normal. Try with less layers.
On Mon, Nov 21, 2016 at 11:16 AM, Hristo Buyukliev<EMAIL_ADDRESS>
wrote:

The bug seems pretty consistent when I make the layers with list
comprehension. I did not have anything else running (maybe chrome, but I
replicated the bug multiple times). It seems that memory used scales not
only with parameters, but with number of layers. However, I've built bigger
nets before, so that shouldn't be that big of a deal. Another thing I
noticed: the error message is repeated multiple times. Is python perhaps
trying to parallelize layer creation or something?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4453#issuecomment-261984638,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALC-yKK8hPEa_3IRb8Je-j_st68vw9nks5rAcPGgaJpZM4K4OQv
.

I know, what I mean was, if model A has 10k params arranged in 4 layers, and model B has 10k params in 3 layers, A is slower - at least it looks that way.


[DUBBO-3137]: step3 - start to use RpcConstants

What is the purpose of the change
issue #3137:
step3 - start to use RpcConstants
Brief changelog
XXXXX
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Run mvn clean install -DskipTests=false & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

but CI fails due to this:
[ERROR] src/main/java/org/apache/dubbo/rpc/RpcContext.java:[19,8] (imports) UnusedImports: Unused import - org.apache.dubbo.common.Constants.
[ERROR] src/main/java/org/apache/dubbo/rpc/protocol/ProtocolListenerWrapper.java:[19,8] (imports) UnusedImports: Unused import - org.apache.dubbo.common.Constants.
[ERROR] src/main/java/org/apache/dubbo/rpc/protocol/ProtocolFilterWrapper.java:[19,8] (imports) UnusedImports: Unused import - org.apache.dubbo.common.Constants.

Codecov Report

Merging #4038 into master will decrease coverage by 0.37%.
The diff coverage is 69.84%.

@@             Coverage Diff              @@
##             master    #4038      +/-   ##
============================================
- Coverage     63.46%   63.08%   -0.38%
  Complexity      565      565
============================================
  Files           753      753
  Lines         32416    32411       -5
  Branches       5143     5143
============================================
- Hits          20572    20448     -124
- Misses         9484     9615     +131
+ Partials       2360     2348      -12

Impacted Files
Coverage Δ
Complexity Δ

...apache/dubbo/rpc/cluster/router/tag/TagRouter.java
35.86% <ø> (ø)
0 <0> (ø)
:arrow_down:

...pache/dubbo/monitor/dubbo/DubboMonitorFactory.java
100% <ø> (ø)
6 <0> (ø)
:arrow_down:

...va/org/apache/dubbo/rpc/filter/TpsLimitFilter.java
100% <ø> (ø)
0 <0> (ø)
:arrow_down:

...g/apache/dubbo/config/AbstractReferenceConfig.java
94% <ø> (ø)
0 <0> (ø)
:arrow_down:

...c/main/java/org/apache/dubbo/common/Constants.java
75% <ø> (-8.34%)
0 <0> (ø)

...e/dubbo/registry/integration/RegistryProtocol.java
82.69% <ø> (ø)
0 <0> (ø)
:arrow_down:

...ache/dubbo/remoting/transport/mina/MinaServer.java
63.33% <ø> (ø)
6 <0> (ø)
:arrow_down:

...src/main/java/org/apache/dubbo/rpc/RpcContext.java
70% <0%> (ø)
0 <0> (ø)
:arrow_down:

...n/java/org/apache/dubbo/rpc/filter/EchoFilter.java
75% <0%> (ø)
0 <0> (ø)
:arrow_down:

.../main/java/org/apache/dubbo/rpc/RpcInvocation.java
63.95% <0%> (ø)
0 <0> (ø)
:arrow_down:

... and 61 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 805f7c7...749976f. Read the comment docs.


[Proposal] Global request callback

Hi,
Is it possible to add something similar to what is found in vue js http resource ?
Vue.http.interceptors.push(function (request, next) {
    request.headers.set('X-CSRF-TOKEN', App.csrfToken);
    next();
});

This code allows to add a 'X-CSRF-TOKEN' to every request

As Semantic UI uses jQuery, https://api.jquery.com/ajaxSend/ should be your friend.

You can also use
$.fn.api.settings.beforeXHR = function(xhr) {
    xhr.setRequestHeader('X-CSRF-TOKEN', 'someValue');
}


Create content for sample Python work to study

An example of how to use a Python package to visualize data.

#sign-off


Battstor outputs for batt_life_model

Fix some inputs to run battery lifetime models from the GUI

Merged in latest from develop without merge conflict. Not sure how to automate the spacing in the vartables. For editorconfig, we're using the one that's included from wxWidgets but the settings are fairly short and don't account for specific use cases like this.


Notice: unserialize(): Error at offset 0 of 4 bytes

dont remember if reported before, but seeing this in pre-last version in logs
2016-11-24T08:20:02+00:00 ERR (3): Notice: unserialize(): Error at offset 0 of 4 bytes  in /var/www/html/web/public_html/app/code/community/Adyen/Payment/Block/Form/Abstract.php on line 77

In 2.8.0 this is caught
if (is_null($this->_paymentData)) {
   $payment = $this->_getQuote()->getPayment();
   try {
      $this->_paymentData = unserialize($payment->getPoNumber());
   } catch (Exception $e) {
      $this->_paymentData = array();
   }
}

Hi @sandermangel ,
Thanks for sharing this. @seansan please upgrade to latest version to solve this issue.
Regards,
Rik
Adyen


Add Arduino to list of C-like languages

Add 'arduino' filetype to list of C-like languages. This is needed in
order to avoid mix-indent-file warnings when using hard tabs for
indentation in Arduino source files (*.ino, *.pde) containing top level
block comments with leading space before a '*'.
Similar to: 460ed028645bdb52ebb4f83a84e64dbf30861ff1
Original fix: 8fde76dd63675b54c918d686036e8de9e057ae8d

thanks


add imagr support

imagr hasn't been updated in 4 yours and probably won't be. Is this really necessary?


The numbering inside "_data" object does not reset

This might not be an issue, but possibly might cause problems in the long run.
I have a node+express REST API server which uses this HashMap library. I have noticed that the _hmuid_ variable is not resetting. For every new request my server receives, the response has an incremented _hmuid_. Its no problem right now, but will it affect performance in the long run? following snippet shows first few lines of the hashmap:
{
    "_data": {
        "♦3": [
            "5b814a2322236100142ac9f6",
            [
                {
                    ...,
                    ...,
                    ...
                }

the ♦3 is not resetting back to ♦1, so I think it is probably using some of my memory. Or is there something I'm doing wrong?

The idea is that objects are unique, if it resets then you don't ensure that.
I would personally don't be concerned about that being a performance issue. The only issue is the keys will get incrementally larger but not in a meaningfully negative way.
If you are still concerned, you can reset the counter by yourself by doing HashMap.uid = 0 at your own risk :)


Allow to set echo arg of sqlalchemy.create_engine() via env. var.

SQLALCHEMY_ECHO
sqlalchemy.create_engine()

The one working on #1908 will run those queries with EXPLAIN ANALYZE, but first they need to know what queries to run, hence this PR.


Just curious, how is going on for this project

NULL as Title

I have not done much to improve it, mainly because Spark 2.3+ performance is already really good, and we have implemented similar functionality at work. I will be spending some time in the future to play with different features to speed up queries on Parquet, e.g. using index pages.

I totally agree with your vision.  for me, I hope expand parquet from storage format, semi-storage engine to full-featured storage engine (with lots of versioned parquet files)

index pages. with short-index definition
rollup based on index definition
define schema-hash for rollups
MVCC as accepting loading data as mini-batch. and compact with designated policy
Metastore, for full-table scan or conditional scan, pickup good rollups and versioned parquet files.

Would it be something similar to Iceberg (https://github.com/Netflix/iceberg)?

I was actually thinking if it would be better to implement something based on Arrow and following paper on Albis file format https://www.usenix.org/system/files/conference/atc18/atc18-trivedi.pdf

Thank you so much for the refer. they are interesting projects in the way~.

Parquet now has its own page index(base on statistics, a little weak, I think), maybe we can do something based on it.

Yes, you are right - we should. I will try reevaluating the project.

Let's open a new issue to follow this 😄. I am very willing to be involved.


VIP-744: Add notify flag to optionally suppress dashboard emails

https://github.com/votinginfoproject/vip_admin/pull/326
https://github.com/votinginfoproject/data-processor/pull/373


⚠ Bofrost.be has degraded performance

In 567404f, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1408 ms

Resolved: Bofrost.be performance has improved in 77518ba.


To add Translate token to TextTokens

It is useful to have the possibility of getting the value returned by a token translated to the current culture.
I would like to add Translate token to TextTokens.
It is a very easy feature to add, What do you think of adding it to the core?
If you accept it can you assign me the issue in order to submit a pull request?

You mean like having a token that takes the output of the previous token and wraps it into a T()? Or what do you have in mind?

I'm not sure either on this. You should translate your entities ... not the tokens. I would say, like @Piedone, do you have an example of what you want to achieve ?

Yes @Piedone it is just that. But only for text not for content.
This answer also your question @Skrypt ? It will translate text not complete entities.
Here is an example of my definition of the token http://orchard.codeplex.com/discussions/641309

A case of use. I have an html widget where I use tokens to render another time the title of a page (it was not enough with the one of the titlePart shape).
This is the token
#{Content.Current.DisplayText}
What if the page needs to support more than one current culture? I would like to have the Translate token for chaining it to text outputs like the one returned by DisplayText.
#{Content.Current.DisplayText.Translate}

I see the issue and how this would be useful. Though unfortunately it would be hard to actually utilize this, as you'd need to write localized strings for each of the possible inputs but targeted at the token provider.

You should also be able to create a widget for each culture. Then your title would already be translated in each of those widgets.

Yes, you are right, I use this solution on this customer cause for some reasons we don't want to use the duplication of content items per language for certain content types. So, it is only useful for me in this specific circumstances.

@skrypt I don't see how localization on html widget can help here. If what I want is to translate current Content Item title the localization part should be at that content item.
I closed the issue because what pointed @Piedone sounds sensible. I'm forced to have translations for my Content titles in .po files and this initially is not a good idea.
So what I have to do is to use localization part for those content types for which I want to another language version. So Translate() token is don't needed because the content item that will be rendered will be just the one I the current language.
The reason why I have reopened the issue is what if I want to translate a projection? No problem, I use a culture filter to force only current culture items are shown by the related query. But how do I translate the title of the projection? To add the localization part to ProjectionPage type is something I don't like. Cause it forces the user to maintain duplicated projections only by the title. Forcing it to remember repeat changes in other properties different from title (the query, the number of items to show, ...) per each culture is adding too much complexity just because you want a translated title.
So, for the sake of simplicity for the user I prefer to add T() to title projections template and avoid use localization part on Projections.
So, at this point Translate token is useful. Don't you think duplication of content item in a projection is a bit too much?
Yes I need to add manually to .po file the translation of new projections but new projections added by a user are something doesn't happen frequently. It usually is done by me in migrations.
Related to make available the translations of .po files to user,  through UI naive this could be the solution http://orchard.codeplex.com/discussions/641356 . I will give it a try to make it work on an Orchard upper version.
At the end of the day I think duplication of content and .po files are valid solutions with their pros and cons. So is good to make it easy to the user combine both to help him make the experience  easy.

Normally the Query that I create have english titles because it is something that most users won't have access to modify. Then I will create a projection for each culture installed in Orchard. I do this because I need to have a different URL for each of those projection pages. So, the title of those projections will be translated. I don't let any editor to be able to change the queries because they could break them.
I understand though that at some point you can't translate every content items to makes this easier for your website editors. I use .po for instance to translate content type names and admin menu names. Though having a way to make this easier would be to have a .po editor inside Orchard that could tell us wich translated string is missing a translation. Then being able to edit those in place. This is a topic that would need to be discussed to find wich way is the best to go.

This is not the first time someone raises the concern. It applies to anything that is in the db and is not a content item data, like field names, content type names, messages in workflows, ... In the end the localization has to be in a po file and this is not manageable.
I think we could have an ad hoc localization strings page that would also taken into account during translation. This way whenever a dev would create a fiel name or workflow string he could add an entry in this page. A specific permission would allow other contributors to edit the translations. This module could even show the technical elements we know are displayed in the ui (like field names).
Another enhancement could be to display the strings which translations haven't been found at some point in time(as an optional feature).

+1 It sounds very convenient.
Related with the multilanguage topic. Many times I have the sensation that should exist another approach for localizing contents different from .po files and localization part in order to avoid not necessary data duplication
What if parts or fields could be configured as localizable at type level?
For example think you can configure Title part within the ProjectionPage type as localizable.
Then when you edit a ProjectionPage content item the UI will offer you a way to complete title for each one of the languages supported by the site. In that way you don't need to duplicate all the content item just because you want to localize only one part. The same approach could by used for fields.
Maybe this is a solution that you have discarded in the past because it has problems I'm not able to see now or because it is complex of develop.

@sebastienros +1 I had the same idea of havng an ad hoc localization strings page.

@jersiovic I had that thought as well initially, but then realized there are many more places that we may need to localize, then concluded a central localization page would make sense (maybe categorized so we have a group of content type translations grouped together).

@sfmskywalker  I see the central localization page is something we need. The localization per part/field in a content item I think is complementary

Yeah I agree it would be nice to have it there as well actually. Maybe like it's done with the AutoroutePart settings using a tabbed UI.

Yeah just that :)


Send proposed transactions

lives on #1010
#1012
#1013

running CI

okay, this is taking ABSURDLY long. that is a bug we have gotta fix


Chem: add a property to package settings to select the default sketcher

The use case: set Sketcher to Marvin by default to all app users (so that the users won’t have to switch on the first use manually).
Sketcher > Marvin, other sketchers...

@MariaDolotova, what do you think of changing the list a bit?
Instead of

openChemLibSketcher
ketcherSketcher
marvinSketcher
chemDrawSketcher

Make a capitalized list of choices:

OpenChemLib
Ketcher
Marvin
ChemDraw


Archive creation does not support pathlib paths

When feeding paths into the HetMat archiving functions, it would be very helpful if pathlib paths were supported inputs, as opposed to just strings.
all_paths_1 = hetmat.metagraph.extract_all_metapaths(1)
paths_1 = []
for path in all_paths_1:
    path = pathlib.Path(f'path-counts/dwpc-0.5/{path}.sparse.npz')
    if path.exists():
        paths_1.append(path)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-31-49cc11dbbf57> in <module>()
      1 hetmech.hetmat.archive.create_archive_by_globs(
----> 2     'dwpc-0.5-len-2.zip', '.', include_globs=paths_2)

~/Documents/hetmech/hetmech/hetmat/archive.py in create_archive_by_globs(destination_path, root_directory, include_globs, exclude_globs, include_paths, **kwargs)
     35     source_paths = set()
     36     for glob in include_globs:
---> 37         source_paths |= set(root_directory.glob(glob))
     38     for glob in exclude_globs:
     39         source_paths -= set(root_directory.glob(glob))

~/.conda/envs/hetmech/lib/python3.6/pathlib.py in glob(self, pattern)
   1072             raise ValueError("Unacceptable pattern: {!r}".format(pattern))
   1073         pattern = self._flavour.casefold(pattern)
-> 1074         drv, root, pattern_parts = self._flavour.parse_parts((pattern,))
   1075         if drv or root:
   1076             raise NotImplementedError("Non-relative patterns are unsupported")

~/.conda/envs/hetmech/lib/python3.6/pathlib.py in parse_parts(self, parts)
     60             if altsep:
     61                 part = part.replace(altsep, sep)
---> 62             drv, root, rel = self.splitroot(part)
     63             if sep in rel:
     64                 for x in reversed(rel.split(sep)):

~/.conda/envs/hetmech/lib/python3.6/pathlib.py in splitroot(self, part, sep)
    281
    282     def splitroot(self, part, sep=sep):
--> 283         if part and part[0] == sep:
    284             stripped_part = part.lstrip(sep)
    285             # According to POSIX path resolution:

TypeError: 'PosixPath' object does not support indexing

Whereas simply changing the append to be a string works correctly.
all_paths_1 = hetmat.metagraph.extract_all_metapaths(1)
paths_1 = []
for path in all_paths_1:
    path = pathlib.Path(f'path-counts/dwpc-0.5/{path}.sparse.npz')
    if path.exists():
        paths_1.append(str(path))

Looks like you're hitting the error in this line:
https://github.com/greenelab/hetmech/blob/240fadb2eaa8d6ed7401d9fb71bd48b20b67d7df/hetmech/hetmat/archive.py#L36-L37
A glob is a pattern and there is no pathlib instantiation of a glob.


🐞 docs: h1 and h2 have the same size

What is the issue?
While reviewing a PR I noticed that H1 and H2 appear to have the same font-size, looking in chrome dev tools I see some conflicting rules for h2, it looks like its being set to 2rem instead of using the variables.
h1  font-size: var(--ifm-h1-font-size);

.markdown>h2 {
    --ifm-h2-font-size: 2rem;
    margin-top: calc(var(--ifm-h2-vertical-rhythm-top)*var(--ifm-leading));
}
h2 {
    font-size: var(--ifm-h2-font-size);
}

It's not clear to me where these are actually defined. I looked in docs/src/css but nothing jumps out at me. It's possible there is some plugin that we are using that is causing this.
Dagger version
any
Steps to reproduce
No response
Log output
No response

thing I found the issue Lev and seems like we are incorrectly using the markdown headings in our pages (https://docusaurus.io/docs/next/markdown-features/toc).
Apparently, pages generally only contain a single h1 heading which is the main title and then go down from there. I've observed that we're currently using two h1 heading in our pages which basically breaks the formatting and the standard layout.
I guess what's left to do here is maybe close this and to fix our pages.


nb_native.init_rand_seed() stuck opening/reading from /dev/random

Environment info

NooBaa Version: 5
Platform: minikube 1.4

Actual behavior

I mainly saw this on minikube but it happened on TravisCI too.
This happens in the core container startup when calling noobaa_init.sh that calls src/upgrade/upgrade_manager.js
We saw that sometimes (not always) this node.js process gets stuck blocking inside init_rand_seed() calling async_open_fd() which doesn't even break when calling process.exit().
In other cases even if the open was successful the read got stuck for a long time waiting for enough random bytes available in /dev/random.
The nb_native.generate_entropy() function that was running did not help because inside a Pod it doesn't find any block devices to read from.

Expected behavior

Add support for process.env.DISABLE_DEV_RANDOM_SEED to avoid it and fallback to openssl default urandom seeding.
Check if there is a node.js fix to the blocking open(/dev/random) that will break once we call process.exit().
Try to fix generate_entropy() to work inside a container without block devices.

Steps to reproduce

See above.

More information - Screenshots / Logs / Other output

See also #4555 and #2721


preview or early release feature

Hi
I do not have these options in the Teams admin center on my tenant. How do we go about getting access to these?
Sean

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6f91604a-35b0-dfbb-55b2-b601cb0841b0
Version Independent ID: 84bcd0d7-b4ed-f13e-c84b-2ec6a7661ae9
Content: Manage app setup policies in Microsoft Teams
Content Source: Teams/teams-app-setup-policies.md
Service: msteams
GitHub Login: @LanaChin
Microsoft Alias: v-lanac

@Sean0029  - thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:  https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.

@officedocsbot close


Speed up ffmpeg & ffprobe if possible

I'm unsure whether there are any more optimizations we can put in, but since extracting screenshots, clips, and file info are a pivotal part of the app, it would be great if these were as fast as possible 😅
Hypothesis:

we can seek to the closest keyframe rather than a specific second in the video (though perhaps we're already doing that)

Weaker hypothesis:

we can use some options when calling ffprobe to extract only the data we want, rather than what we get now (though from my reading of the documentation this doesn't seem possible 🤷♂ )

I welcome other ideas and suggestions 🙇

I think we may already be using keyframes at the moment - I spent a large amount of time researching the best methods for this, and I think we have the current best practice! 👍
But if anyone know otherwise, please tell! 😄

🙇 thank you @cal2195 🙇
I suspected your code was already doing the keyframe thing.
🎉 yay 🎉

I'll just close the issue -- I'd rather have fewer open issues and more user-initiated issues going forward 👍


mobile maps incompatibility

cannot load maps from mobile, it'll just say "Corrupt map file" and "Missing data"

Please note, this web version is still pre-1.4.4.


Integrate async JoinSplit Groth16 verifier with transaction verifier

In https://github.com/ZcashFoundation/zebra/pull/1713/files we integrated the Spend and Output Groth16 async verification, after loading the public Sapling verification params from the waygu crates. We need to do the same with the JoinSplit proofs, which doesn't have a handy crate available. This verifier now needs to be integrated with the rest of the system and fully tested.
Todo

[ ] load the JoinSplit Groth16 verification params into  memory
[ ] Integrate the JoinSplit groth16 async verifier instance with the transaction verifier using the aforementioned keys

This validation isn't in our current list of exceptions for testnet NU5 activation. But we could add it if needed.


allow context on short form service calls

allow short form service calls (i.e. input_boolean.turn_on(entity_id='whatever')) to send context

Superseded by #62


Investigate type errors not showing for .vue files

Apparently it was caused by an update of Volar, which was fixed.


[TO-DO] Refactor real-time counter components

The current real-time counter component works well.   However we want to showcase a higher level of componentization for our community builder.
Two areas need to be addressed with a re-factor:

multiple instances - allowing community builders to use multiple instances of this component on a page  (5 or 10 instances)
higher level of component features bundling - into a more functional component with less configuration / parameterization at the page level

For 2 -  we'd like to refactor it into two components - each one incorporating MULTIPLE COUNTERS:
a)   users-messages-online counters all bundled into a server stats counters component  (one that can be easily multiple instanced, showing the stats from 10 servers on the same page, for example)
b)    a  new team channel stats counters component -  parameterized by a team name   (such as #gsoc2023 )  that will show 3 counters --
i.   number of team channels
ii.  number of messages(sum across all channels of the team)
iii.  number of users (of the team);
This component must also be multi-instance-able   ... showing say 10 teams (from one server) on the same page

Hi @Sing-Li
I'm interested in working on the issue of refactoring the real-time counter component
could you please assign this to me?

hello @Sing-Li  I am interested in working on this issue can you assign this to me.


Add docker compose

Description
Add support to run the sandbox application in a Docker environment
Context
Adding docker-compose.yml will help developers quickly set up their environments and ensure consistency across different machines. It will also make it easier to manage dependencies and services required for the application and reduce "it works on my machine" type issues.
decorators.py/sandbox
Possible Implementation

Create a Dockerfile file
Create a docker-compose.yml, which

runs sandbox service
mounts a volume for output data

Create a run_sandbox.py script as an entrypoint (see example_use.py)
Update instructions in the README on usage

Some adjustments to the start_sandbox function may also be needed. Possible changes:

Get the service and host configs from environment variable instead of self.service_config
Run without threading

We can introduce a flag e.g. use_docker to determine the running environment and adjust the behavior accordingly that users are able to run both locally and using Docker.

@jenniferjiangkells do we need this now? Also can we add hacktoberfest label to this issue?

@TanyaKansal you're welcome to work on this if you want! I will add the label and description

Thanks

@TanyaKansal I've updated the description and added it to hacktoberfest. Apologies the sandbox isn't as well documented so please let me know if you have any questions!

sure, thanks @jenniferjiangkells

Can I contribute to this issue?

@jenniferjiangkells please assign this issue to me so that others don't work on it

@TanyaKansal We don't typically assign issues, your comment lets us and others know that it's already being worked on - just open the PR when you're ready 🙂
@iniminimini It looks like this issue is already being worked on, but you're welcome to take a look at the other open issues to see if there's anything else you'd like to work on!


policy listeners do not get the correct policy assigned

Policy-listeners aren't working properly right now. i noticed that the problem only happens if you have more than one policy file with a listener specified (that is the AuthNetAddr and AuthAddr fields of the policy file).

https://github.com/subgraph/roflcoptor/commit/99ed0458e013719b4c10dfc35324952a02717e6a


Removed Duplicates & Added Google Translate Definition

I removed duplicate characters from the file reducing the row count from 1666 to 1501, including the header row.  Before doing so, I sorted the file and confirmed that the definitions for these 165 deleted rows were an exact match to the original row.
I am pretty excited about the Google Translate definition.  It is not perfect, but at a glance I would guess it gets us a clean, usable definition for probably 70% of the words.  Take a look and give me your thoughts.
After deleting the duplicates yesterday, I realized there may be some value in keeping a list of the words that each character appears in.  As an example, the character "们" appears in 你们, 我们, 他们, and 她们.  Or another example, “学" appears in 学生,上学,放学, 学习, 同学, 科学, 小学, 留学, 数学, and 大学.  These vocabulary words appear across many lessons throughout Duolingo, and this is related to the reason there were so many duplicates in the file.  I would like to note the vocabulary words and lessons where each character appears, and I began working on that yesterday.  Admittedly I am uncertain whether we will print these on an Anki card, which means I might be going down a bit of a rabbit hole, but since I am also doing this for my own benefit, I see personal value in it and I think I can finish it pretty quickly.

I merged for now, but I don't know yet whether we can keep the Google Translate column or not.
What API did you use? See https://opendata.stackexchange.com/questions/4390/google-translate-english-monolingual-definitions
In any case we should still continue to fill the Primary English definition column manually, as Google Translate has some questionable data such as 敦=London that we should not convert into a note.

I have actually been doing my cleaning work in a Google Sheet, and there is a function in there called "GOOGLETRANSLATE" which I used to pull these.  I agree with you on NOT keeping this as a permanent column.  Right now, I am sort of mixing our "publishable" words.tsv file with a working file, which might not be a best practice.
My intention was to add another potentially useful reference to assist with the inevitable manual definition process.  The CC-CEDICT definitions are thorough and complete, but the Google Translate definitions are succinct and likely represent the most common usage of that vocabulary word or character.
But I definitely agree with keeping the "Primary English Definition" as the "Publishable" definition, and I further agree that there are many issues with the Google Translate definitions.  This column represents the definition(s) we have personally selected as most useful to our Anki deck users.


More fixes and improvements for docstrings.

Coverage remained the same at 98.771% when pulling 51f47ae9fbd360775ab4cc93ed0f373d5c06a70d on docs into 0568050b34fb315383e3c0a5d8de5cfca0907eae on master.


fix(event_source): centralizing helper functions for query, header and base64

Issue number: #2424
Summary
Changes
This PR unifies all the helper functions in one file to avoid drifts and fix minor problems in event source data classes.
User experience
There is no change in the user experience.
Checklist
If your change doesn't seem to apply, please leave them unchecked.

[x] Meet tenets criteria
[x] I have performed a self-review of this change
[x] Changes have been tested
[x] Changes are documented
[x] PR title follows conventional commit semantics

Is this a breaking change?
RFC issue number:
Checklist:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

Good to go.... still don't have my access fixed :/

Codecov Report
Patch coverage: 100.00% and project coverage change: +0.01 :tada:

Comparison is base (f2db924) 97.15% compared to head (8a257b0) 97.16%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2496      +/-   ##
===========================================
+ Coverage    97.15%   97.16%   +0.01%
===========================================
  Files          157      158       +1
  Lines         7345     7348       +3
  Branches       533      531       -2
===========================================
+ Hits          7136     7140       +4
  Misses         160      160
+ Partials        49       48       -1

Impacted Files
Coverage Δ

...wertools/utilities/data_classes/active_mq_event.py
100.00% <100.00%> (ø)

...ities/data_classes/api_gateway_authorizer_event.py
100.00% <100.00%> (ø)

...s/utilities/data_classes/appsync_resolver_event.py
100.00% <100.00%> (ø)

...ls/utilities/data_classes/aws_config_rule_event.py
99.50% <100.00%> (-0.01%)
:arrow_down:

...s/utilities/data_classes/cloud_watch_logs_event.py
100.00% <100.00%> (ø)

.../utilities/data_classes/code_pipeline_job_event.py
100.00% <100.00%> (ø)

...lambda_powertools/utilities/data_classes/common.py
100.00% <100.00%> (ø)

...a_powertools/utilities/data_classes/kafka_event.py
96.96% <100.00%> (+4.32%)
:arrow_up:

...wertools/utilities/data_classes/rabbit_mq_event.py
100.00% <100.00%> (ø)

...wertools/utilities/data_classes/s3_object_event.py
100.00% <100.00%> (ø)

... and 2 more

... and 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


KOGITO-7156 : SWF Viewer - Enable SVG generation

Hi @romartin this is the PR for the SVG Generation, can you please review?

@romartin we went over it in private, this is good to go in :)

good, thanks guys!!


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/75330

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/75973

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/76019

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/76024


Flesh out Format

This type adds support for describing all of the things in a data stream header as described in #46 .

Fill out stub for type Format struct, with most fields unexported.
Create a new function, V1(), which accepts no parameters and returns a Format that represents the old style data format, compatible with reading non-headered rezi data streams all the way back to Rezi v1.x.y.
Create a new function, V2(), which accepts parameters related to compression and returns a Format that represents the new style data format.
Add Format.MarshalBinary for encoding it as bytes. It must return empty bytes for V1 formats.
Add Format.UnmarshalBinary for decoding it from bytes. It must accept empty bytes and return a V1 if so.

This should be a new type that is not created by hand - function V2() creates a new format in the new one and accepts any parameters that would be given. Function V1() is how a Format representing the old format is obtained and will be used by default by any routines that are modified to accept an optional Format.

Noted that this will already exist from #11 for forward compatibility


Problem Instance Accessibility for End Users

With our current setup we are only able to direct users to /home/problems/problem_n which is not an ideal user experience. @tim-becker and I have discussed as a possible implementation that would require some performance testing but seems promising.
The basic idea is this: Upon unlocking a new problem, a symbolic link to that problem folder becomes available in the user's home directory. So even though my instance is located at /home/problems/crashme-1_199, I'll be able to refer to my home directory to see ~/crashme-1.
{
  ...,
  submission_heads: [
    {tid: tid, head: sid},
    {tid: "3416a75f4cea9109507cacd8e2f2aefc", head: "a5771bce93e200c36f7cd9dfd0e5deaa"}, ...
  ]
}

Note that head refers to a submission id. We are really storing a reference to the last processed submission for any given server for any given team.
These values will be managed by a daemon that will use the submission_heads object under the shell_server documents to keep things in order.
Consider this possible pseudo code.

teams = api.teams.filter(score > 0)
for server in api.shell_servers.get_online_servers():
  shell = server.get_ssh()
  for team in teams:
    current_head = db.submission_heads.find(tid)
    most_recent_sub = db.submissions.get_most_recent(tid)

    if current_head is None or current_head < most_recent_sub:
      unlocked_problems = api.problems.get_unlocked(team)

      for user in team.get_enabled_users():
        user_sym_links = shell.get_sym_links(user)
        additional_sym_links = unlocked_problems / user_sym_links
        removable_sym_links = user_sym_links / unlocked_problems

        shell.sym_links.add(user, additional_sym_links)
        shell.sym_links.remove(user, removable_sym_links)

This daemon would run in batch at a set period.
This solves a few notable issues.

What if my shell account does not yet exist?

This one is up to us. We can either choose to be conservative and skip the user for the time being which would result in the creation of fewer shell accounts for large scale competitions. Or we could go ahead and create the user on demand as the PAM module would if the user chose to login and process the sym_links normally.

What if a new user joins my team mid competition?

We would invalidate the team's submission_head causing sym_links to be reprocessed. This would then reflect the current team composition.

What if I disable/enable a problem or change a weightmap?

In this case I think the only safe solution would be to invalidate all current_submission heads. This would result in processing shell_servers x team results. In reality, this would be fine. It would just take a bit.

What if a shell server goes offline/I add a new one?

The heads would become stale or not exist at all for the server in question. The daemon would bring them back to speed once they come online.
This should be reasonably performant as we just need one active SSH connection for each server and can do these things en masse. This would be a major win for the end user shell experience.

We should ensure the instance ids are not predictable and that the problems directory is not listable.


pull request for e2e test at 12/23/2016 02:56:13

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


wq/add one_iter_model_test for ascend

add one_iter_model_test configs for ascend

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


package did not export their package.json file

yarn build yields the following complaint:
rollup-plugin-svelte: The following packages did not export their `package.json` file so we could not check the `svelte` field. If you had difficulties importing svelte components from a package, then please contact the author and ask them to export the package.json file.

- hashids

¯\_(ツ)_/¯

https://github.com/niieani/hashids.js/pull/330


dubboctl initial phase

What is the purpose of the change
dubboctl initial phase based on istioctl design.
Brief changelog

code entrance: /pkg/dubboctl/cmd/manifest.go
main code packages: /pkg/dubboctl/internal/apis, /pkg/dubboctl/internal/controlplane, /pkg/dubboctl/internal/manifest, /pkg/dubboctl/internal/util, deploy/profiles

Verifying this change
See /pkg/dubboctl/cmd/manifest_test.go, it shows a simple performing link of manifest generate cmd.

Codecov Report

Merging #1073 (16613a5) into refactor-with-go (8badcd3) will decrease coverage by 8.41%.
The diff coverage is 35.62%.

@@                 Coverage Diff                  @@
##           refactor-with-go    #1073      +/-   ##
====================================================
- Coverage             69.56%   61.16%   -8.41%
====================================================
  Files                    31       41      +10
  Lines                  2389     3175     +786
====================================================
+ Hits                   1662     1942     +280
- Misses                  608     1060     +452
- Partials                119      173      +54

Impacted Files
Coverage Δ

pkg/kube/common.go
9.52% <9.52%> (ø)

pkg/dubboctl/internal/manifest/tree.go
13.77% <13.77%> (ø)

pkg/dubboctl/internal/manifest/common.go
15.90% <15.90%> (ø)

pkg/dubboctl/internal/manifest/util.go
33.33% <33.33%> (ø)

pkg/dubboctl/cmd/root.go
40.00% <40.00%> (ø)

pkg/kube/client.go
45.45% <45.45%> (ø)

pkg/dubboctl/internal/manifest/render/render.go
61.02% <61.02%> (ø)

pkg/kube/object.go
75.38% <75.38%> (ø)

pkg/dubboctl/cmd/manifest.go
100.00% <100.00%> (ø)

pkg/dubboctl/internal/manifest/render/filter.go
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


11474 Updated layouts and styles in some components

Issue #: /bcgov/entity#11474
Description of changes:
THIS IS A MINOR BREAKING CHANGE, since these components not longer render their own "surrounding" v-card; they are now more pure/atomic (div only). The parent page that uses these components will need to wrap these in a v-card / section / sheet / whatever else the parent wants to do to display this component consistently with its other components.

updated layouts (rows/cols) and misc styles in Certify component
updated layouts (rows/cols) and misc styles in ContactInfo component
updated layouts (rows/cols) and misc styles in StaffPayment component
changed Certify from v-card to div
changed StaffPayment from v-card to div
changed CourtOrderPoa from v-card to div + updated layout
updated unit tests
updated Storyboook

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the business-filings-ui license (Apache 2.0).

Were some requirements updated for these components? Do they still align in there current implementations in the Edit UI as well?

These were implicit requirements: as part of layout cleanup in Create UI for registrations, I took this opportunity to remove the wrappers (v-cards in this case) from these components. The v-cards have styling that should not be baked in to these shared components; they should be more pure -- and leave wrapper styling to the parent page. You will see in my forthcoming Create UI PR that the parent pages no longer reach deep (::v-deep) into various components to override their styles; the pages now simply style the components consistently with the other components on their page. (Also these components are now responsive.)
No, you cannot pull these into Edit UI directly. I will update the release numbers for these components to indicate a "major" change. Edit UI can continue to use the old components, or even better: import the new components and wrap them with a v-card or whatever is needed. These are layout changes only (no functional changes).


Add Copy Value as Text to the inspector

Editor properties have Copy Value context option, but it copies the value as Variant into some internal inspector clipboard. I thought it would be useful if there was a way to copy that value as text (as I tried to copy a position from remote node).
https://user-images.githubusercontent.com/2223172/229380357-ebf00b7e-40b3-469f-8669-3302220ce3b0.mp4
"Supersedes" part of #75274 (which adds a similar option)
Doesn't work well with Objects.
Alternatively we could make Copy Value copy both to the inspector clipboard and the text one.

Alternatively we could make Copy Value copy both to the inspector clipboard and the text one.

Alternative PR: #71275. :upside_down_face:

Part of this proposal https://github.com/godotengine/godot-proposals/issues/2951

Alternatively we could make Copy Value copy both to the inspector clipboard and the text one.

Alternative PR: #71275.

I think this might be the better / simpler option.
We just need to figure out what to do in cases where the construct string isn't useful, like Object/Resource.

I am one of the people who are in favor of either (I often bemoaned the fact that I had to type the value over in the script window just because it was in some other scene) and I have no preference towards either solution.


[migrate_parsetree] Update to 407 ast and workaround metaquot

ocaml-migrate-parsetree has deprecated unqualified access, however
the metaquot ppx generates unqualified references; thus we open the
modules in a different fashion as to avoid this problem and have a
clean dev build.
We also migrate to the latest AST version that we can support without
migration of the Types module.
This is the continuation of #38.

I will merge this soon if no further comment.


Right wrist is stuck

Hi,
The right wrist of our iCub  (Hertfordshire01) is stuck. I mean that it doesn't move at all, even with no power, when the robot is off (the movements corresponding to joints 5 and 6 in the right arm).
The last thing I was doing was running some pretty stock stuff in the ARE. Then, when I turned it off, I noticed that in the parking position the wrist was still stiff.
The yarp/icub software is a few days old. The firmware is at v80.
RobotmotorGui reports the joints as operational (however, moving the slider for joints 5 and 6 doesn't make any movement.
I have rebooted the robot completely a few times.
The log is here (taken from the YarpLogger).
https://dl.dropboxusercontent.com/u/22756225/log_pc104_robotInterface_3361
Some errors that are showing are during the shutdown procedure, and not during the startup, and they are there forever :)

@kt10aan, for the time being it would be still better to forward such issues to mechies - since it seems the problem regards the mechanics - thus relying on<EMAIL_ADDRESS>However we are about to harmonize Q&A also for them.

If a joint does not move at all, even with no power, most likely the corresponding motor is burned or the gearbox is stuck/broken.

@randaz81, it doesn't sound good :( Is there a way to determine if it is the motor or the gearbox? How can this be fixed?


Consider a flag to know that the overlay is the last in the stack

When opening nested overlays e.g a vaadin-combo-box inside a vaadin-dialog we might need a way to know if the vaadin-dialog is the last in the stack to prevent closing it on escape or outside clicks.

The element that handles the Esc-keypress should stop the propagation of that event once it has handled it (i.e. closed its overlay). It should not reach the parent elements then.
Same for outside clicks – the element should have a modality curtain that blocks clicks outside it and stop the click event from propagating after that also.


Implement webcvs read payment result API (MG01-00101-701)

Implements webcvs payment result API (MG01-00101-701).
Unfortunately, SBPS's sandbox (API type) seems not working well for webcvs API payment type (payment API result returns "70122999" error code)

LGTM :+1:

Thank youuuuu :cactus:

@kyanny Thank you! The released version 0.4.2 contains this PR :sunglasses:

:tada:


Support for the ability to add schema files via cluster configuration

We have
apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "simple-01"
spec:
  configuration:
    clusters:
      - name: "simple"
    settings:
      format_schema_path: config.d/
    files:
      schema.proto: |

Schema file placed in /etc/clickhouse-server/config.d/schema.proto
But would like to have it in the right place - /etc/clickhouse-server/format_schemas/schema.proto
To do this, you need to specify something like this in the configuration
apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "simple-01"
spec:
  configuration:
    clusters:
      - name: "simple"
    settings:
      format_schema_path: format_schemas/
    files:
      FORMAT_SCHEMAS/schema.proto: |

You can use initContainer in custom podTemplate
Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

You can use initContainer in custom podTemplate Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

This looks like a temporary solution. I can make my own pull request. Just not sure if that would be in line with the general policy.

This example doesnt work due to the following lines: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml#L46-L47
Any insight on how to fix this?

Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

I think this has some errors. Both in practice since the volumes are not declared and the code as is won't start, both in theory since when you declare the said volumes you overwrite the conf.d folder and the InitContainer will be stuck without the file its looking for.
I managed to write my proto schema in /var/lib/clickhouse/schema_formats/ by simply mounting a ConfigMap on the path but using the SecurityContext construct you are using in the example.
          securityContext:
            runAsUser: 101
            runAsGroup: 101
            fsGroup: 101

I'm not sure if this is a security breach since this changes the user my containers run with into the clickhouse user.
@Slach

@qrpike

This example doesnt work due to the following lines: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml#L46-L47

this example works as expected
chi-protobuf-example-common-configd means configmap name which generated by clickhouse-operator
so, if you change your metadata name from protobuf-example to other you need to change this name in manifest

@czephyr

Would you please clarify on the reason it's not possible to write on the /var/lib/clickhouse/ folder without the securityContext and why we should use the initContainer approach just to move a file?

in example I just tried to realize all inside kind: ClickHouseInstallation  if you will manage external kind: ConfigMap
you could mount it inside /var/lib/clickhouse/schema_formats/ directly
But, clickhouse-server and change their process UID after start to clickhouse (101) even if your container user is root
So, it requires applying securityContext to allow reading .proto files inside clickhouse-server process.

chi-protobuf-example-common-configd means configmap name which generated by clickhouse-operator
so, if you change your metadata name from protobuf-example to other you need to change this name in manifest
the wouldnt you need a volume definition in there like

            - name: chi-protobuf-example-common-configd
              configMap:
                name: name-of-configmap-generated-by-operator


[IDEA] Backup file name format

Hey there, would be cool to have the option to provide a backup file name format. I'm using the java.time formats here.
Like this:
backup:
  # (true/false) Automatic backup once in a specified period
  autoBackup: true
  # Set a format for backup file names (see java.time docs)
  backupFileNameFormat: "dd-MM-yyyy HH-mm-ss"

My use case: I want my file system to automatically sort backups. So 01-08-2024 18-11-49.zip should be 2024-08-01 18-11-49.zip to achieve this. Therefore, my backupFileNameFormat would be set to "yyyy-MM-dd HH-mm-ss".

Nice idea! I will implement it in near future


Possible bug

Hello I was wonder if in buoy_client line 64 is not :
{ok, [{request_id(RequestsIn), Response}], State#state

instead :
{ok, [{RequestsIn, Response}], State#state

Otherwise I cannot see how RequestId from handle_request can match correctly the response.
Silviu

Hi @silviucpp,
I just released version 0.1.0. Any bugs you experienced using the very alpha version should be fixed.
Cheers,
LP


Sidebar menu displaying only the last item.

Sidebar is displaying only the last menu item.
I created my own event and even copied the given example but nothing.
So i looked into SidebarMenuEvent.php and changed this on line 59 $this->menuRootItems[] = $item; and worked. But i know that there is something wrong. Here is my code:
http://pastebin.com/WS8YxBza
What am i doing wrong?

Nevermind.
I realize that the ItemId are all the same.
 Sorry

Alright, this issue can be closed then!


Add KV Store for holding queue

Currently we use a simple object for storing recent posts. If the server is reset/crashes this data is lost. We should look into using a javascript built in KV Store to persist our values.
Something similar to bolt for Go https://github.com/boltdb/bolt

I did some research on this - and here are the different options I see.
NeDB

Subset of MongoDB features
- Not maintained (see NestDB)

NestDB

Most active fork of NeDB
- Only 3 public releases (once per year)
- Small number of stars on the project - perhaps because they are all on NeDB

PouchDB

Works very well with CouchDB
Great for offline use and then syncs with another database.
- Really a wrapper of other databases (CouchDB, IndexedDB in Browsers, LevelDB in Nodejs)

Level

Pure KV support
Interface to LevelDB (C++ DB by Google)
Seems to target features to align with the browser standard IndexedDB
- No support for queries beyond a get

RocksDB

Pure KV support
LevelDB compliant multi-CPU optimized DB (built by FB)
- Same as Level (no support for queries beyond a get)

SQLite

+ Supports complex queries
- Slower than LevelDB

LowDB

+ Pure JS
? Lodash based
+ and extremely popular
- Likely scales worst among other options - seems like a great in memory API for small collection with a read/write adaptor.

Loki

+ Pure JS
+ As popular as Level, SQLite, LowDB
+ More performant than LowDB.
+ Supports complex queries
- Likely slower than LevelDB and SQLite.

I don't have any biases after doing this research. Feel free to share any thoughts.
Our needs do not need complex queries, so I am gravitating towards trying LevelDB.

this is gold! thanks @vineet-sinha


OSError: [Errno 22] Invalid argument

Hello,
I'm testting the asdf library at the moment and it seems that I have it some kind of shape limit:
Here the snippet:
sequence = np.ndarray(shape=(10 ** 6, 1000), dtype=int)
tree = {
    'key': 'value',
    'name': 'Test data',
    'type': 'Numpy matrix',
    'sequence': sequence,
}

af = asdf.AsdfFile(tree)
af.set_array_storage(sequence, 'external')
af.write_to('example.asdf', all_array_storage='external')

My env
(asdf) ➜  benchmarks pip freeze                                                                                10:28:57
appnope==0.1.0
asdf==2.0.1
backcall==0.1.0
certifi==2018.4.16
decorator==4.3.0
ipython==6.4.0
ipython-genutils==0.2.0
jedi==0.12.1
jsonschema==2.6.0
numpy==1.15.0
parso==0.3.1
pexpect==4.6.0
pickleshare==0.7.4
prompt-toolkit==1.0.15
ptyprocess==0.6.0
Pygments==2.2.0
PyYAML==3.13
semantic-version==2.6.0
simplegeneric==0.8.1
six==1.11.0
traitlets==4.3.2
wcwidth==0.1.7

Traceback:
(asdf) ➜  benchmarks python run.py                                                                                10:30:47
Traceback (most recent call last):
  File "run.py", line 5, in <module>
    asdf_test.write()
  File "/Users/yj/LCSB/programmation/benchmarks/benchmarks/asdf_test/write.py", line 26, in write
    af.write_to('example.asdf', all_array_storage='external')
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/asdf.py", line 1059, in write_to
    self._serial_write(fd, pad_blocks, include_block_index)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/asdf.py", line 822, in _serial_write
    self.blocks.write_external_blocks(fd.uri, pad_blocks)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/block.py", line 353, in write_external_blocks
    asdffile.write_to(subfd, pad_blocks=pad_blocks)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/asdf.py", line 1059, in write_to
    self._serial_write(fd, pad_blocks, include_block_index)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/asdf.py", line 821, in _serial_write
    self.blocks.write_internal_blocks_serial(fd, pad_blocks)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/block.py", line 294, in write_internal_blocks_serial
    block.write(fd)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/block.py", line 1085, in write
    fd.write_array(self._data)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/generic_io.py", line 754, in write_array
    _array_tofile(self._fd, self._fd.write, arr)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/generic_io.py", line 112, in _array_tofile
    return _array_tofile_chunked(write, array, OSX_WRITE_LIMIT)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/site-packages/asdf/generic_io.py", line 101, in _array_tofile_chunked
    write(array[i:i + chunksize].data)
  File "/Users/yj/.miniconda3/envs/asdf/lib/python3.7/tempfile.py", line 481, in func_wrapper
    return func(*args, **kwargs)
OSError: [Errno 22] Invalid argument

However, if I use sequence = np.ndarray(shape=(10 ** 5, 1000), dtype=int), I have no issue at all.

Hi @yjarosz, thanks very much for the report.
This possibly appears to be related to https://bugs.python.org/issue24658. In your code when you change the array size from 10 ** 6 * 1000 to 10 ** 5 * 1000, you are downsizing from a roughly 8GB file to an 800MB file, which crosses the 2GB threshold reported in the Python issue.
It's possible that this can be worked around on the ASDF end by changing the size of the partial writes. I'll look into it.

@yjarosz see #521 for the workaround solution in ASDF. This will probably be merged later today, and I can try to get a bugfix release out by tomorrow or Friday. If it's practical for you, you can install the development version of ASDF in the meantime once this is merged.

Indeed, taht works perfectly :) Thanks @drdavella


Unenforce StreamHandler in tf_logging

System information

TensorFlow version (you are using): 1.12
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
Currently the tf_logging API will always add its own StreamHandler to the tensorflow logger (see: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/platform/tf_logging.py#L121-L124). According to the Python logging documentation this is discouraged:

Note: It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers. This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers ‘under the hood’, you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

Will this change the current api? How?
If some optional configuration parameter is added to not use the StreamHandler, this does not need to affect how the tf_logging API is used today.
Who will benefit with this feature?
Any developer (like myself) that work on complex projects where tensorflow is not the only component that performs logging. I want to propagate tensorflow LogRecords to a handler at the root logger object, so that I might decide myself how best to print them or save them.

I am lazy and should have read the code with a bit more thoroughness. I see that an attempt have been made at avoiding adding the handler if the root logger is already configured. From _get_logger():
# Don't further configure the TensorFlow logger if the root logger is
# already configured. This prevents double logging in those cases.
if not _logging.getLogger().handlers:
...

The problem is that this function is already called when importing tensorflow. So if you import tensorflow in your __main__ module, you don't get to configure the root logger before the tensorflow logger is initialized, and the StreamHandler is added. If this is not the intended behaviour I guess this issue should rather be made into a bug report.

Nagging Assignee @drpngx: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@harahu as we move to tf 2.0, the plan is that the logging will disappear. So you will essentially use your own logging solution. I'm happy to review PRs if you have a good fix, but at the end of the day, the plan is that this will become more or less dead code.

I consider removing tf logging an acceptable solution to the initial problem. But I am curious as to the reasoning behind it. Is there no need for logging in the core tf codebase? I see some logging is happening in C++ at the moment. Any plans to make that part of logging configurable through the python API in 2.0?

@harahu,
The code you mentioned in this PR works without any error. Please find the Gist of the working code. Thanks!

@harahu,
Sorry for the delayed response The code you mentioned in this PR works without any error. Please find the Gist of the working code. Thanks!

This is fixed, at least in tf 2.x, so we can close this issue.


Cannot enforce audio input (microphone) format parameter

Hi,
Seems that /microphone:format parameter does not work properly, except for 0x0001 (PCM).

When used without format, the ALAW codec is chosen (0x0006) by default, the microphone is properly forwarded to the server, but there is no sound there, level meters are down. On client side, pavucontrol shows, that FreeRDP is using the mic, and the level meters are lit.
When used with format:0x0006, the same default behavior is observed
When used with format:0x0001, the microphone works properly
When used with any other format, for example format:0xA106 (as attached in the debug log), the microphone is not forwarded at all. On the client side the PA stream is not created, pavucontrol does not show FreeRDP using the mic. There is no "microphone uses [CODEC]" message in the FreeRDP log, instead rather mysterious message is shown:

[16:07:33:897] [19864:19913] [ERROR][com.freerdp.channels.audin.client] - invalid format index 0 (total 0)
[16:07:33:366] [19864:19913] [ERROR][com.freerdp.channels.audin.client] - invalid format index 0 (total 0)

The problem reproduces in xfreerdp also.
The microphone selected as the default source in Pulseaudio (PipeWire) and works correctly on the client side, i.e. it records and plays with parecord/paplay.
The server is Windows 10 Pro 22H2. The client is Gentoo v6.1.46 amd64 running FreeRDP version 2.10.0
Now what is it for
While the PCM format appears to be problem-free and it just works, at the same time it is rather bandwidth-hungry and the significant voice delay is observed (e.g. Teams meetings).
Forcing the needed codec (or not advertising one) in some working way would be really sweet.
The command line is
wlfreerdp /log-level:debug /v:[HOST] /port:[PORT] /u:[USER] /d:[DOMAIN] /from-stdin:force /cert:ignore -sec-rdp -aero -decorations -fonts -gestures -menu-anims -multitouch -themes -wallpaper -window-drag +clipboard +fast-path +suppress-output +auto-reconnect +multitransport /network:auto -drives -home-drive /microphone:sys:pulse,format:0xA106 /sound:sys:pulse /dynamic-resolution /f /gfx:avc444 +gfx-progressive >/tmp/freerdp.log 2>&1
This is Build configuration
BUILD_TESTING=OFF BUILTIN_CHANNELS=ON HAVE_AIO_H=1 HAVE_EXECINFO_BACKTRACE=1 HAVE_EXECINFO_BACKTRACE_SYMBOLS=1 HAVE_EXECINFO_BACKTRACE_SYMBOLS_FD=1 HAVE_EXECINFO_H=ON HAVE_EXECINFO_HEADER=1 HAVE_FCNTL_H=1 HAVE_GETLOGIN_R=1 HAVE_GETPWUID_R=1 HAVE_INTTYPES_H=1 HAVE_MATH_C99_LONG_DOUBLE=1 HAVE_PIXMAN_REGION=OFF HAVE_POLL_H=1 HAVE_PTHREAD_MUTEX_TIMEDLOCK=ON HAVE_PTHREAD_MUTEX_TIMEDLOCK_LIBS= HAVE_PTHREAD_MUTEX_TIMEDLOCK_SYMBOL=1 HAVE_SYSLOG_H=1 HAVE_SYS_EVENTFD_H=1 HAVE_SYS_FILIO_H= HAVE_SYS_MODEM_H= HAVE_SYS_SELECT_H=1 HAVE_SYS_SOCKIO_H= HAVE_SYS_STRTIO_H= HAVE_SYS_TIMERFD_H=1 HAVE_TM_GMTOFF=1 HAVE_UNISTD_H=1 WITH_ALSA=OFF WITH_CAIRO=OFF WITH_CCACHE=OFF WITH_CHANNELS=ON WITH_CLANG_FORMAT=ON WITH_CLIENT=ON WITH_CLIENT_AVAILABLE=1 WITH_CLIENT_CHANNELS=ON WITH_CLIENT_CHANNELS_AVAILABLE=1 WITH_CLIENT_COMMON=ON WITH_CLIENT_INTERFACE=OFF WITH_CUPS=OFF WITH_DEBUG_ALL=OFF WITH_DEBUG_CAPABILITIES=OFF WITH_DEBUG_CERTIFICATE=OFF WITH_DEBUG_CHANNELS=OFF WITH_DEBUG_CLIPRDR=OFF WITH_DEBUG_DVC=OFF WITH_DEBUG_KBD=OFF WITH_DEBUG_LICENSE=OFF WITH_DEBUG_MUTEX=OFF WITH_DEBUG_NEGO=OFF WITH_DEBUG_NLA=OFF WITH_DEBUG_NTLM=OFF WITH_DEBUG_RAIL=OFF WITH_DEBUG_RDP=OFF WITH_DEBUG_RDPDR=OFF WITH_DEBUG_RDPEI=OFF WITH_DEBUG_RDPGFX=OFF WITH_DEBUG_REDIR=OFF WITH_DEBUG_RFX=OFF WITH_DEBUG_RINGBUFFER=OFF WITH_DEBUG_SCARD=OFF WITH_DEBUG_SND=OFF WITH_DEBUG_SVC=OFF WITH_DEBUG_SYMBOLS=OFF WITH_DEBUG_THREADS=OFF WITH_DEBUG_TIMEZONE=OFF WITH_DEBUG_TRANSPORT=OFF WITH_DEBUG_TSG=OFF WITH_DEBUG_TSMF=OFF WITH_DEBUG_TSMF=OFF WITH_DEBUG_TSMF_AVAILABLE=0 WITH_DEBUG_WND=OFF WITH_DEBUG_X11=OFF WITH_DEBUG_X11_CLIPRDR=OFF WITH_DEBUG_X11_LOCAL_MOVESIZE=OFF WITH_DEBUG_XV=OFF WITH_DSP_EXPERIMENTAL=OFF WITH_DSP_FFMPEG=ON WITH_EVENTFD_READ_WRITE=1 WITH_FAAC=OFF WITH_FAAD2=OFF WITH_FFMPEG=TRUE WITH_FFMPEG=TRUE WITH_GFX_H264=ON WITH_GPROF=OFF WITH_GSM=OFF WITH_GSSAPI=OFF WITH_GSTREAMER_1_0=OFF WITH_ICU=OFF WITH_INTERNAL_MD4=OFF WITH_INTERNAL_MD5=OFF WITH_IPP=OFF WITH_JPEG=ON WITH_LAME=OFF WITH_LIBRARY_VERSIONING=ON WITH_LIBSYSTEMD=OFF WITH_MACAUDIO=OFF WITH_MACAUDIO=OFF WITH_MACAUDIO_AVAILABLE=0 WITH_MANPAGES=OFF WITH_MBEDTLS=OFF WITH_NEON=OFF WITH_OPENCL=OFF WITH_OPENH264=TRUE WITH_OPENH264=TRUE WITH_OPENH264_LOADING=OFF WITH_OPENSLES=OFF WITH_OPENSSL=ON WITH_OSS=ON WITH_PCSC=OFF WITH_PROFILER=OFF WITH_PULSE=ON WITH_SAMPLE=OFF WITH_SANITIZE_ADDRESS=OFF WITH_SANITIZE_ADDRESS_AVAILABLE=1 WITH_SANITIZE_MEMORY=OFF WITH_SANITIZE_MEMORY_AVAILABLE=1 WITH_SANITIZE_THREAD=OFF WITH_SANITIZE_THREAD_AVAILABLE=1 WITH_SERVER=OFF WITH_SERVER_INTERFACE=ON WITH_SMARTCARD_INSPECT=OFF WITH_SOXR=OFF WITH_SSE2=ON WITH_SWSCALE=ON WITH_THIRD_PARTY=OFF WITH_VAAPI=OFF WITH_VALGRIND_MEMCHECK=OFF WITH_VALGRIND_MEMCHECK_AVAILABLE=1 WITH_VERBOSE_WINPR_ASSERT=ON WITH_WAYLAND=ON WITH_WINPR_TOOLS=ON WITH_X11=OFF WITH_XINERAMA=OFF WITH_XV=OFF WITH_ZLIB=ON
Build type:          RelWithDebInfo
CFLAGS:              -O2 -pipe -march=native -Wno-unused-parameter -Wno-unused-macros -Wno-padded -Wno-c11-extensions -Wno-gnu -Wno-unused-command-line-argument -Wno-deprecated-declarations -fno-omit-frame-pointer -DWINPR_DLL
Compiler:            Clang, 16.0.6
Target architecture: x64
Program versions
net-misc/freerdp-2.10.0-r3::gentoo
media-libs/libpulse-16.1-r2::gentoo
media-video/pipewire-0.3.77-r1::gentoo (compiled with ffmpeg)
media-video/ffmpeg-4.4.4-r3
Debug log
freerdp.log

that is not a bug, you must compile with WITH_DSP_EXPERIMENTAL to get the format encoder working.
as for ALAW, there seems to be a bug with pulse? need to check, maybe we should just deactivate that format

Sadly, Gentoo's portage does not have this USE flag.
Does it mean, that WITH_DSP_EXPERIMENTAL replaces WITH_DSP_FFMPEG and uses its own codecs?
Is there a way to force the required codec or format to use ffmpeg for encoding?
Well, alaw is really old, but it has good characteristics for voice applications, keeping very low bandwidth and latency, while maintaining decent quality. So does gsm. I don't know for sure, but it seems that it is working in pipewire/pulseaudio, as the pavucontrol shows the FreeRDP's meters go up and down, as I speak to the microphone. On the other side, the Windows meters are dead.

@HazelHex no, that is an additional CMake parameter that enables aac encoding. Does not work reliable with all FFMPEG versions though, therefore disabled.
as for ALAW, I know the properties, it just does not work in practice (maybe the same reason as aac?) and therefore should also be disabled until the issues are resolved.

Hello dear friends,
Now the same remote side (Windows 10) just hangs, when I enable the microphone forwarding, even with the PCM! I have rebooted everything.
It seems that even with format:1, when some app under Windows actually starts to use the mic (e.g. Sound Settings -> Record -> Remote input), the mic tray icon shows up, but the whole desktop just freezes and only pkill wlfreerdp helps.
When using other than PCM codec, the mic is just silent, so as stated under my first post.
Please advice what additional workarounds/tests I must follow, to make the microphone just work with any codec. Thanks.
freerdp.log


allowing consumers without username

Fix: https://github.com/mybuilder/kongfig/issues/29

This doesn't fix the issue of looking up a customer with custom_id.

Indeed, this is not what I wanted.
I wanted to export my kong config, with users populated from gelato that doesn't create usernames but only cusom_id

It shouldn't be too hard to do, as both must be unique.


Option to bypass SSL cert verification

See: https://github.com/cyrus-and/chrome-remote-interface/wiki/Bypass-certificate-errors-("Your-connection-is-not-private")

On second thought, this is probably a bad idea and doesn't make sense.