#238 Add support for Confluent's Avro Serializers / Deserializers

Adds ability to use KafkaAvroDeserializer for the value_deserializer_class

@dkelley-accretive I've looking at reviewing this to get it merged asap, any tips on how to setup a test for this functionality?

@jsvd I have manually tested it (we are running it in production currently).  I took a look at the automated tests for the plugin.  To write some automated tests I would look to write some avro data to a new topic in kafka_test_setup.sh and write a new spec to consume from that topic and do some simple validation.  I can try and take a stab at it if you want but I'm not really a ruby expert.

@jsvd An instance of the confluent schema registry would need to be spun up in kafka_test_setup.sh as well

thanks for the input @dkelley-accretive, I'll take a stab at manually testing this this week.

@jsvd These might be handy as well.
Sample plugin config:
input {
kafka {
id => "kafka_avro_events"
group_id => "logstash_kafka_avro"
topics_pattern => "some_avro_topic"
bootstrap_servers => "kafka:9092"
decorate_events => true
value_deserializer_class => "io.confluent.kafka.serializers.KafkaAvroDeserializer"
schema_registry_url => "http://schemaregistry:8081"
metadata_max_age_ms => "5000"
codec => "json"
}
}
Sample docker-compose file with kafka / schema registry:
version: "3.3"
services:
zookeeper:
image: confluentinc/cp-zookeeper:3.3.0
container_name: zookeeper
restart: always
environment:
ZOOKEEPER_CLIENT_PORT: "2181"
kafka:
image: confluentinc/cp-kafka:3.3.0
container_name: kafka
environment:
KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
KAFKA_ADVERTISED_HOST_NAME: "kafka"
KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:9092"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
depends_on:
- zookeeper
restart: always
ports:
- "9092:9092"
schemaregistry:
image: confluentinc/cp-schema-registry:3.3.0
container_name: schemaregistry
environment:
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181
SCHEMA_REGISTRY_HOST_NAME: schemaregistry
depends_on:
- zookeeper
restart: always
ports:
- "8081:8081"

ok I got past the build errors (strangely I can't seem to use version 4.1.1, I can only go up to 3.3.1)
I can make logstash start up again normally. is there any schema and setup for a producer of events for that schema I could use to test?
On another note, I'm a bit concerned about added the confluent jars in this plugins. It brings another piece dimension to the compatibility matrix, where we must ensure the kafka client + kafka broker + schema registry are all aligned correctly

@jsvd I can try and see what the issue is with versions > 3.3.1 later if you would like.  To test it you can use the kafka-avro-console-producer that comes with the schema registry to write avro data to a topic, there are some examples here https://docs.confluent.io/3.3.0/quickstart.html
For example:
cat '{"f1": "value1"}' | $ <path-to-confluent>/bin/kafka-avro-console-producer \          --broker-list localhost:9092 --topic test \          --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'

I understand the proposed approach would typically be combined with the JSON codec. Does that mean the data gets decoded/parsed twice? Once during Avro "parsing" and once by the JSON codec?
If that's the case, isn't there an approach possible to leverage the fact that the Avro deserialization already outputs an object with all the fields?
The logstash-codec-avro_schema_registry codec seems to do that.
I don't know the Logstash Input and Codec API's, but below the difference I notice.
The Avro codec somehow manages here to "just" feed the output object of the Avro deserializer to a Logstash event constructor.
While in the Kafka plugin, whatever gets deserialized seems to be converted to string here and inputted into the configured codec.

Thanks for your reply. I was referring to incoming events. So go from Avro encoded message in Kafka to an event in Logstash which for example allows filtering on a specific field in Logstash pipeline.
This change implements I believe an approach whereby the message gets decoded (with awareness of the structure) and then serialized to a single JSON string after which that string gets decoded again by the JSON codec.
If you have, say, 10k messages a second, that overhead sounds significant.

I don't know the internals, but I don't see why there would be decoding back into JSON if you can preserve the schema object and the Avro bytes. Unless the Logstash event must be preserved and transferred always via JSON.

I assume it gets decoded twice based on:

First time by the Confluent Avro Deserializer - it's in the name, it decodes the Kafka message from Avro binary.
Second time here - that's not code added by this pull request, but in the base Kafka plugin. Record is a Kafka related object (result of the deserialier) on which .to_s is executed. So a string is fed into the codec. Which as indicated in the example config here above, should be the json codec.
I only have limited knowledge about the internals and wanted some confirmation.
There is work ongoing for E2E delivery guarantees, if implemented for Kafka, it could be the right moment to change the approach and allow looping over the 'fields' in the Kafka record and keep them as Logstash fields instead of passing everything as a single string.

Any update on this PR?

We believe this issue has now been solved in version 10.6.0 of the Kafka input from logstash-integration-kafka plugin.
This version allows defining the URL for the Confluent Schema Registry used to manage the Avro schemas (it also offers proxy settings to access this scheme specifically). More technical information can be found in the meta issue 1, or you can view the documentation in the plugin docs
As such, I'm closing. Thank you again for the contribution.

@andsel May I know where do i mention the schema file in the Kafka input? - I currently have the following config:
kafka {
    bootstrap_servers => "localhost:9092"
    topics => "customer-avro-value"
    schema_registry_url => "localhost:8081"
  }

And It is giving me the below error:
[2020-11-26T22:07:29,059][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<URI::InvalidURIError: bad URI(is not URI?): ://localhost>, :backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/uri/rfc3986_parser.rb:67:in `split'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/uri/rfc3986_parser.rb:73:in `parse'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/uri/common.rb:237:in `parse'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:454:in `proxy_from_env'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:86:in `initialize'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday.rb:70:in `new'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.6.0-java/lib/logstash/plugin_mixins/common.rb:48:in `check_for_schema_registry_connectivity_and_subjects'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.6.0-java/lib/logstash/plugin_mixins/common.rb:32:in `check_schema_registry_parameters'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.6.0-java/lib/logstash/inputs/kafka.rb:249:in `register'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:386:in `start_inputs'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:311:in `start_workers'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/Users/Smit/Documents/Dev/ELK/logstash-7.10.0/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/usr/local/etc/logstash/logstash.conf.v1_5"], :thread=>"#<Thread:0x1c64d125 run>"}

@shah-smit the schema name is a composition of the topic name and -value https://www.elastic.co/guide/en/logstash/master/plugins-inputs-kafka.html#plugins-inputs-kafka-schema_registry_url you have to load on Schema Registry instance before using it from Kafka input. The schema_registry_url should be http://localhost:8081

@andsel Noted. It seems that accidentally i have configured my schema to be with the same format as suggested above:

Also, the same config, I am using to send the data to the Kafka:

However I am still getting the same error. Any further suggestions
Also here is the avsc file:
{
     "type": "record",
     "namespace": "com.example",
     "name": "Customer",
     "version": "1",
     "fields": [
       { "name": "first_name", "type": "string", "doc": "First Name of Customer" },
       { "name": "last_name", "type": "string", "doc": "Last Name of Customer" },
       { "name": "age", "type": "int", "doc": "Age at the time of registration" },
       { "name": "height", "type": "float", "doc": "Height at the time of registration in cm" },
       { "name": "weight", "type": "float", "doc": "Weight at the time of registration in kg" },
       { "name": "automated_email", "type": "boolean", "default": true, "doc": "Field indicating if the user is enrolled in marketing emails" }
     ]
}

@andsel Let me know, if there is any update on the above. Thanks.

Created bug for this: https://github.com/logstash-plugins/logstash-integration-kafka/issues/63

@shah-smit This has been fixed with version 10.7.0 of logstash-integration-kafka

Perfect. Thank you @andsel

@andsel I am getting an error whilst connecting to Schema Registry that is placed on another Server.
Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Schema registry service doesn't respond, error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target>, :backtrace=>["/elkstack/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.7.0-java/lib/logstash/plugin_mixins/common.rb:60:in check_for_schema_registry_connectivity_and_subjects'", "/elkstack/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.7.0-java/lib/logstash/plugin_mixins/common.rb:32:in check_schema_registry_parameters'", "/elkstack/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.7.0-java/lib/logstash/inputs/kafka.rb:249:in register'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:228:in block in register_plugins'", "org/jruby/RubyArray.java:1809:in each'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:227:in register_plugins'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:386:in start_inputs'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:311:in start_workers'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:185:in run'", "/elkstack/logstash-7.10.1/logstash-core/lib/logstash/java_pipeline.rb:137:in block in start'"]

@shah-smit a couple of questions:

are you using https to connect to the other Sever which hosts the SchemaRegistry?
have you tried plain HTTP?
could you try a cURL to schema_registry_url + /subjects

@andsel Yes, I tried curl-ing the endpoint and also it is HTTPs.
However I manage to solve by adding the certs(.cer) in the jdk (logstash) cacerts using keytool.

Good point, we could think to add the cert settings to reach in Schema Registry configuration, close to the uri


MINOR: fix comment in ClusterConnectionStates

Found minor typo in ClusterConnectionStates.java
More detailed description of your change,
iff --> if
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Call for review @dajac  @junrao  @guozhangwang  @ijuma


feat: Update get-higress.sh

Ⅰ. Describe what this PR did
Update get-higress.sh:

Support arm64 platforms
Update the description of file config URL
Exclude test folder when decompressing the install package

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #500 (7d8a7c0) into main (2b9e3a1) will increase coverage by 0.04%.
The diff coverage is n/a.

:exclamation: Current head 7d8a7c0 differs from pull request most recent head 3554bbb. Consider uploading reports for the commit 3554bbb to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #500      +/-   ##
==========================================
+ Coverage   40.31%   40.36%   +0.04%
==========================================
  Files          45       45
  Lines        6186     6186
==========================================
+ Hits         2494     2497       +3
+ Misses       3519     3517       -2
+ Partials      173      172       -1

see 1 file with indirect coverage changes


nixos-option crashes when used to list every option in NixOS

Describe the bug
nixos-option throws errors constantly in the middle of standard output and if you run it recursively across the entire NixOS operating system, it will crash.
Steps To Reproduce
Steps to reproduce the behavior:

run `nixos-option -r > file.txt 2> file2.txt

Expected behavior:
There would be no stack traces in the middle of standard output, and the program would not crash.
Additional context
I switched from NixOS stable to NixOS unstable, if that matters.
@emilazy @tropf @zowoq
$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.42, NixOS, 23.11 (Tapir), 23.11pre509987.66aedfd0102`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos-23.05-23.05, nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Running into the same issue. For me, the last few lines of the output are the following:
terminate called after throwing an instance of 'nix::ThrownError'
  what():  The option `virtualisation.vmVariant.services.grafana.settings' is used but not defined.
Aborted (core dumped)

Is it tripping up in the same place for you @beepingcities ?

I have same issue
terminate called after throwing an instance of 'nix::ThrownError'
  what():  error:
       … while evaluating the attribute 'options'

         at /home/badele/.nix-defexpr/channels/nixpkgs/nixos/default.nix:15:17:

           14| {
           15|   inherit (eval) pkgs config options;
             |                 ^
           16|

       … while evaluating the attribute 'options'

         at /home/badele/.nix-defexpr/channels/nixpkgs/lib/modules.nix:319:9:

          318|         _type = "configuration";
          319|         options = checked options;
             |         ^
          320|         config = checked (removeAttrs config [ "_module" ]);

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: file 'nixos-config' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at «none»:0: (source not available)
[1]    52351 IOT instruction (core dumped)  nixos-option


run Gratipay 166

165

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/25439083-run-gratipay-166?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github).

Took a pass through https://github.com/gratipay/applications/issues.

[gratipay] $ heroku config -s | run_dammit /dev/stdin -c ./env/bin/python ./bin/migrate-tips.py
'introtopython' already migrated.
'duo' already migrated.
'sudo-room' already migrated.
'nuvola-player' already migrated.
'cyberpipe' already migrated.
'mojolicious' already migrated.
'pageres' already migrated.
'Gratipay' already migrated.
'citizens-cooperative' already migrated.
'catapultpgh' already migrated.
'sudomesh' already migrated.
'perl-maven' already migrated.
'exercism' already migrated.
'devdocs' already migrated.
'weblate' already migrated.
'sublimelinter' already migrated.
'pillow' already migrated.
'simplytest-me' already migrated.
'json-resume' already migrated.
'encommuns' already migrated.
'perlweekly' already migrated.
'cuberite' already migrated.
'totalism-hackbase' already migrated.
'bevry' already migrated.
'cpan-io' already migrated.
Migrated 0 tip(s) for 'shields'
Migrated 0 tip(s) for 'kivy'
'gammu' already migrated.
Migrated 0 tip(s) for 'knopproject'
'the-charis-project' already migrated.
Migrated 0 tip(s) for 'libravatar'
'debian-lts-by-freexian' already migrated.
'doit' already migrated.
'northern-plains-athletics' already migrated.
'the-mountain-spirit' already migrated.
'jshttp' already migrated.
'aegirproject' already migrated.
'image-optim' already migrated.
'survivejs' already migrated.
'nanoc' already migrated.
'sailor' already migrated.
'cider' already migrated.
'wdrl' already migrated.
Migrated 0 tip(s) for 'detexify'
'cpan-book' already migrated.
'trakt-for-plex' already migrated.
'act' already migrated.
'esdiscuss' already migrated.
'rhombus-tech' already migrated.
Migrated 0 tip(s) for 'kriasoft'
'jsbin' already migrated.
'ucoin' already migrated.
Migrated 0 tip(s) for 'piloting-postdevelopment'
Migrated 0 tip(s) for 'rtail'
Migrated 0 tip(s) for 'tech-store-club'
'davorg' already migrated.
'zed' already migrated.
'endless-parentheses' already migrated.
'gkz-open-source' already migrated.
Migrated 0 tip(s) for 'matlab-schemer'
Migrated 18 tip(s) for 'webtorrent'
Done.

Backup started. I'm on a slow connection, so it's going to take a while ...

Backup finished and verified.

Script started.

Script ran for two minutes (0:02:58.094770).

Decline rate today was 9% (8 / 88).

> => update exchange_routes set fee_cap=20 where fee_cap is null and network='paypal';
UPDATE 36

MassPay done for 40 users. Posting back ...

Escrow shuffled.

Logs uploaded in https://github.com/gratipay/logs/commit/dd453389ae21f9592610be8190c802131eb275a3.

MassPay posted back.


add toJSON function

In this PR,add toJSON function.
Without toJSON function,once invoke JSON.stringify,it will return like
{buffer: {type: "Buffer", data: [0, 0, 0, 0, 85, 93, 158, 131]}, offset: 0}

It is better to specify the major version of biginteger module in package.json to avoid conflicts.

@tematema done
@broofa would you review this PR?


support for _integrate_forces?

From what I've been able to tell this state chart implementation doesn't appear to work with _integrate_forces? If I'm understanding that correctly, is that something that could be added to the State class?
_integrate_forces is used primarily when controlling StaticBody2D objects in kinematic mode.

Hi @tupcakes , could you elaborate on your use case for this?
Right now the approach of using signals this library uses does not lend itself very well to _integrate_forces because _integrate_forces is a member method of RigidBody2D/RigidBody3D and as such the state machine has no way of simulating this call because it is called at the right time by the RigidBody2D/RigidBody3D itself.
What could work is some kind of dispatching mechanism that reads current state machine state and then dispatches to a certain method but that's a lot less elegant as it's basically a glorified switch/case and it would require you to either refer to states in your code (which I think miss the point of using this library) or repeat your state structure somehow in a secondary node structure which isn't great from a maintenance point of view.  Both options are not very appealing.
But in any case, if you could tell me a bit more about your use case, maybe a better way comes to mind, so I'd very much appreciate if you could give me some details. Thanks a lot!

Thanks for the explanation. I cannot see however where the state charts come into play here. Does the ship have any states that would affect how it moves physically? Do states limit what input is available?

I'm closing this as I would need more information to derive any meaningful action from it and it's been inactive for two weeks. Please reopen it if you want to continue the discussion.


[Feature] Support Client Status Discovery Service (CSDS)

Title: [Feature] Support Client Status Discovery Service (CSDS)
Description:
Client Status Discovery Service (CSDS) is a service that exposes xDS config of a given client. It’s commonly used to query control planes for the synced xDS config of a particular sidecar proxy. However, it can also be used to query Envoy for its received xDS configuration.
As a xDS client, Envoy can dump xDS config and expose internal status of each xDS resource (https://github.com/envoyproxy/envoy/pull/13121). The xDS config dump will share the same proto definition as admin interface. It will be an alternative way of getting config dump. As for internal status, it describes if the Envoy accepts or rejects or pending on a specific xDS resource.
Relevant Links:

For Googler, go/grpc-csds
CSDS proto definition

CC @htuch for more potential benefits.

It seems like we should ultimately converge on one dump/status API. Long term should we deprecate the ad-hoc config dump admin API in Envoy and move to CSDS? cc @dschaller @htuch

Always a fan of moving bits of admin to RPC, but I think you've pointed out in the past that there is a significant class of users that wants a simple curl interface? Are you thinking we might have some way to still curl the CSDS output through admin?

I'm suggesting some sharing of protos at minimum, and possibly yes supporting a REST fetch of the API (which is easily supported with gRPC unary requests).


Can I use NCRF for punctuation restoration?

 And after 20 epochs training, decode result almost give me label O.
Here is my training logs:
Seed num: 42
MODEL: train
Load pretrained word embedding, norm: False, dir: /data/hieunv11/resources/cc.vi.300.vec
Embedding:
     pretrain word:1, prefect match:0, case_match:0, oov:175965, oov%:0.9999943170839821
Training model...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
DATA SUMMARY START:
 I/O:
     Start   Sequence   Laebling   task...
     Tag          scheme: NoSeg
     Split         token:  |||
     MAX SENTENCE LENGTH: 250
     MAX   WORD   LENGTH: -1
     Number   normalized: True
     Word  alphabet size: 175966
     Char  alphabet size: 313
     Label alphabet size: 5
     Word embedding  dir: /data/hieunv11/resources/cc.vi.300.vec
     Char embedding  dir: None
     Word embedding size: 1
     Char embedding size: 50
     Norm   word     emb: False
     Norm   char     emb: False
     Train  file directory: data/punc_500k/train.txt
     Dev    file directory: data/punc_500k/dev.txt
     Test   file directory: data/punc_500k/test.txt
     Raw    file directory: None
     Dset   file directory: None
     Model  file directory: data/punc_500k/Model_500K_l60_e5_b128_lr0.015
     Loadmodel   directory: None
     Decode file directory: None
     Train instance number: 500000
     Dev   instance number: 100000
     Test  instance number: 100000
     Raw   instance number: 0
     FEATURE num: 0
 ++++++++++++++++++++++++++++++++++++++++
 Model Network:
     Model        use_crf: True
     Model word extractor: LSTM
     Model       use_char: True
     Model char extractor: LSTM
     Model char_hidden_dim: 100
 ++++++++++++++++++++++++++++++++++++++++
 Training:
     Optimizer: SGD
     Iteration: 20
     BatchSize: 128
     Average  batch   loss: True
 ++++++++++++++++++++++++++++++++++++++++
 Hyperparameters:
     Hyper              lr: 0.015
     Hyper        lr_decay: 0.05
     Hyper         HP_clip: None
     Hyper        momentum: 0.0
     Hyper              l2: 1e-08
     Hyper      hidden_dim: 200
     Hyper         dropout: 0.5
     Hyper      lstm_layer: 2
     Hyper          bilstm: True
     Hyper             GPU: True
DATA SUMMARY END.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
build sequence labeling network...
use_char:  True
char feature extractor:  LSTM
word feature extractor:  LSTM
use crf:  True
build word sequence feature extractor: LSTM...
build word representation...
build char sequence feature extractor: LSTM ...
build CRF...
   Time: 2048.75s, speed: 244.05st/s,  total loss: 60318.23893737793
totalloss: 60318.23893737793
Right token =  4873121  All token =  5369601  acc =  0.9075387538105718
Dev: time: 316.09s, speed: 321.13st/s; acc: 0.9075, p: -1.0000, r: -1.0000, f: -1.0000
Right token =  4876656  All token =  5369818  acc =  0.9081603883036632
Test: time: 346.18s, speed: 292.97st/s; acc: 0.9082, p: -1.0000, r: -1.0000, f: -1.0000

What am i wrong?

This is not a Segmentation or NER problem, so the P/R/F metric is not proper here. You need to set ‘seg=False’ in the configuration file.

 Double check the format of pretrained embeddings.


write JS function to create post request for creating a new tag

https://github.com/otekolste/answer-hive/pull/74


Add pagination support to IPAM Endpoint

Description
Added pagination support to IPAM Endpoints
Release Note
ENHANCEMENTS

* Add pagination support to IPAM Endpoint

Closing due to no response and addition to old-style client


Update MAINTAINERS

Proposed Changes

Update MAINTAINERS list
Types of Changes

Non-breaking change to MAINTAINERS file
Verification

Ensure the changes are sane during PR review
Linked Issues

Not required.
User-Facing Change

NONE

This should not affect end users, only maintainers.
Further Comments

Signed-off-by: David Nuzik<EMAIL_ADDRESS>

Codecov Report

Merging #3744 (f02e439) into master (09457a5) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3744   +/-   ##
=======================================
  Coverage   11.56%   11.56%
=======================================
  Files         136      136
  Lines        8809     8809
=======================================
  Hits         1019     1019
  Misses       7569     7569
  Partials      221      221

Flag
Coverage Δ

inttests
0.62% <ø> (ø)

unittests
11.28% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09457a5...f02e439. Read the comment docs.


665 - Extend IPunchItemUpdatedV1 w AffectsCompletionStatus

https://github.com/equinor/cc-toolbox/issues/665

This PR is not merged. The issue is fixed in Main API in https://statoildeveloper.visualstudio.com/Statoil.NextGenProCoSys/_git/ProCoSysWebApi/pullrequest/9944


Request a limited number of messages at a time

Is it possible to request a range of messages?
Say I want to process a batch of 1000 messages at a time.

Now I'm looking for the same thing now. @vjpr did you succeed?

https://github.com/nats-io/nats-streaming-server/issues/229#issuecomment-347590905


API-Docs: All links to component implementations are broken.

In the API docs, every "check the implementation of the component for more detail" links at the bottom of each page is broken.

[X] The issue is present in the latest release.
[X] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
Links are broken 😥

Expected Behavior 🤔
Links take you to the component source code. 😍
Steps to Reproduce 🕹
Steps:

Visit  https://material-ui.com/api/checkbox/
Scroll to bottom
Click on "If that's not sufficient, you can check the implementation of the component for more detail."

Context 🔦
Did some digging and it looks like the links are
// current
https://github.com/mui-org/material-ui/blob/master/packages/material-ui/src/Checkbox/Checkbox.js
                                                            ^^^^^^^^^^^

and they should be
// expected
https://github.com/mui-org/material-ui/blob/master/packages/mui-material/src/Checkbox/Checkbox.js
                                                            ^^^^^^^^^^^

The links are defined in the JSON files in:  material-ui/docs/pages/api-docs/ (link)
You can see an example of one on Line 42 of checkbox.json
// checkbox.json
{
  // ...
  "filename": "/packages/mui-material/src/Checkbox/Checkbox.js",
}

Where it seems they are correct, but the pages are still wrong? So I guess someone needs to re-publish the docs.
Your Environment 🌎
I'm on chrome, but it's not particularly relevant.

Thanks for the feedback.
This is an unfortunate side-effect of using no permalinks. We should really just use permalinks in future deploys. We can't do anything about past deploys though unfortunately.

Duplicate of #23406?

Duplicate of #23406?

No.

Unfortunately, we can't fix this for older deploys. We can fix it for for new deploys for the v4.x branch. But documentations for earlier versions will remain broken.

Closed by #28358


Pass token by Header[X-Access-Token]

Currently, the package has only authenticated a user on the basis of "HTTP_AUTHORIZATION". Sometimes client passes the token in header['X-Access-Token']. So, we can use an api_settings variable which may be default as "HTTP_AUTHORIZATION" but can be changed to headers['X-Access-Token'] or something else. It'll provide some more flexibility.
What do you guys think?
If it is not possible then please tell me the way to authorize a user with headers[X-Access-Token'].

You need to change the JWTAuthentication.get_header method in authentication.py.
You can subclass it in your project and then just use your class in the AUTHORIZATION_CLASSES list.

It's standard to simply use the Authorization Bearer header for JWT tokens. Although, flexibility has been an issue in this library for many, and I'd consider adding it for sure. Shouldn't be much overhead.

Mind if I take a crack at this?
It seems like something that might be super useful to some:)

Sure @dgmouris go for it!

Hey folks,
I took a crack at this (sorry it took a while). Please let me know if this is suitable for the package, and if the tests/documentation fit the style of the package.
Thanks so much for your time!

@sandeepsajan0 You should be able to change the AUTH_HEADER_NAME to whatever you'd like now based on the PR above.
As well @Andrew-Chen-Wang should this issue be closed now?
Thank you maintainers for your hard work!

It'll come out in the 5.0.0/next release.
Cheers all!


feat(svc): resolve cluster name id op

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#751  👈
#750
#747
#740
#743
#741
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @AngelOnFira and the rest of your teammates on  Graphite


Feat/add shared state

Modified the script so that:

You can now select either use postgres or duckdb for the state db.
If you specify GATEWAY variable in .env var or -e GATEWAY=shared_state in docker run commands, you'll use a postgres db for the state db. It'll use duckdb otherwise.
I have a free tier managed postgres set up, so let me know if you want to use that to give this a try.

Also, this PR includes a small change this other PR made.

@db-tycoon-stephen I think we just need to add postgres configs in docker compose yml. I'm not at my desk but can take a look on Monday


Support variable byte no dictionary columns in Consuming segments

In realtime consuming segemnts, we don't support STRING or BYTES data type. We should be able to add it using the class MutableOffHeapByteArrayStore

Will work on this

PR https://github.com/apache/incubator-pinot/pull/4791 merged fixing this issue


Calibration problems with the WiiMotionPlus

Hi ! First of all thank you for your work !
I encountered a problem while using the wiimotionplus extension: when I play your demo scene, the wiimote slowly unsyncs with the virtual model that seems to keep rotating even while the actual wiimote is left on a table without moving.
I calibrated it with the apropriate button of your GUI and reset the offset multiple time.
Plus it seems that my movments are 1/4th of what happens onscreen. (I tried compensating that doing "wmpOffset += offset * 4f" in the Update funtion of WiimoteDemo.cs" it seems to work aproximatly but the perpetual rotation I described above keeps ruining everything !
I am currently woking on a 64bit version of Windows 10 with Unity  5.2.0f3.

Make sure that when you calibrate it it is left face down - if it is face up you can't guarantee that it is completely still (due to the rounded bottom).
Even then, the Wii Motion Plus hardware is not 100% accurate.  Because it measures change in angle (not absolute angle), floating point error slowly desyncs the virtual model from the real orientation.  So you can get a pretty good short-term approximation, but expect to re-sync often.  In practice what you can also try is to reference the accelerometer.  When the accelerometer is relatively still you can assume that it is pointing in the direction of gravity (down).  So you can use this as a reference.
If this isn't your problem (that is, the virtual model never rotates at the correct speed) it is a problem with your hardware (IE your bluetooth card / dongle) or the OS (Windows 10 has been pretty hit or miss for me, but Windows 7 works correctly).  I can say with certainty that the Wiimote reports at a rate of 95 updates per second.  However, often on Win10 I only get 30 UPS (this seems in line with your estimates).  One thing you can try is to dynamically measure the incoming report rate and change the multiplier here from 95 to whatever rate you measure (30 for example).  Of course, the virtual model will not be as accurate though (because you are polling the Wiimote at a slower rate).
You can also get a DolphinBar which has custom drivers specifically for Wiimotes.  This should alleviate any driver problems.

I'm closing this issue for bookkeeping / organizational purposes. If you have any further inquiries or questions feel free to ask me to reopen this issue or create a new one.


Collapse TestPlan/TestPlanner/TestPlanBuilder/TestEnvironment

Collapse TestPlanBuilder, TestPlanner and TestEnvironment into TestPlan. TestPlanner.createPlan was left in place as an entry point to avoid touching all the other files that use it at least for the moment.

This doesn't work in CI, so I need to figure out what's different there.

With the latest changes to lua-apps, all the tests there pass with this new code.

This changes the behavior of duplicate it blocks (#93) but doesn't really resolve that issue quite yet.


Add locale selector to log in page

The goal of this ticket is to add the Locale Selector to the login page of dashboard.
Currently it is only available in the slide-in menu once you are logged in (bottom-right of the slide-in).
I suggest adding it to the bottom-left of the login page - you should be able to leverage the control straight from the slide-in menu - see: TopLevelMenue.vue around line 362. You will need to extract that into a new component that can be used both in the TopLevelMenu and the login screen (shell/pages/auth/login.vue).
On the login screen, we don't know who the user is, so we can't retrieve their locale preference from the backend.
I suggest that we cache the locale in browser local storage
On the login screen:

If there is no cached locale, we default to English, otherwise we use the locale that has been cached
When the user changes the locale on the login screen, we update the cache and store in the app that the locale was changed
Once the user logs in, we will have their locale preference from the backend
if the locale was changed on the login screen, we update the user preference with the new locale - this will then be stored by the backend
If the locale was not changed, we update the cached locale in the browser with the value from the user preference if this is different to what is currently cached

Use case: Testing Local in two browser
Browser 1 Status already logged in, currently on the index page( current local Eng)
Browser 2  Login page changed language to Chines => Logged in current local Chines
Browser 1 Refresh home page still load local Eng ( but pref is changed toChines in browser 2 ) it should load local from the backend.

tested on v2.6-head (68215bc):

language selection is available on the login page - pass
with multiple windows/tabs open, if the language is changed in one session, it is updated for all sessions -- pass
with multiple browsers/incognito sessions open, changes in one session do not update other sessions --TBD
@richard-cox or @bisht-richa could I get confirmation that this is intended behavior? I assume it is, but based on this line from the original ticket, I was a little confused

if the locale was changed on the login screen, we update the user preference with the new locale - this will then be stored by the backend

reopening for clarification on the above comment

@slickwarren Regarding: with multiple browsers/incognito sessions open, changes in one session do not update other sessions
No - you have to refresh the browser.

closing based on the above conversation and testing


changing layers on export does not keep the export validated

On export if you have an AOI defined and you change layers the current settings are not being applied to the new layer for validation.

Fixed.


Advanced Paste: When "AI" is not enabled, don't show it at all

Description of the new feature / enhancement

I don't (and won't) enable AI features, the bar at the top of the advanced paste dialog shouldn't be shown
Scenario when this would be used?
Thankfully, AI is not enabled by default in Advanced Paste, so this change would mean the bar is also not shown by default
Supporting information
No response

Of which issue is this a duplicate?
All the other issues mentioned (2 of them) have been closed with the same excuse, if they are all duplicates which one is the original issue?


No stats, but 0 scrape errors

I am running this from the docker container. Prometheus is showing 0 haproxy cvs scrape errors, but the only haproxy stats I see are up, haproxy_exporter_csv_parse_failures, and haproxy_exporter_total_scrapes. Is there something else I should be doing to get the haproxy metrics to show?

@jmstone617 I think that should work. How are you running the HAProxy exporter? And when you go to the configured CSV endpoint on the haproxy, you get working metrics output?

Please reopen if this is still an issue.


Dev15 build

Disable warning for catch.hpp that happens on dev15
Use MSBuildToolsVersion to check for which toolset to use when using msbuild15 directly

This change is

@curtisman I am wondering why VisualStudioVersion is not set in msbuild15. When I tried to build from command line, it complained on the toolset.


Update ug image

Codecov Report

Merging #274 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master     #274   +/-   ##
=========================================
  Coverage     83.92%   83.92%
  Complexity     1101     1101
=========================================
  Files           148      148
  Lines          3209     3209
  Branches        391      391
=========================================
  Hits           2693     2693
  Misses          421      421
  Partials         95       95

Impacted Files
Coverage Δ
Complexity Δ

...t/logic/commands/attendance/StatisticsCommand.java
93.61% <100.00%> (ø)
15.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ee6400...5b12d95. Read the comment docs.


22 New Auth Middleware

Ref #22
Added middleware which reads the user & api_key from the ZerigoDNS.config every time it is executed.  This allows for the configuration to be changed by the user after the client has been instantiated.

looks good


url not null exception

PDOException in Connection.php line 408:
SQLSTATE[23000]: Integrity constraint violation: 19 NOT NULL constraint failed: notifications.url

\Notifynder::loop(User::all(), function(NotifynderBuilder $builder, User $receiver) use($user) {
            $builder->category('user.registered')
                ->from($user)
                ->to($receiver);
        })->send();

fixed, merged, tagged, done, closed.


Remove backports of JDK8 classes

Motivation:
Java8 is out now for some time and JDK7 is no longer supported officially. We should remove all our backports and just use what the JDK provides us. This also will allow us to use intrinsics that are offered by the JDK implementations.
Modifications:
Remove all backports of jdk8 classes.
Result:
Use what the JDK offers us. This also fixes [#5458]

@CodingFabian FYI...

Cherry-picked into 4.1 (84188395be36893b214fcf63cea1544e35b2a654) and 4.0 (945addc)


Don't run findbugs and checkstyle as default during the ITs

As mentioned in #1184 findbugs and checkstyle slow down the test execution. I therefore removed the from the verify maven phase. They are still executed on travis, so if we introduce a massive issue in a PR travis should fail and report about it.
BTW: Findbugs seems to have issues with the Java 9 build.

This PR should be good to merge now. The speed of mvn clean test verify is no back to normal. On demand you can run mvn clean test verify checkstyle:check findbugs:check  locally to execute findbugs and checkstyle. These will be executed on travis as well.

Thanks!


info.archinnov.achilles.configuration.ArgumentExtractor#initSession should use the session only and not the cluster

Hi
when passing a native session, info.archinnov.achilles.configuration.ArgumentExtractor#initSession shouldn't call connect on the cluster and therefore should use orElseGet instead of orElse

Ok but if you use orElseGet you need to pass in a Supplier<Session>
So the question is what to put in the Supplier ? cluster.connect() ?

cluster::connect yes which will be called lazily instead of being called whatever is set as session
assuming you pass the session as param, current code does:
session = get(SESSION);
if (session == null) {
    session = cluster.connect();
} else {
    cluster.connect();
}

vs (orElseGet())
session = get(SESSION);
if (session == null) {
    session = cluster.connect();
}

Ok I get your point, the orElse() block is evaluated right away even if the Session object is present. Can you please do a small PR to fix this ?

here it is https://github.com/doanduyhai/Achilles/pull/286


Ubuntu 20.04.4 Environment Locks Up While Cloning a Repo (Github-hosted runner)

Description
I have an action that sometimes completes in three hours, but often locks-up at the exact same spot while cloning a repository. When this lock-up occurs, it resolves itself consistently after 3 hours and 50 minutes. This lock-up causes my action to fail as it pushes the total action time beyond 6 hours. Git is used to clone the repo via a script in another repo. Hundreds of other repositories are cloned before and after this particular repository is cloned. The same mirror is used for other repositories.
I can't replicate this on my local machine and I never observed this when I was using self-hosted runners. I've upgraded the github-hosted runner to the newly released Ubuntu 22.0.4 environment for now to see if anything changes. Perhaps this has something to do with the network stack, cloud provider, or some other VM resource limitation.
I can't identify exactly what is locking up at this time (git, the virtual environment, etc).
EDIT: The first run with 22.04 did not lock up. The clone step in question took only 13 seconds.
Platforms affected

[ ] Azure DevOps
[X] GitHub Actions

Virtual environments affected

[ ] Ubuntu 18.04
[X] Ubuntu 20.04
[x] Ubuntu 22.04
[ ] macOS 10.15
[ ] macOS 11
[ ] macOS 12
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
20220729.1
Is it regression?
No
Expected behavior
No lock-up occurs
Actual behavior
3.8 hour lockup sometimes occurs while cloning a repository
Repro steps

Run action
Sometimes the action takes 3.8 hours to clone a particular repository, and sometimes it does not.

@bhjelstrom hello!
Could you please provide real examples of the problem so we take a look.

I now have 4 basically identical workflows (Ubuntu 22.0.4) running on every commit. Each workflow will sometimes pause for the aforementioned 3.8 hours while downloading ncurses. Through hundreds of self-hosted and local builds I have never seen this. Note the time difference:
2022-08-04T20:52:30.0232682Z >>> Downloading ncurses  6.1
2022-08-05T00:43:04.0877252Z >>> Extracting ncurses 6.1

Sometimes when I try to access the logs of workflows that are stuck on this step, this is all I see:
2022-08-08T18:29:41.5275692Z Requested labels: ubuntu-22.04
2022-08-08T18:29:41.5275736Z Job defined at: <redacted>
2022-08-08T18:29:41.5275754Z Waiting for a runner to pick up this job...
2022-08-08T18:29:42.9712121Z Job is waiting for a hosted runner to come online.
2022-08-08T18:29:46.2681587Z Job is about to start running on the hosted runner: GitHub Actions 2 (hosted)

Is there another log file I can extract from the github-hosted runners? Not sure what else to provide.

@bhjelstrom do you have any updates? And maybe you have a repo that we can clone and experiment with?

I believe this has been fixed by reducing in the number of make jobs from 4 (via nproc) to 2. The build completes an entire hour earlier as well. This implies that the old builds were locking up more than once.
Most likely cause: My build required too much memory for the 7GB runner and sometimes spilled over into swap. I should have seen this coming, but resource warnings for action runners might generally add value.
Also, I would definitely pay for bigger/faster runners to help keep our workflows in github.

@bhjelstrom thanks for the update!
More powerful runners are on our radar but I can't provide ETA when they will be available.
I'm going to close the issue please let me know if you have any concerns.
Thank you!

@bhjelstrom beta is announced by the way https://github.blog/2022-09-01-github-actions-introducing-the-new-larger-github-hosted-runners-beta/

@miketimofeev Great timing! I signed up for the beta. Thanks.


Updates for Aff

I'm going around prepping PRs for Aff v4.

Could you update this PR sometime? Thanks!

In the meanwhile I pushed up https://github.com/justinwoo/purescript-spec/commit/3a0ac612af654ff74e9183fd1d46b8ef8b505f8f, but it needs downstream Pipes and IO updates.

@justinwoo pipes and io are both updated now (both on v5 now). Would you be able to update? @owickstrom which version should we release this under? Considering the major changes to aff, do we have to do a major bump as well?

@felixSchl I think this will warrant a new major.

Pushed an update with a rebase here: https://github.com/justinwoo/purescript-spec/commit/b558e0d613a39a81da8548c0129968ecae95722f

@justinwoo would you like to create a new PR?

Merged via #64. Thanks @natefaubion and @justinwoo


Issue #88 Partial Fix

This is a conversion of the brainwave tab from pysimplegui to a version in qt. This tab has attempted functionality, (connecting, reading brainwaves, etc) but nothing seems to work. So for now the purpose of this pull request is to provide a gui window for others who want to fix this and make it functional with the rest of the project.

The original ticket will remain open for some of the teams that is trying to finish a fully functional version to pull request for merging.


Logo and launch screen are needed

Hello @kindraywind
I'm a graphic designer and I design logos for open source projects.
Usually I ask the following questions to assist me in the creative process:
Do you have something in your mind for the logo?
Do you want something specific for the logo?
What colors do you want in the logo?
With your answers I will design a logo for your project and if you like it you can keep it and used.
Best Regards
Zuur

Hello @zuuritaly !
We really appreciate your kindness.
Unfortunately, I've forgotten to close this issue at the time I got the logos from another contributor. 😭
By the way, I think my fellow would love to finally have a logo for his networking library 👍


Add User Stories and Use Cases to Developer Guide

Temporary for now, more use cases need to be added.

Codecov Report

Merging #61 (330ef6e) into master (bcc64e7) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #61   +/-   ##
=========================================
  Coverage     66.64%   66.64%
  Complexity      400      400
=========================================
  Files            74       74
  Lines          1346     1346
  Branches        143      143
=========================================
  Hits            897      897
  Misses          404      404
  Partials         45       45

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bcc64e7...330ef6e. Read the comment docs.


Update google ads api to v15

Migrating google ads API from v14 to v15 with new consent requirments for the consent field.
API v14 will be sunset end of May 2024 [see timetable].
API v15 will function until September 2024.

Hi.
can someone please check
https://github.com/looker-open-source/actions/issues/604
We face the problem, that the action is not working anymore because there is no authentication possible since the sunset of API v14.
According to my basic understanding, this is a result of hard-coded API version in api_client.ts
https://github.com/looker-open-source/actions/blob/1adbb8acc7b02b402cdff3cef713a466a7b079e0/src/actions/google/ads/lib/api_client.ts#L132
We faced the same issue two months ago with the Facebook Customer Match Action.
Can somebody please have a look because the action is not working atm.
THX!


Update license badge to Apache 2.0

Updating License Badge to Apache 2.0 since that's the actual license version

Codecov Report

Merging #18 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #18   +/-   ##
=========================================
  Coverage     18.55%   18.55%
  Complexity       57       57
=========================================
  Files            23       23
  Lines           539      539
  Branches         89       89
=========================================
  Hits            100      100
  Misses          434      434
  Partials          5        5

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb20b4c...6cda397. Read the comment docs.


[FLINK-32021][Connectors/Kafka] Improvement the Javadoc for SpecifiedOffsetsInitializer and TimestampOffsetsInitializer

What is the purpose of the change
As described in [FLIP-288](https://cwiki.apache.org/confluence/display/FLINK/FLIP-288%3A+Enable+Dynamic+Partition+Discovery+by+Default+in+Kafka+Source), Current JavaDoc does not fully explain the behavior of OffsetsInitializers. When the partition does not meet the condition, there will be a different offset strategy.
This may lead to misunderstandings in the design and usage.
Brief change log
Add to SpecifiedOffsetsInitializer: "Use Specified offset for specified partitions while use commit offset or Earliest for unspecified partitions. Specified partition offset should be less than the latest offset, otherwise it will start from the earliest."
Add to TimestampOffsetsInitializer:Initialize the offsets based on a timestamp. If the message meeting the requirement of the timestamp have not been produced to Kafka yet, just use the latest offset.
Verifying this change
no code changes.

Thanks for the PR @loserwang1024 and @RamanVerma for the reviews!
Merging this.


Update for Cabal >= 2.2 and base >= 4.11

This fixes #2, which is Setup.hs not working for Cabal > <IP_ADDRESS>, and an issue without a ticket, which is base >= 4.11 breaking the Monoid instance.
I have not tested anything except compilation with Cabal <IP_ADDRESS> and base <IP_ADDRESS> (GHC 9.4.7). It should also work on base 4.18/GHC 9.6, and I've tried to maintain backward-compatibility with older Cabal/GHC/base versions, too.
While I didn't test compilation with older versions, my HLS seemed to be smart enough to pick up on the CPP macros and warn me about, e.g. mappend being (<>) (or vice versa, I forget) only on the older version when I copy-pasted a small piece of code. That same chunk of code didn't have the warning on the #ifdef pertaining to the version I had installed. That was pretty cool.

See discussion in #1 for why I've now closed this.


CAUSEWAY-3834: Factor out mutable Aspects from ObjectSpecification and its Loader

[x] ObjectSpecificationMutable interface to extend ObjectSpecification encapsualting mutable aspects
[x] ObjectSpecificationLoaderInternal to extend ObjectSpecificationLoader encapsulating mutable aspects
[x] introduce ObjectSpecificationRecord, to ultimately model an unmodifiable ObjectSpecification
[x] ObjectSpecificationRecord to implement Specification
[ ] ObjectSpecificationRecord to implement Hierarchical
[ ] ObjectSpecificationRecord to implement ObjectActionContainer
[ ] ObjectSpecificationRecord to implement ObjectAssociationContainer

merging all the refactoring work, prior to reaching the initial goal of providing an immutable ObjectSpecification variant

Shouldn't lombok's @nonnull be replaced by the SpringBoot variant?

As I see it, for the various applib modules yes. For implementation/integration modules no. But that is not the focus of this PR/task.


ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type list).

Im getting this valueError in colab gpu env with keras tensorflow ver 2.
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type list). on running the following code :
def ctc_lambda_func(args):
y_pred, labels, input_length, label_length = args
# the 2 is critical here since the first couple outputs of the RNN
# tend to be garbage:
# print "y_pred_shape: ", y_pred.shape
y_pred = y_pred[:, 2:, :]
# print "y_pred_shape: ", y_pred.shape
return K.ctc_batch_cost(labels, y_pred, input_length, label_length)
---- CTC ----
labels = Input(name='the_labels', shape=[None], dtype=dtype)       # transcription data (batch_size * y_seq_size)
input_length = Input(name='input_length', shape=X_train_length.shape, dtype=dtype)  # unpadded len of all x_sequences in batch
label_length = Input(name='label_length', shape=label_length.shape, dtype=dtype)  # unpadded len of all y_sequences in batch
Lambda layer with ctc_loss function due to Keras not supporting CTC layers
loss_out = Lambda(function=ctc_lambda_func, name='ctc', output_shape=(1,))([y_pred, np.float32, Y_train, np.float32, X_train_length, np.float32, label_length, np.float32])
network_model = Model(inputs=[input_data, labels, input_length, label_length], outputs=y_pred)

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Wednesday, January 13, 2021, 09:30:59 PM GMT+5:30, Abhilash Mahendrakar<EMAIL_ADDRESS>wrote:

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Wednesday, January 13, 2021, 09:30:59 PM GMT+5:30, Abhilash Mahendrakar<EMAIL_ADDRESS>wrote:

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Could you please update the progress. Also, please confirm if you were able to access the dataset folder.
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Friday, January 15, 2021, 02:52:39 PM GMT+5:30, Neena Aloysius<EMAIL_ADDRESS>wrote:

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Wednesday, January 13, 2021, 09:30:59 PM GMT+5:30, Abhilash Mahendrakar<EMAIL_ADDRESS>wrote:

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Could you please update the progress. Also, please confirm if you were able to access the dataset folder.
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Friday, January 15, 2021, 02:52:39 PM GMT+5:30, Neena Aloysius<EMAIL_ADDRESS>wrote:

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details
Thanks,
Neena Aloysius
Research Scholar
Dept. of CSE,
Amrita School of Engineering
Ph: +91-9061414992.
On Wednesday, January 13, 2021, 09:30:59 PM GMT+5:30, Abhilash Mahendrakar<EMAIL_ADDRESS>wrote:

@neenaloysius,
On running the given code snippet with TF v2.4, I am facing an error stating NameError: name 'dtype' is not defined. Please find the gist of it here.
In order to reproduce the issue reported here, could you please provide the exact TensorFlow version, the complete code and the dataset you are using. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details

@neenaloysius,
I do not see a Colab notebook or a dataset attached. Could you please share the links of it here? Thanks!

PFA the colab file. The dataset folder I have shared with you. Please let me  know if you need more details

@neenaloysius,
I do not see a Colab notebook or a dataset attached. Could you please share the links of it here? Thanks!

The colab file in zipped format:
CSLR (1).zip
This is the link for dataset:
https://drive.google.com/drive/folders/1kus7GxIYTkHFb1HJBLd_ALQGIGlQUSw5?usp=sharing

The colab file in zipped format:
CSLR (1).zip
This is the link for dataset:
https://drive.google.com/drive/folders/1kus7GxIYTkHFb1HJBLd_ALQGIGlQUSw5?usp=sharing

@neenaloysius,
I did not face the ValueError: Failed to convert a NumPy array to a Tensor on running the code with the TensorFlow v2.4.
However, I am facing a different error stating ValueError: Can not squeeze dim[1], expected a dimension of 1, got 3 for '{{node ctc/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Placeholder_1)' with input shapes: [?,3].. Please find the gist of it here. Thanks!

@neenaloysius,
I did not face the ValueError: Failed to convert a NumPy array to a Tensor on running the code with the TensorFlow v2.4.
However, I am facing a different error stating ValueError: Can not squeeze dim[1], expected a dimension of 1, got 3 for '{{node ctc/Squeeze}} = Squeeze[T=DT_FLOAT, squeeze_dims=[-1]](Placeholder_1)' with input shapes: [?,3].. Please find the gist of it here. Thanks!

Yeah, I slightly changed the code and this is the issue im facing now. Not sure which variable is causing the issue.

Yeah, I slightly changed the code and this is the issue im facing now. Not sure which variable is causing the issue.

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.
Thanks!


Decorator docs - "these aren't very good regexes, don't use them!" - Why?

The decorator docs include an example of decorator usage to highlight handles and hashtags. In a block of code, it's written:
// Note: these aren't very good regexes, don't use them!
const HANDLE_REGEX = /\@[\w]+/g;
const HASHTAG_REGEX = /\#[\w\u0590-\u05ff]+/g;

Why are these not very good regexes? Is it something performance-related?
I know this is not a DraftJS question, just providing links for any further regex-related reading for me would be highly appreciated.

Hey @vlopp!
Mostly they're just not very accurate.
For example:
const HANDLE_REGEX = /\@[\w]+/g; will also match email domains (e.g. it'll match bar in<EMAIL_ADDRESS>const HASHTAG_REGEX = /\#[\w\u0590-\u05ff]+/g will match anything with a # symbol in it (for example, "I'm #1!")
Neither of which is probably great for user-entered data.

Closing, let me know if you have additional questions :).


No such file or directory @ rb_sysopen /path_to_repo/.rubocop_todo.yml

Got error No such file or directory @ rb_sysopen /path_to_repo/.rubocop_todo.yml
When there is no .rubocop_todo.yml file in repo.
rubocop -v
0.34.2

ruby -v
ruby 2.1.6p336 (2015-04-13 revision 50298) [x86_64-darwin14.0]

I think you have the line
inherit_from: .rubocop_todo.yml

in your .rubocop.yml file. Then the error message is expected.

Exactly. Sorry.
Just a copy-paste yml file and miss this first line (usually it's shebang string in this place and I automatically skip it, when read file)


Can't start Sipi following instructions in README

If SIPI_VERSION is not set, I get an error message, but it's not clear what I should put in SIPI_VERSION. I tried develop, but that didn't work, either:
% cd git/knora-api/sipi
% make up
docker-compose up -d sipi
WARNING: The SIPI_VERSION variable is not set. Defaulting to a blank string.
ERROR: no such image: dhlabbasel/sipi:: invalid reference format
make: *** [up] Error 1
% export SIPI_VERSION=develop
% make up
docker-compose up -d sipi
Pulling sipi (dhlabbasel/sipi:2.0.1)...
ERROR: manifest for dhlabbasel/sipi:2.0.1 not found: manifest unknown: manifest unknown
make: *** [up] Error 1

Sorry about that.  PR is coming right away.

No worries!


*(engine): defines some infrastructure to better handle gRPC exceptions

What problem does this PR solve?

Issue Number: close #xxx
What is changed and how it works?
Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

Replaced by https://github.com/pingcap/tiflow/pull/7516.


Checkbox not working inside Charm

I can't get the checkbox to work when placed in a charm.  I can programmatically change the state of the chekbox, but I don't feel like having to write my own onclick handlers for such a simple task... There is already another issue open identifying the same problem but I have responded to that one on January 20th (almost a year ago now) and so far it has not been looked at.

please, create pen on codepen.io with your problem

Hi! I don't have a codepen account, but I am attaching a zip file that illustrates the problem. All you will have to do to recreate the problem is set the css and js references to point to files on your system.
CHECKBOX-TEST.zip

@swb0417 you can create an account on codepen.io absolutely free. I do not have the opportunity to deploy all the archives of anyone who uses Metro and has some problems with it. Sorry.

@olton I tried but I can't seem to properly link to metro that is hosted on my own site from within the code on codepen... Meanwhile I am pretty certain the problem has something to do with the fact that you are setting the "checked" property of one control (input type="checkbox") while responding to an event handler of another (span) control.
When I do this:

<input type="checkbox" checked="" id="switch1" onclick="tickThis(this)">

CHECKBOX

When I inspect the element in my tickThis function, like so:
function tickThis(ctrl) {
alert(ctrl.id);
};

the alert shows "clickme" even though to switch the state of the checkbox I will have to do this:
$('#switch1').prop("checked", false)

Do you have this problem against the latest version?

@swb0417 why do you have these two helper elements include the onclick handler?
``

CHECKBOX
Then the event pops up on a label that changes the state of the checkbox. Place onclick on the label

@swb0417 I do not understand the goal that you want to achieve :( Why set onclick on additional elements? Set onclick to label.

The onclick on the additional elements was to figure out/test which of the elements was actually firing the onclick event.
In the example above, the UI shows all 4 elements (label with 3 nested elements) as one switch/checkbox. To programatically check/uncheck the element you will have to set the property of the INPUT element, yet the onclick event fires from the SPAN element with class of "check".
The "goal" I am trying to achive is to get Metro checkboxes/switches working in charms. my question is, have you confirmed that checkboxes are fully functional in charms in the latest version of Metro.

To programmatically change the status of the checkbox, you need to change the attribute of the input, the onclick event is not generated. The state of the checkbox also changes when the user clicks on the label.