Cannot parse  schema

Describe the bug
Reconnecting an updated consumer application that had NOT changed any of the following:

Any connection or topic details
Any message object details used to generate the AvroSchema

This system was restarted from an update of application code unrelated to pulsar handling, and has two consumers.
At first both consumers were failing but after a restart one is OK and the other continues to fail with the error:
org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema

I could see the schemas using pulsar-admin, and they look exactly as they should, and the producers are tied to the same.
For the two consumar connections in this application they should have been binding to schema versions 1 & 2 respectively, but both had the error mentioning schema version 0
My connection code:
consumerIngressV1 = pulsarClient.newConsumer(AvroSchema.of(IngressMessageV1.class))
    .subscriptionName(subscriptionName)
    .topic(topics)
    .subscribe();

The stacktrace:
2020-10-27 22:48:12,322 [pool-3-thread-8] INFO o.a.p.client.impl.schema.AvroSchema - Load schema reader for version(0), schema is : , schemaInfo: {
 "name": "",
 "schema": "",
 "type": "NONE",
 "properties": {}
} --
2020-10-27 22:48:12,323 [pool-3-thread-8] ERROR io.pethau.pl.PipelineService - Exception: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema - {} - {} - {} --
org.apache.pulsar.shade.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3973)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4957)
    at org.apache.pulsar.client.impl.schema.StructSchema.decode(StructSchema.java:107)
    at org.apache.pulsar.client.impl.MessageImpl.getValue(MessageImpl.java:293)
    at io.pethau.pl.PipelineService.lambda$run$2(PipelineService.java:103)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema
    at org.apache.pulsar.shade.org.apache.avro.Schema.parse(Schema.java:1597)
    at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1396)
    at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1384)
    at org.apache.pulsar.client.impl.schema.StructSchema.parseAvroSchema(StructSchema.java:182)
    at org.apache.pulsar.client.impl.schema.AvroSchema.loadReader(AvroSchema.java:99)
    at org.apache.pulsar.client.impl.schema.StructSchema$1.load(StructSchema.java:75)
    at org.apache.pulsar.client.impl.schema.StructSchema$1.load(StructSchema.java:72)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2276)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
    at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
    ... 11 common frames omitted

To Reproduce
Unable to re-produce.
Expected behaviour
Consumers connect to the topics using the latest version from before the restart.
Desktop (please complete the following information):

OS: Docker, standalone, v2.6.0

Additional context
None.

A log file showing the same issue encountered just now.
A restart of the application recovered, but seemingly wasn't able to recover itself.
thingping.log

@robshep Could you please provide more information about the schema configuration? Which schema compatibility policy are you using? how the schema of the topic looks like now(You can use the pulsar-admin schemas to get the current schema of the topic)

Thank you,
I'm afraid I don't know how to get the schema compatibility policy of a topic - please can you advise.  I don't see a cli command to show this)
On the few occasions this has happened, it typically works OK after restarting the application.
On once occasion it has failed to recover so I used a fresh broker cluster.  (in testing environment so no data lost)
You can see in the original post the NULL schema that is returned as a symptom of this problem,
whereas below is a proper one as logged upon connection.
  "name": "myorg/myapp/ingress",
  "schema": {
    "type": "record",
    "name": "IngressMessageV1",
    "namespace": "io.myapp.shared.pl",
    "fields": [
      {
        "name": "attributes",
        "type": [
          "null",
          {
            "type": "map",
            "values": "string"
          }
        ]
      },
      {
        "name": "timestamp",
        "type": "long"
      },
      {
        "name": "rawMessage",
        "type": [
          "null",
          {
            "type": "bytes",
            "java-class": "[B"
          }
        ]
      }
    ]
  },
  "type": "AVRO",
  "properties": {
    "__alwaysAllowNull": "true",
    "__jsr310ConversionEnabled": "false"
  }
}```

@robshep Thanks, seems the schema definition is not the problem, I think maybe the problem is the reader cache in the StructSchema.
@congbobo184 Could you please help take a look at this issue?

@robshep hello, I look up your log and are you consume more than one topic in your consumer?  it look like one topic does not have schema.

Hi, thanks for taking a look.
There are 3 topics in use in this deployment.
All have schemas.
None of this code, or the Payload objects have changed for many months.
I notice in App1 there is another topic -RETRY created presumably to handle redelivery.  I don't maintain a schema here.
The log above is from restarting App1 below but I have seen the same issue with the "Null Schema" also when restarting App3 also.
App1:
1 x consumer, 1 x producer
@Bean(destroyMethod = "close")
@Autowired
public Consumer<UpdateMessageV1> updateListener(PulsarUpdateListener pulsarUpdateListener) throws PulsarClientException {
        return pulsarClient().newConsumer(AvroSchema.of(UpdateMessageV1.class))
                .messageListener(pulsarUpdateListener)
                .subscriptionMode(SubscriptionMode.Durable)
                .subscriptionName("app-tp-updater")
                .negativeAckRedeliveryDelay(20, TimeUnit.SECONDS) // try again after 20 seconds
                .topic("persistent://myorg/myappB/update")
                .enableRetry(true)
                .subscriptionType(SubscriptionType.Shared)
                .subscribe();
    }

    @Bean(destroyMethod = "close")
    public Producer notifyExpiryProducer() throws PulsarClientException {
        return pulsarClient().newProducer(AvroSchema.of(ExpiryNotificationV1.class))
                .topic("persistent://myorg/myappB/notify:expiry")
                .create();
    }

App 2:
1 x producer
final Producer<IngressMessageV1> producer = pulsarClient.newProducer(AvroSchema.of(IngressMessageV1.class))
    .topic("persistent://myorg/myappA/ingress")
    .create();

App3:
1 x producer, 2 x consumer
producer = pulsarClient.newProducer(AvroSchema.of(UpdateMessageV1.class))
                        .producerName("myapp-tp-update-consumer")
                        .topic("persistent://myorg/myappB/update")
                        .create();

consumerTPNotifsV1 = pulsarClient.newConsumer(AvroSchema.of(ExpiryNotificationV1.class))
                .subscriptionName("app-tp-expiry")
                .topic( "persistent://myorg/appB/notify:expiry")
                .subscribe();

consumerIngressV1 = pulsarClient.newConsumer(AvroSchema.of(IngressMessageV1.class))
                .subscriptionName("myapp-ingress")
                .topic("persistent://myorg/myappA/ingress")
                .subscribe();

Thanks
Rob

@robshep could you please provide more detailed information from app3, log and which topic the consumer Cannot parse <null> schema

Thank you,
Attached is the application log from the first time this happened.
It isn't clear exaclty which topic the consumer finds a null schema, but you can see in this example it was both consumers.
On this occasion a plain restart recovered one of them (persistent://myorg/appB/notify:expiry) but the other (persistent://myorg/myappA/ingress) did not recover (same error of Null schema on each message received) and so I just trashed the standalone pulsar and started a new instance.
It has done it since, but a restart recovered them.
Thanks again.
main.log.2020-10-27.txt

if restart, this topic persistent://myorg/myappA/ingress not recover you can use admin api to check is the schema exist.

Hi,
This is still happening and we still need to keep restarting connected producers/consumers after updating our application.
Occasionally if 20+restarts does not clear it, we destroy the (docker standalone) broker and re-build.

I'm afraid we may not validate Pulsar into production  and will have to fall back to kafka :disappointed:

@congbobo184 All the expected topics appear using pulsar-admin at any stage in the process: before, during a failure, after etc.

Hi,
This is still happening and we still need to keep restarting connected producers/consumers after updating our application.
Occasionally if 20+restarts does not clear it, we destroy the (docker standalone) broker and re-build.

I'm afraid we may not validate Pulsar into production  and will have to fall back to kafka :disappointed:

@congbobo184 All the expected topics appear using pulsar-admin at any stage in the process: before, during a failure, after etc.

@robshep
Sorry to hear that,
AvroSchema has been in the production environment for a long time. It is a stable feature.
I think you can use one topic to test and use ./pulsar-admin schemas get persistent://tenant/namespace/topic --version to find out this version of schema in broker or provide a stable way to reproduce for me, I will reproduce it. include. (schema compatibility, all version schema, producer and consumer update order and update schema.)
I think this is not a problem that can prevent you from entering the production environment, we can try to solve this problem together. You can ping me in pulsar slack general channel. my clack name is CongBo.  Expect you to find me 🔢

@robshep
Sorry to hear that,
AvroSchema has been in the production environment for a long time. It is a stable feature.
I think you can use one topic to test and use ./pulsar-admin schemas get persistent://tenant/namespace/topic --version to find out this version of schema in broker or provide a stable way to reproduce for me, I will reproduce it. include. (schema compatibility, all version schema, producer and consumer update order and update schema.)
I think this is not a problem that can prevent you from entering the production environment, we can try to solve this problem together. You can ping me in pulsar slack general channel. my clack name is CongBo.  Expect you to find me 🔢

Thank you,
If the components are generally stable then there could be issues with my setup?
In this case I am grateful for your support, and will look you up on slack
With thanks
R
Get Outlook for iOShttps://aka.ms/o0ukef

From: congbo<EMAIL_ADDRESS>Sent: Monday, January 18, 2021 2:23:49 AM
To: apache/pulsar<EMAIL_ADDRESS>Cc: Rob Shepherd<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/pulsar] Cannot parse  schema (#8429)
@robshephttps://github.com/robshep
Sorry to hear that,
AvroSchema has been in the production environment for a long time. It is a stable feature.
I think you can use one topic to test and use ./pulsar-admin schemas get persistent://tenant/namespace/topic --version to find out this version of schema in broker or provide a stable way to reproduce for me, I will reproduce it. include. (schema compatibility, all version schema, producer and consumer update order and update schema.)
I think this is not a problem that can prevent you from entering the production environment, we can try to solve this problem together. You can ping me in pulsar slack general channel. my clack name is CongBo. Expect you to find me 🔢
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/apache/pulsar/issues/8429#issuecomment-761937645, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANXMU4WPGB3KXWTSILZTPTS2OLTLANCNFSM4THYHL3Q.

Thank you,
If the components are generally stable then there could be issues with my setup?
In this case I am grateful for your support, and will look you up on slack
With thanks
R
Get Outlook for iOShttps://aka.ms/o0ukef

From: congbo<EMAIL_ADDRESS>Sent: Monday, January 18, 2021 2:23:49 AM
To: apache/pulsar<EMAIL_ADDRESS>Cc: Rob Shepherd<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/pulsar] Cannot parse  schema (#8429)
@robshephttps://github.com/robshep
Sorry to hear that,
AvroSchema has been in the production environment for a long time. It is a stable feature.
I think you can use one topic to test and use ./pulsar-admin schemas get persistent://tenant/namespace/topic --version to find out this version of schema in broker or provide a stable way to reproduce for me, I will reproduce it. include. (schema compatibility, all version schema, producer and consumer update order and update schema.)
I think this is not a problem that can prevent you from entering the production environment, we can try to solve this problem together. You can ping me in pulsar slack general channel. my clack name is CongBo. Expect you to find me 🔢
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/apache/pulsar/issues/8429#issuecomment-761937645, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANXMU4WPGB3KXWTSILZTPTS2OLTLANCNFSM4THYHL3Q.

Hey,
I am facing a similar issue.
I am running pulsar 2.7.1 and I have a topic with AVRO schema where my producers push avro messages and consumer receives and process the avro data. My consumer application receives the message through pulsar client's MessageListener. All my topic messages are tagged with the latest avro schema version. And I could see that schema in the Pulsar cluster's SchemaRegistry.
But I am facing the following issue when my consumer application starts and consumes message.
org.apache.pulsar.shade.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache.get(LocalCache.java:3951) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) 	at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader.read(AbstractMultiVersionReader.java:86) 	at org.apache.pulsar.client.impl.schema.AbstractStructSchema.decode(AbstractStructSchema.java:60) 	at org.apache.pulsar.client.impl.MessageImpl.getValue(MessageImpl.java:301) 	at org.apache.pulsar.client.impl.TopicMessageImpl.getValue(TopicMessageImpl.java:143) 	at com.kohia.galaxy.listener.MessageListenerImpl.received(MessageListenerImpl.kt:20) 	at org.apache.pulsar.client.impl.ConsumerBase.callMessageListener(ConsumerBase.java:882) 	at org.apache.pulsar.client.impl.ConsumerBase.lambda$triggerListener$5(ConsumerBase.java:862) 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 	at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 	at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema 	at org.apache.pulsar.shade.org.apache.avro.Schema.parse(Schema.java:1597) 	at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1396) 	at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1384) 	at org.apache.pulsar.client.impl.schema.util.SchemaUtil.parseAvroSchema(SchemaUtil.java:46) 	at org.apache.pulsar.client.impl.schema.reader.MultiVersionAvroReader.loadReader(MultiVersionAvroReader.java:53) 	at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader$1.load(AbstractMultiVersionReader.java:52) 	at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader$1.load(AbstractMultiVersionReader.java:49) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) 	at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) 	... 18 common frames omitted
The above exception occurs for all the message that is received by the consumer.
But when I restart my consumer application, it starts working fine and the above exception is not occurring.
This issue occurs randomly when I restart my consumer application and when I again restarted it works fine and all the message are processed properly.
Here is my subcription code,
pulsarClient             .newConsumer(AvroSchema.of(MyMetric::class))             .topic(topicName).subscriptionName(subscriptionName)             .subscriptionType(SubscriptionType.Shared)             .receiverQueueSize(localQueueMsgCount)             .messageListener(myMessageListener)             .subscribe()
Here is my Listener code(short version),
import org.apache.pulsar.client.api.Consumer import org.apache.pulsar.client.api.Message import org.apache.pulsar.client.api.MessageListener import java.net.InetAddress class MyMessageListener<T>() : MessageListener<T> {     override fun received(consumer: Consumer<T>, msg: Message<T>) {         try { 	    process(msg.value)             consumer.acknowledge(msg)         } catch (ex: Exception) {             consumer.negativeAcknowledge(msg)         }     }     fun process(metric:MyMetric){}  }
When I debug, I could find the following log at the time when my consumer application is working perfectly.
Load schema reader for version(2),  schema is : {"type":"record","name":"MyMetric","namespace":"com.example","fields":[{"name":"assetId","type":"long"},{"name":"utctime","type":{"type":"string","avro.java.string":"String"}}]},  schemaInfo: {   "name": "mytenant/mynamesapce/mytopic",   "schema": {     "type": "record",     "name": "MyMetric",     "namespace": "com.example",     "fields": [       {         "name": "assetId",         "type": "long"       },       {         "name": "utctime",         "type": {           "type": "string",           "avro.java.string": "String"         }       }     ]   },  "type": "AVRO",  "properties": {     "__alwaysAllowNull": "true",     "__jsr310ConversionEnabled": "false"   } }
But at the time when my consumer application works weird, I could find the following log which doesn't have any schema info,
Load schema reader for version(2),  schema is : ,  schemaInfo: {   "name": "",   "schema": "",   "type": "NONE",   "properties": {} }
Can u guys please help me resolve this issue. We are already in production and we find it diffficult to restart our consumer applications each and every time when this issue occurs.
@robshep @congbobo184 Did you find any root cause for this issue or any way to fix it, if so kindly share so that it may help us.
Eagerly waiting for your inputs. :)

Hi
No, sadly I cannot recreate this issue in isolation.
But It occurs on ~50% of redeploys and requires repeated restarts of the application to clear it.
We use TestContainers as part of the integration tests to spin up pulsar in isolation and it never happens in this scenario. (Empty schema registry when consumer and producer connects, then schema def on first use)
I’d be happy to spend time on this if anyone has any ideas on how I might recreate it.
Rob
Get Outlook for iOShttps://aka.ms/o0ukef

From: dhinesherode91 @.>
Sent: Thursday, June 24, 2021 2:56:18 AM
To: apache/pulsar @.>
Cc: Rob Shepherd @.>; Mention @.>
Subject: Re: [apache/pulsar] Cannot parse  schema (#8429)
Hey,
I am facing a similar issue.
I am running pulsar 2.7.1 and I have a topic with AVRO schema where my producers push avro messages and consumer receives and process the avro data. My consumer application receives the message through pulsar client's MessageListener. All my topic messages are tagged with the latest avro schema version. And I could see that schema in the Pulsar cluster's SchemaRegistry.
But I am facing the following issue when my consumer application starts and consumes message.
org.apache.pulsar.shade.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse  schema at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) at org.apache.pulsar.shade.com.google.common.cache.LocalCache.get(LocalCache.java:3951) at org.apache.pulsar.shade.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935) at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader.read(AbstractMultiVersionReader.java:86) at org.apache.pulsar.client.impl.schema.AbstractStructSchema.decode(AbstractStructSchema.java:60) at org.apache.pulsar.client.impl.MessageImpl.getValue(MessageImpl.java:301) at org.apache.pulsar.client.impl.TopicMessageImpl.getValue(TopicMessageImpl.java:143) at com.kohia.galaxy.listener.MessageListenerImpl.received(MessageListenerImpl.kt:20) at org.apache.pulsar.client.impl.ConsumerBase.callMessageListener(ConsumerBase.java:882) at org.apache.pulsar.client.impl.ConsumerBase.lambda$triggerListener$5(ConsumerBase.java:862) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse  schema at org.apache.pulsar.shade.org.apache.avro.Schema.parse(Schema.java:1597) at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1396) at org.apache.pulsar.shade.org.apache.avro.Schema$Parser.parse(Schema.java:1384) at org.apache.pulsar.client.impl.schema.util.SchemaUtil.parseAvroSchema(SchemaUtil.java:46) at org.apache.pulsar.client.impl.schema.reader.MultiVersionAvroReader.loadReader(MultiVersionAvroReader.java:53) at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader$1.load(AbstractMultiVersionReader.java:52) at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader$1.load(AbstractMultiVersionReader.java:49) at org.apache.pulsar.shade.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) at org.apache.pulsar.shade.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ... 18 common frames omitted
The above exception occurs for all the message that is received by the consumer.
But when I restart my consumer application, it starts working fine and the above exception is not occurring.
This issue occurs randomly when I restart my consumer application and when I again restarted it works fine and all the message are processed properly.
Here is my subcription code,
pulsarClient .newConsumer(AvroSchema.of(MyMetric::class)) .topic(topicName).subscriptionName(subscriptionName) .subscriptionType(SubscriptionType.Shared) .receiverQueueSize(localQueueMsgCount) .messageListener(myMessageListener) .subscribe()
Here is my Listener code(short version),
import org.apache.pulsar.client.api.Consumer import org.apache.pulsar.client.api.Message import org.apache.pulsar.client.api.MessageListener import java.net.InetAddress class MyMessageListener() : MessageListener { override fun received(consumer: Consumer, msg: Message) { try { process(msg.value) consumer.acknowledge(msg) } catch (ex: Exception) { consumer.negativeAcknowledge(msg) } } fun process(metric:MyMetric){} }
When I debug, I could find the following log at the time when my consumer application is working perfectly.
Load schema reader for version(2), schema is : {"type":"record","name":"MyMetric","namespace":"com.example","fields":[{"name":"assetId","type":"long"},{"name":"utctime","type":{"type":"string","avro.java.string":"String"}}]}, schemaInfo: { "name": "mytenant/mynamesapce/mytopic", "schema": { "type": "record", "name": "MyMetric", "namespace": "com.example", "fields": [ { "name": "assetId", "type": "long" }, { "name": "utctime", "type": { "type": "string", "avro.java.string": "String" } } ] }, "type": "AVRO", "properties": { "__alwaysAllowNull": "true", "__jsr310ConversionEnabled": "false" } }
But at the time when my consumer application works weird, I could find the following log which doesn't have any schema info,
Load schema reader for version(2), schema is : , schemaInfo: { "name": "", "schema": "", "type": "NONE", "properties": {} }
Can u guys please help me resolve this issue. We are already in production and we find it diffficult to restart our consumer applications each and every time when this issue occurs.
@robshephttps://github.com/robshep @congbobo184https://github.com/congbobo184 Did you find any root cause for this issue or any way to fix it, if so kindly share so that it may help us.
Eagerly waiting for your inputs. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/apache/pulsar/issues/8429#issuecomment-867272525, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANXMU74YNJET3RKHWSFL63TUKGEFANCNFSM4THYHL3Q.

Thanks for the information.
In my case, in my local I am running a standalone Pulsar and Consumer applications are Spring boot application. The above issue is not occurring in local.
But in staging and production setups, we are running Pulsar in kubernetes and our Consumer spring boot applications are running as Containers. In those setups I could see this issue more often and I have to restart my containers to make it work.
On debugging we could see like the schema is not available in the consumer application pulsar client local cache and due to that on every message receive when the pulsar client tries to convert that respective message into respective class object by parsing the schema, it throws this 'Cannot parse  schema' error.
Something related to having/loading SchemaInfo in the memory.
It would be great if someone from the team could help us in fixing it.

@congbobo184 The issue is not yet resolved. We are using Pulsar 2.7.1. Both broker and Pulsar client are using the same version. Our consumer is subscribed to a single topic only. Ours is an AVRO schema based topic.
On debugging the code, I could arrive at one point, Once the consumer receives the first message it  checks the schema version of the message and asks the broker for the schema details. At that time when the broker is serving heavy loads and is unable to process the request, then the client cannot get the schema and it can throw the above exception. Is it possible ?
Also I would like to know what is the proper configuration for Pulsar production load ?
Currently our Pulsar will receive 10K messages per minute and will grow to 100K in near future.
We have deployed our Pulsar in AWS Kubernetes (m5.xlarge instances nodegroup) with,
5 brokers (each 1CPU, 3Gi Memory),
5 bookies (each 1CPU, 4Gi Memory with separate ledger and journal mounted as EBS),
5 zookeeper (each 500m CPU, 2Gi Memory) ,
5 proxy instances (each 500m CPU, 2Gi Memory).
With this above configuration, we are facing connection exception occasionally. Kindly advise.

Do you have a stable way to reproduce this issue?

I can't relaiably re-produce this, but the issue persists.
Thanks @dhinesherode91 for adding your observations.
We do not have this load. We have a single standalone running on ample VM (no cpu, memory , i/o or disk pressures) with usual statistics output of 0.3msg/sec
We get this error when re-deploying the only consumer application. (producer remains connected).
Restarting one or more times resolves the issue to the point where a valid schema definition is printed to the console, not a null one. (so we deploy manually for now)
I can't comment on this timing related workaround (maybe, possibly, there is nothing in the queue, then on a restart there is - next time we re-deploy I will poll the admin interface for queue length and feedback
If we leave it, after a null schema is found, it does not recover and no msgs are processed on the topic.
However,
We run the same application in integration tests daily, using an ephemeral pular-standalone in a docker container to test against and never see this issue.

Thanks @robshep feedback.

I did a test to reproduce this case:
public class Main {
    public static void main(String[] args) throws Exception {
        var pulsarClient = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
        var topicName = "test-topic-" + System.currentTimeMillis();

        var producer1 = pulsarClient
                .newProducer(AvroSchema.of(MyPojo.class))
                .topic(topicName).create();
        producer1.send(new MyPojo());

        var consumer = pulsarClient
                .newConsumer(Schema.AVRO(MyPojo.class))
                .topic(topicName).subscriptionName("test")
                .subscriptionType(SubscriptionType.Shared)
                .subscriptionInitialPosition(SubscriptionInitialPosition.Latest)
                .subscribe();

        var producer2 = pulsarClient
                .newProducer(Schema.STRING)
                .topic(topicName).create();
        producer2.send("test2");

        var msg = consumer.receive();
        msg.getValue();   // org.apache.pulsar.shade.org.apache.avro.SchemaParseException: Cannot parse <null> schema
    }
}

We cannot use the old schema to decode the message with the latest schema.
If you have such a scenario, we need to avoid this.

@nodece is this issue a bug in the Pulsar side or depending on how users use it?

@robshep  @dhinesherode91  could you please check if the topic has multiple schemas by the problem when it happens?
@tisonkun it seems like a user use problem.

Moved to the Q&A category. If we find some bugs later, we can create issues correspondingly.


DragonTamer - Active Training gives unnecessary choice

Active Training still gives the choice to move a card under a dragon when there are no dragons in play

Fixed.

Fixed.


testing a new official case study submission

I have some typos and want to rearrange things... but I think this is a good start!
General Infromation
Update the following URL to point to the GitHub repository of
the case study repo you wish to submit to OpenCaseStudies:

Repository: https://github.com/yourusername/yourcasestudyname

Confirmations
Confirm the following by editing each check box to '[x]'

[ ] I understand that by submitting my case study to OpenCaseStudies,
ownership of the case study repo will eventually need to be transferred to the OpenCaseStudies organization. All source and all review commentary will be made visible to the general public.

[ ] I have read the OpenCaseStudy Submission Guideline
instructions. My package is consistent with the Bioconductor
Submission Guidelines.

[ ] My case study uses only open source programming languages
[ ] My case study uses data that is publicly available

[ ] If I made the data available, I am allowed to do so

[ ] I provide transparent descriptions of the source of the data and how it was generated when possible
[ ] All general steps are described within the case study or included within click-to-expand sections
[ ] Attributions: all outside sources are referenced and cited appropriately
-[ ] disclaimers are included
-[ ] licences for use are included
-[ ] All images/gif/videos andother forms of media that are not original to the case study include sources
-[ ] Information on how to cite the case study is included
[ ] All Core sections of the case study are present or within another case study

[ ] Motivation
[ ] Main Question(s)
[ ] Learning Objectives
[ ] Context
[ ] Limitations
[ ] What are the data?
[ ] Data Import
[ ] Data Exploration and Wrangling
[ ] Data Visualization
[ ] Data Analysis
[ ] Summary
[ ] Suggested Homework
[ ] Additional Information

note that some variation in the location of data exploration and data visualization is acceptable

Data

[ ] Data is set up using the file structure as outlined in the OpenCaseStudy Submission Guideline

[ ] I understand that a minimum requirement for case study acceptance
is to successfully knit with no ERRORS or WARNINGS.
Passing these checks does not result in automatic acceptance. The
case study will then undergo a formal review and recommendations for
acceptance regarding other standards will be addressed.

[ ] My case study addresses statistical or data science learning objectives.

[ ] I am committed to the long-term maintenance of my case study. This
includes monitoring the repository for issues that users may
have and responding promptly to requests for updates from the Core team   in response to changes in R or underlying software.

[ ] I am familiar with the Open Case Study code of conduct and
agree to abide by it.

For questions/help about the submission process, please contact us on our website.

@stephaniehicks This has some typos and I want to rearrange things... but I think this is a good start. Also, want to make an open case study code of conduct like bioconductor but one that talks about case study specifics. I would love your thoughts about that!

this is now out of date  -  I have updated based on a meeting with stephanie hicks


feat: resolveRoute function

#10282. after this is merged into master, and master is merged into version-2, we can open another PR that removes resolvePath
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[x] Ideally, include a test that fails without this PR but passes with it.

Tests

[x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

We could do this - though it will probably mean that people end up migrating it manually as opposed to via the migration tool and I'm not sure if we have a preference there

I trust deprecation warnings more than I trust automated migrations. I definitely don't think the existence of a migration tool should prevent us from providing useful feedback — there will be plenty of people who want to address deprecation warnings (possibly over a period of time) who aren't ready to bump majors

If it's not, we might want to include that update as part of this PR

this is the deprecation PR. the migration instructions will be part of the removal PR


Not possible to change client name or policies

It's currently not possible to assign a policy to a client or change its device name. An error message pops up, saying that there was an error.

thanks for reporting these issues @mbecks - have added both to list of tasks for the week

now fixed in beta - thanks for raising


[GHSA-53ph-2r2x-vqw8] Cross-site WebSocket hijacking vulnerability in the Jenkins CLI

Updates

Affected products
Description
References
Source code location

Comments
Add source code link and patch link related to CVE-2024-23898.
In https://www.jenkins.io/security/advisory/2024-01-24/#SECURITY-3315 shows it is related to SECURITY-3315.

Hi @sunSUNQ, there appears to be an error with the PR, but I added https://github.com/jenkinsci/jenkins/commit/de450967f38398169650b55c002f1229a3fcdb1b to GHSA-53ph-2r2x-vqw8.


Wrong masks for 512 512 image

Hello, I use example from https://github.com/facebookresearch/segment-anything/blob/main/notebooks/predictor_example.ipynb
But before I resized image to (512, 512)
So here code snippet to reproduce:
from predictor import SamPredictor
sam = sam_model_registry['vit_h'](checkpoint='/weights/sam/sam_vit_h_4b8939(1).pth', custom_img_size=512).cuda()
predictor = SamPredictor(sam)

image = cv2.imread('/storage/share/paul/sam_onnx_export/truck.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = cv2.resize(image, (512, 512),
               interpolation = cv2.INTER_LINEAR)

predictor.set_image(image)

input_point = np.array([[150, 150]])
input_label = np.array([1])

masks, scores, logits = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    multimask_output=True,
)

for i, (mask, score) in enumerate(zip(masks, scores)):
    plt.figure(figsize=(10,10))
    plt.imshow(image)
    show_mask(mask, plt.gca())
    show_points(input_point, input_label, plt.gca())
    plt.title(f"Mask {i+1}, Score: {score:.3f}", fontsize=18)
    plt.axis('off')
    plt.show()

So when I run this code I get images like this:

You can see that point is located inside window, but for some reasons segmentation goes wrong. What am I doing wrong?

The code in the official Meta SAM and my code is somewhat different and it will work when you run my example code

Hm, I see, so there is only way to work with different  img_size, only using sam.individual_forward, correct?

Yes correct! it would be better for you to modify my code to fit your purpose!


Task/popovers

Importing popover in components where needed and initializing them by querying for class when mounted. Seems to be working fine but willing to explore better way if possible. Some missing descriptions but working on that.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
35

Relevant Lines:
35

💛  - Coveralls


Improve typing

This PR introduces two changes to the declared type signatures:

The allowed colors, highlights, and attributes are now checked against defined string literals. This means that, e.g., cprint("hello", "orange") will result inerror: Argument 2 to "cprint" has incompatible type "Literal['orange']"; expected "Optional[Literal['black', 'grey', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'light_grey', 'dark_grey', 'light_red', 'light_green', 'light_yellow', 'light_blue', 'light_magenta', 'light_cyan', 'white']]"

when using type checker.
The type of first argument of colored and cprint was relaxed to accept objects and not just strings. This makes cprint more in line with print, which accepts arbitrary objects and fixes the example in README which readsfor i in range(10):
    cprint(i, "magenta", end=" ")

which currently results in error: Argument 1 to "cprint" has incompatible type "int"; expected "str" when running through mypy.

any way this could be merged? this seems like an amazing upgrade for usability, and the only blocker seems to be impossible to do in the current state of python typing.

Thanks for ping. If the conflicts are resolved, I'll take another look.


Try out compare_exchange inside compare_and_swap

The compare_and_swap implementation uses compare_and_swap internally. Maybe we could use compare_exchange and weaken the Ordering of the failure mode.

Tried, didn't help. Furthermore, the library now promises to contain at least one SeqCst operation in each method, to synchronize with other such operations.


pre-commit: PR118493

Link: https://github.com/llvm/llvm-project/pull/118493
Requested by: @dtcxzyw

/close


OSF-5265 Opens the link in a new tab

Purpose
Changes the template file so that new links open in new tabs.
Changes
Add "target= "_blank"" to the url.

Confirmed that the links now open in a new tab in FF/Chrome/IE.
The images should also be center aligned, as detailed in the Jira ticket. When that is done this will be ready to merge. :octopus:

Center align images done.

Looks like flexbox is IE 10+, and buggy at that.
@adlius What's our browser support policy? Does this at least work in IE 10 and 11?

@whit537 we don't have a formal browser support policy yet, but a proposal is in the works and should be through the review process within a few days.

@mfraezz @whit537 I use the flexbox feature as it seems to be a more "modern" way of aligning responsive elements in CSS. However that does depend on what browsers and which versions we eventually decide to support.
Tell me when we have a standard on browser compatibility. I can make changes that supports older browsers. The current fix works on both the newest Safari and the newest Chrome. I do not have IE so I can't speak to how my changes render in IEs.

a proposal is in the works and should be through the review process within a few days.

Bump.


Persistent

Made a new NanPersistent class.

@bnoordhuis Could you review this, please? Or suggest someone else who can do it.

Aaaand the weak callback will change again in 4.3, so that will require more rewriting again.

Maybe @trevnorris is willing to review?  I'm a bit short on time at the moment.

I'm fine to do a review, but it's been a while since I've looked at the nan source.

No need to look at old source (much of it is getting rewritten anyway). This is more or less standalone, it's a backport of the new Persistent handles to old v8, with a unified API, plus a redone PersistentBase<T>::SetWeak which accepts a different weak callback signature.

Changes look good, all tests pass. LGTM.

Fixed placement of * and &.

And added a comment statind that the memcpy thing assumes that v8::Persistent<T> exhibits bitwise copy semantics.

Redid weak callback.

Merged as 4a2ae84f84465b2a47aacbdb02c5c0b09b76dd52


Open resource links from any organization/sandbox

Motivation and context
Server part:

[x] Update swagger docs
[x] For object requests: define iam_context according to the obj.organization_id
[x] For list and create requests: iam_context is defined according to the query parameters org_id or org (slug)
[x] Fix tests

UI part:

[x] Change organization when link leads to resource in different organization
[x] Fix tests

How has this been tested?

Checklist

[x] I submit my changes into the develop branch
[x] I have added a description of my changes into the CHANGELOG file
[ ] I have updated the documentation accordingly
[x] I have added tests to cover my changes
[x] I have linked related issues (see GitHub docs)
[x] I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

[x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

@kirill-sizov , job cannot be found if I'm not inside an organization. Is it expected?

@kirill-sizov , job cannot be found if I'm not inside an organization. Is it expected?

Yes, this is a bug. The server returns 404 for the request GET jobs/{id}/data/meta?org. So I fixed it. Now query parameter will be ignored for detailed requests. Such ignoring also promotes backward compatibility (if such users already use our API with using org parameters this patch won't break their pipelines)

@klakhov Could you please check this test it seems that this step Logout, the third user login. The user does not see the project, the task. The user can't open the job using direct link. is not valid anymore?

I think there are a few more API endpoints that don't need the X-Organization/org/org_id parameters:

events_create
organizations_list
users_list
webhooks_list_deliveries

Could you double-check and remove those parameters if they are indeed unnecessary?

I think there are a few more API endpoints that don't need the X-Organization/org/org_id parameters:

events_create
organizations_list
users_list
webhooks_list_deliveries

Could you double-check and remove those parameters if they are indeed unnecessary?

Yes, thanks for your attention. I have fixed this by applying such strategy: if view is not detailed and view has iam_organization_field then this endpoints will have org parameters.
org for endpoint users_list helps to get all members in organization
org for endpoints events_create needs for IAM (as for others POST endpoints)

org for endpoints events_create needs for IAM (as for others POST endpoints)

Why? I don't think events belong to organizations.

Events associated with organizations. We manage access to events based on which organization particular events belong.
It's a unified approach, currently, we use the org_id parameter for POST requests for each object that has an organization attribute. I see no reason why we should use a different strategy for events.

I got the error below when I requested a task.
Could not receive the requested project from the server
Error: Request failed with status code 403. {"detail":"You do not have permission to perform this action."}.

@kirill-sizov , /api/jobs request doesn't work in debug at least. Could you please look?

@nmanovic I fixed the performance problems and with getting jobs. But confused about this issue:

I got the error below when I requested a task.
Could not receive the requested project from the server
Error: Request failed with status code 403. {"detail":"You do not have permission to perform this action."}.

I have checked accessing tasks with different parameters of privileges and organization roles (this is also covered by our test) and couldn't reproduce this problem. Could please clarify how did you get this error?
Did you get this error in the development instance or in docker?
Does this task belong to the organization or project?
Which privilege does the requester have?
Which organization role does the requester have?
Does the requester assign to the task or corresponding project?

But I would suggest to postpone this idea until we have other such objects besides labels.

There is a couple more in the Honey Pot feature endpoints. Probably, an option to provide a list or even a mapping would be useful, similarly to what is done for other list filters.


Version Packages

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
Releases
<EMAIL_ADDRESS>Patch Changes

#598 29ec788 Thanks @threepointone! - warn when running in replit without a token

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Question text is moving when changing from Javascript to Python snippet

screen-capture (19).webm

This is an odd bug since both sides should have a width of 1/2 regardless of the change in the code snippet, will experiment.


Spring Sleuth not woring for SOAP outgoing calls.

Hi, guys!
I'm trying to set up the following Spring Sleuth configuration to propagate remote fields (HTTP request headers) across the projects out-of-the-box:
  sleuth:
    baggage:
      remote-fields:
        - Caller-Id

For the outgoing Rest calls, it's working fine:
Request headers: {Accept=[application/json; distances], Authorization=[Bearer ...], X-B3-TraceId=[3f3f61757851d371], X-B3-SpanId=[3f3f61757851d371], X-B3-Sampled=[1], **caller-id=[value]**}
However, it's not true for the SOAP outgoing calls:
SOAP Headers - {Authorization=[Bearer...]}
We're using Feign as Rest client and jaxws-spring for SOAP.
         <dependency>
            <groupId>com.revinate</groupId>
            <artifactId>jaxws-spring</artifactId>
        </dependency>

Am I doing something wrong?

com.revinate:jaxws-spring is not instrumented by Sleuth. In order to make context propagation work somebody needs to instrument it, i.e.: set those headers. I guess this library belongs to your employer and the maintainers of this client should take care about instrumenting this.
Can you contact them to do it?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.


Adding the beginnings of code for feeding building data into Mapbox maps for the building page

Something to review
In the building page, I'm working on a way to grab data for Mapbox maps. As shown in the building page prototype, we're displaying various layers of data (probably geoJSON) on Mapbox maps: layers of public transport, other public housing, building permits and so on. I was thinking of using app.getData() to do this, grabbing the .csv files that we use for Charts and manipulating the data into geoJSON (if it's not in that format already). This way, I could add geoJSON layers to Mapbox maps while sharing data with any other Charts that needed it.
There's a possible issue here: the parameters of app.getData() currently assume that we will be using the data for a Chart. In the comments I've suggested two possibilities, either (a) adjusting app.getData() to work with both Charts and Mapbox objects; or (b) having Mapbox maps make separate requests, pulling geoJSON straight from the server. Let me know which you prefer and I can add to my '80-map-on-building-page' branch.
A more general possibility
Since different Charts (or Mapbox maps) will be on different pages of the site, we could think about putting the 'app' object in some kind of client-side storage, e.g. Window.sessionStorage. This way a user could, say, access the moving block chart, then access the building page, reusing the data from the former for the latter without the code having to process the .csv file again.

@ptgott I think your session storage idea is right on and wouldn't be hard to implement. The code could check for the object in local storage (if local storage exists) and fetch new data only if it's not there already. I or someone could work on that tonight. It occurs to me, too, that the Chart constructor shouldn't require a developer to know, regardless of local storage or not, whether their chart's data would already have been loaded by another chart. The code should instead check to see if there's already a data object from the requested source and, if so, skip reloading it. That could be another task for tonight. Currently I have us putting in a null value for the source—that's pretty unnecessary.
As for modifying Chart to serve the needs of mapbox data objects, I think it would be easiest and perhaps more maintainable to keep the methods separate but have the mapbox data-getting method append its results to the same app.dataCollection object. Does that make sense to you?

@ostermanj I won't be at the meeting tonight, but just want to say that having Chart constructors check for relevant data objects before loading them is a good idea. I'll check on what you/others get done tonight and help pick up any remainders later this week. I'll also work on constructors that are separate from Chart, fetch data for Mapbox maps and store it in app.dataCollection (checking to see if it's there first).

On Feb 21, 2017, at 13:59, John Osterman<EMAIL_ADDRESS>wrote:
@ptgott I think your session storage idea is right on and wouldn't be hard to implement. The code could check for the object in local storage (if local storage exists) and fetch new data only if it's not there already. I or someone could work on that tonight. It occurs to me, too, that the Chart constructor shouldn't require a developer to know, regardless of local storage or not, whether their chart's data would already have been loaded by another chart. The code should instead check to see if there's already a data object from the requested source and, if so, skip reloading it. That could be another task for tonight. Currently I have us putting in a null value for the source—that's pretty unnecessary.
As for modifying Chart to serve the needs of mapbox data objects, I think it would be easiest and perhaps more maintainable to keep the methods separate but have the mapbox data-getting method append its results to the same app.dataCollection object. Does that make sense to you?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@ostermanj and @ptgott are you guys wanting to merge this in yet, or still in discussion? Just want to make sure PRs get in when they're ready (but of course not before : )

I'm happy to merge it now as the beginning scaffolding for getting the map onto the building page. We'll need to fix or replace the now-missing project.csv problem to go further with it, but that's true for all pieces.
@ptgott I have some local commits that have chart constructors automatically using an existing data object if it exists and creating it if it doesn't. Currently the developer has to know whether the data has been fetched already or not—not very sustainable. I tried out the sessionStorage but found two problems. One, it's slow: local storage has to be stringified on save and parsed on load; that and having to access the disk actually made it slower than fetching the data again. And two: I was having difficulty with the chart constructors doing their thing before the object from storage was ready. I couldn't figure that out because JSON.parse should be synchronous and have subsequent actions wait, but it wasn't working. Anyway, it might not be worth the performance hit, but maybe in production fetching the data would take longer than it is now (?).


MSAL(@1.3.3)'s  acquireToken not  sets UPN to X-MS-client-principal-name.

When I upgraded to<EMAIL_ADDRESS>my program didn't work well.
I also confirmed that the avoidance action is to version down to<EMAIL_ADDRESS>I seem to have encountered some incompatible change.
As before, MSALl@2 and<EMAIL_ADDRESS>and later versions provide a configuration method that allows  to set "UPN" for "X-MS--client-principal-name".
Library

[v ]<EMAIL_ADDRESS>
Framework

-- Client SPA

<EMAIL_ADDRESS><EMAIL_ADDRESS>

-- Azure Functions

"@azure/functions": "^1.2.2",

   Project Sdk="Microsoft.NET.Sdk"
     TargetFramework>netstandard2.0</TargetFramework
    ItemGroup
       PackageReference Include="Microsoft.Azure.WebJobs.Extensions.CosmosDB" Version="3.0.7"
      PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="1.8.5"
      PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="3.0.11"
      PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.0.2"

Description
I have created a function app for the http trigger that authenticates with Azure AD UserPrincipalName in the Azuer function environment.
When the function kicks the URL from the browser, the UserPrincipalName arrives and it authenticates correctly.
However, if you acquire a token from MSAL.js, you will get a short name instead of UserPrincipalName, resulting in http 401 error unautolize.

Error Message
   * HTTP Status Code: 401 Unauthorized

* MSAL samplle apps Error message: Unable to acquire access token for get ResourceKeys.
Security

[v ] Is this issue security related?
Setting'x-MS-CLIENT-PRINCIPAL-NAME' to a shorter name than UserPrincipalName facilitates impersonation.

Regression

[v ] Did this behavior work before?
Version<EMAIL_ADDRESS>
<EMAIL_ADDRESS>https://alcdn.msauth.net/lib/1.2.1/js/msal.js

MSAL Configuration
// Provide configuration values here.
```js
// Provide relevant code snippets here.
export const GRAPH_SCOPES = {
    OPENID: "openid",
    PROFILE: "profile",
    USER_READ: "User.Read",
    MAIL_READ: "Mail.Read",
    RESOURCEKEYS: [
        "api://d1789387-f340-464a-ace5-1df84244b661/User.Read",    // "http://supportersystem-demo1-makedataactivity2.kazumatakedapeerconnectcojp.onmicrosoft.com/User.Read"
        "api://d1789387-f340-464a-ace5-1df84244b661/user_impersonation"  // "http://supportersystem-demo1-makedataactivity2.kazumatakedapeerconnectcojp.onmicrosoft.com/user_impersonation"
    ]
};

export const GRAPH_ENDPOINTS = {
    ME: "https://graph.microsoft.com/v1.0/me",
    MAIL: "https://graph.microsoft.com/v1.0/me/messages",
    RESOURCEKEYS:
        "https://supportersystem-demo1-makedataactivity2.azurewebsites.net/api/WebApps_getResourceKeys1?tenantName=%E3%83%94%E3%82%A2%E3%82%B3%E3%83%8D%E3%82%AF%E3%83%88%E3%81%AE%E5%BE%8C%E6%8F%B4%E4%BC%9A&code=?????????????????????????????????????????"
};

export const GRAPH_REQUESTS = {
    LOGIN: {
        scopes: [
            GRAPH_SCOPES.OPENID,
            GRAPH_SCOPES.PROFILE,
            GRAPH_SCOPES.USER_READ,
            ...GRAPH_SCOPES.RESOURCEKEYS
        ]
    },
    EMAIL: {
        scopes: [
            GRAPH_SCOPES.MAIL_READ
        ]
    },
    RESOURCEKEYS: {
        scopes: [...GRAPH_SCOPES.RESOURCEKEYS]
    }
};

export const msalApp = new UserAgentApplication({
    // export const msalApp = new msal.PublicClientApplication({
    auth: {
        clientId:
            "d1789387-f340-464a-ace5-1df84244b661",
        authority:
            "https://login.microsoftonline.com/common",
        redirectUri:
            window.location.protocol + "//" + window.location.hostname + ":" + window.location.port + "/" + "auth.html" // :
        // "https://supportersystem-demo1-makedataactivity2.azurewebsites.net/.auth/login/aad/callback"
        //  このcallbackを指定すると、下記URLが開いたままとなった。 auth.htmlでも認証状態は保持できた。
        //  https://supportersystem-demo1-makedataactivity2.azurewebsites.net/.auth/login/done
        // )
        ,
        validateAuthority: true,
        postLogoutRedirectUri: this_first_URL1, // document.URL,
        navigateToLoginRequestUrl: true
    },
    cache: {
        cacheLocation: "sessionStorage",
        storeAuthStateInCookie: isIE()
    },
    system: {
        navigateFrameWait: 500,
        logger: new Logger((logLevel: any, message: string) => {
            console.log(message);
        }),
        telemetry: {
            applicationName: "supportrt Association demo app",
            applicationVersion: "1.0.0",
            telemetryEmitter: (events: object) => {
                console.log('Telemetry Events:', events);
            }
        }
    }
});

Azure functions/Microsoft_AAD_RegisteredApps manifest

Reproduction steps
After signing on using MSAL.js, the URL of the http trigger function of Azure function was fetched using the token obtained from msalApp.acquireTokenSilent.
When you look at the request header that arrived at the http trigger function of that Azure function, the name is set instead of the UPN.

Expected behavior
When the URL of the http trigger function of the Azure function is kicked from the browser, UPN is set in the request header that reaches the Azure function. I expect that similar values will be set via MSAL.js.

Browsers/Environment

[v ] Chrome
[ v] Edge

@master-maintenance1-peer-connect Can you please clarify if this behavior happens using 1.2 today?

@jasonnutter
<EMAIL_ADDRESS>worked well.
<EMAIL_ADDRESS>worked well.
<EMAIL_ADDRESS>Do NOT worked well.

Having this issue as well

Hi, @master-maintenance1-peer-connect. Could you please clarify the 'x-MS-CLIENT-PRINCIPAL-NAME' header is being set?

@rfink Thank you for checking the reproducibility.
＠technical-boy　Here's the request header when it works.
I can't set anything directly to 'x-MS-CLIENT-PRINCIPAL-NAME' header.
The 'x-MS-CLIENT-PRINCIPAL-NAME' header is set to the INVALID value that is not in UPN (USER PRINCIPAL-NAME) by something that<EMAIL_ADDRESS>passed to Azure Function.
  req:
   Request {
     method: 'GET',
     url:
      'http://supportersystem-demo1-makedataactivity2.azurewebsites.net/api/WebApps_getResourceKeys1?tenantName=%E3%83%94%E3%82%A2%E3%82%B3%E3%83%8D%E3%82%AF%E3%83%88%E3%81%AE%E5%BE%8C%E6%8F%B4%E4%BC%9A&code=5KTWXP4BvexMSq2bpAcGahWGXWs3GGIjLbadL1euTuGDCCbeXTLBCw==',
     originalUrl:
      'http://supportersystem-demo1-makedataactivity2.azurewebsites.net/api/WebApps_getResourceKeys1?tenantName=%E3%83%94%E3%82%A2%E3%82%B3%E3%83%8D%E3%82%AF%E3%83%88%E3%81%AE%E5%BE%8C%E6%8F%B4%E4%BC%9A&code=5KTWXP4BvexMSq2bpAcGahWGXWs3GGIjLbadL1euTuGDCCbeXTLBCw==',
     headers:
      { connection: 'Keep-Alive',
        accept: '*/*',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'ja, en-US; q=0.9, en; q=0.8',
        authorization:
         'Bearer eyJxxxxxxxxxxx',
        host: 'supportersystem-demo1-makedataactivity2.azurewebsites.net',
        'max-forwards': '10',
        referer:
         'https://supportersassociation-demo1-web-svc-d2.azurewebsites.net/demo1/index.html',
        'user-agent':
         'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36',

        'x-original-url':
         '/api/WebApps_getResourceKeys1?tenantName=%E3%83%94%E3%82%A2%E3%82%B3%E3%83%8D%E3%82%AF%E3%83%88%E3%81%AE%E5%BE%8C%E6%8F%B4%E4%BC%9A&code=5KTWXP4BvexMSq2bpAcGahWGXWs3GGIjLbadL1euTuGDCCbeXTLBCw==',
        'x-forwarded-for': '<IP_ADDRESS>:64082',
        'x-arr-ssl':
         '2048|256|C=US, S=Washington, L=Redmond, O=Microsoft Corporation, OU=Microsoft IT, CN=Microsoft IT TLS CA 5|CN=*.azurewebsites.net',
        'x-forwarded-proto': 'https',
        'x-appservice-proto': 'https',
        'x-forwarded-tlsversion': '1.2',
        'x-ms-client-principal-name':
<EMAIL_ADDRESS>        'x-ms-client-principal-id': 'f2cddec7-8438-487f-873f-8296ef73f96a',
        'x-ms-client-principal-idp': 'aad',
        'x-ms-client-principal':         'xxxxx',
        'request-id': '|8dc61940-444c4fe33cf6b260.1.' },

Please tell me,
What should be tracked back in MSAL authentication flow to determine the cause of this Issue?
Now I saw the difference in cookie values using Azure functions ".auth/me" endpoint.

cookie  When it dosn't Work:
Cookie.user_name(== 'x-MS-CLIENT-PRINCIPAL-NAME') Equal to Cookie.user_claims.name(Short Name).

cookie  When it  Work well:
Cookie.user_name(== 'x-MS-CLIENT-PRINCIPAL-NAME') Equal to Cookie.user_claims.preferred_username(Long Name).

**  Perhaps the whims of user_id (='x-MS-CLIENT-PRINCIPAL-NAME') are related to Azure functions token configuration.

** Or maybe openID_configlation(isser) is involved.


What's  differents between the models extracted from trained_models.rar?

I downloaded the trained_models.rar and decompressed it.Several files were extracted there.But I don't know the exact meaning and what they stand for.
Could you tell me?

Hello,
you need to study the code.
It is a complex problem that requires some effort to grasp.


Support for easyOCR

Since easyOCR / google has a way better support for non-latin languages, it'd be great if the addon is engine agnostic.

Hey, I'm going to close this issue as I don't have the time to add support for a different OCR engine, you're more than welcome to fork this project and add support, but it may be move involved than it appears.


Advise running ntpd on Brooklyn server

One tiny typo, but otherwise fine and dandy.

Thanks; incorporated fixes - merging now.


Unify checkbox, star, and radio buttons in the table

There's a background discussion in https://iterativeai.slack.com/archives/C01APS0FHDM/p1676329398900699.
I get confused by when to use each of these:

I especially find it confusing because the checkbox actions for multiple experiments only impact the other two columns:

For example, if I go to star some experiments to compare, I can't then plot them automatically:
https://user-images.githubusercontent.com/2308172/223852141-233e54cf-c216-49e6-88bf-b55aa0e14f52.mov

For example, if I go to star some experiments to compare, I can't then plot them automatically:

Q: what would you expect for this? A separate action, like plot all starred as part of the Add experiment drop down?

There is some unification happening in Studio that is related to this. Checkboxes will behave like they do in all the other products (including MLFlow, for example) - pick items to do some mass action (vs showing plots by default). That alone should reduce the confusion.
Also, there will an eye icon (that is srandard for W&B, MLFlow, comet, neptune, etc) to toggle the plots visibility.
I wish we could have adopted an eye icon as well here, but it's a limitation on the VS Code side, @mattseddon can add more color to this.
Stars - a bit different story. I'm trying to understand the confusion better - do you feel for example that stars are responsible for plots? If we change a circle to an icon - would that help or not? If we add quick actions as we discussed - would that help to clarify things or not?

I wish we could have adopted an eye icon as well here, but it's a limitation on the VS Code side, @mattseddon can add more color to this.

Tl;dr is that we can only have 1 icon per row in the experiments tree. If we want to blend a spinner + selected/unselected for plotting then it needs to be a circle. Unless there is some smarter option that we have missed or we want to add a custom webview to the sidebar.
For more background, you can start here: https://github.com/iterative/vscode-dvc/issues/712
Note: Adding a webview opens up a different set of problems (matching styling/custom context menu etc) but it can be done.

I realise we only have a very small sample (138) but could we look at our engaged users' usage to help here? Could even look at total usage?

This definitely won't solve the whole thing, but adding a bigger margin after the checkbox might help understand that it behaves differently than the other two actions. It's a quick change that should help at least a little until we figure out a more complete solution.

Thanks all! This is definitely in discussion phase, so no need to rush to action.
If I use 🔘 then I find I can't filter the table to those experiments. If I use ⭐ , I can't plots those experiments. How do I choose a set of experiments that I can filter, sort, and plot? I think it would be nice to be able to filter, sort, plot, and star all 🔘 experiments.
Is the idea of ⭐ to save them so they can't be lost? If so, I have a couple questions about it:

Is it okay that they disappear once newer commits are added?
When would this be recommended vs doing dvc exp branch?

I also think the UI could make it more clear that this is the expected behavior of ⭐:

Can we make ⭐ be selected/deselected from the checkbox actions and not directly clickable? If these are supposed to be saved, I think it's too easy to accidentally select/deselect and doesn't seem to fit the purpose.
Maybe ⭐ is too vague of a visual cue? @daavoo suggested 📌 here (or maybe 🔒?).
Are there other visual cues that could help? For example, move all of these rows to the top of their commits or even the top of the table entirely? Color/format these rows differently?

@dberenbaum let's forget about the stars for a moment (I can describe the context and intuition I had when we prioritized that a bit later). Do checkboxes and circles behave in the way you would expect for you?

If I use 🔘 then I find I can't filter the table to those experiments. If I use ⭐ , I can't plots those experiments. How do I choose a set of experiments that I can filter, sort, and plot? I think it would be nice to be able to filter, sort, plot, and star all 🔘 experiments.

This is my only request/confusion regarding the circles.

Okay, the plot part comes by default with circles (they are kind a toggle for plot/unplot).
Filtering - we can add an option to filter them I guess, similar to the way we do with stars. I wonder if this should be even a quick toggle (similar to the "show only changes" toggle that we are discussing)- since I assume your use case it quickly compare metrics among plotted experiments?
Star / unstar - can be done via checkboxes. Any mass action should be done via checkboxes + context menu or a button that applies to the selected set of exps. Question here is do we want to have a quick way to select only plotted experiments? And how we do this in UX.
Am I going in the right direction, Dave?

@dberenbaum let's sync on this tmr one more time. It's not clear if we can come up with some action items here.

Okay, as we discussed the issue here is that it's not clear what is the workflow with the stars and radio buttons is expected. What "I can do with them"? Let's say I selected them, but what then? Can I see quickly see them in one place? Can I do actions on them at once? etc
Some possible practical steps:

Add a quick way to filter by starts and plot buttons
Still need a better explanation what radio buttons do (toggle plots). Change them to an eye icon after all? At least in the table even if we can't unify them with the side bar.
Remove stars to simplify things (it was based on my personal bias  - when I was running a lot of things, I needed some way to "remember" about iteration. Convert them into branches?

Thanks @shcheklein. TBH I would not prioritize it myself, but all of those sound like good steps to me.


Fix TS Errors from resolveAssetSource and styleMap

Description
I experienced multiple TSC errors when depending on the current #main in an Expo project with the following tsconfig.json:
{
  "extends": "expo/tsconfig.base",
  "compilerOptions": {
    "strict": true
  }
}

Errors thrown bei tsc --noemit:
node_modules/@rnmapbox/maps/javascript/utils/StyleValue.ts:2:32 - error TS7016: Could not find a declaration file for module 'react-native/Libraries/Image/resolveAssetSource'. '/Users/finger/Development/rrph-app/node_modules/react-native/Libraries/Image/resolveAssetSource.js' implicitly has an 'any' type.
  If the 'react-native' package actually exposes this module, consider sending a pull request to amend 'https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/react-native'

import resolveAssetSource from 'react-native/Libraries/Image/resolveAssetSource';
                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@rnmapbox/maps/javascript/utils/StyleValue.ts:4:30 - error TS7016: Could not find a declaration file for module './styleMap'<EMAIL_ADDRESS>implicitly has an 'any' type.

import { getStyleType } from './styleMap';
                             ~~~~~~~~~~~~

Checklist

[x] I have tested this on a device/simulator for each compatible OS
[ ] I updated the documentation with running yarn generate in the root folder
[ ] I mentioned this change in CHANGELOG.md
[ ] I updated the typings files (index.d.ts)
[ ] I added/ updated a sample (/example)

@mfazekas this should now be up-to-date with #main.


Issue with Custom Request File Path

Describe the bug
The command specified in the options section is not functioning as expected. When I specify a different file path for the request option, it still defaults to using the request/core/request.ts file instead of the custom path provided.
To Reproduce
Steps to reproduce the behavior:

Execute the following command: openapi-rq -i ./openapi.json -c axios --request=./src/services/request.ts -o ./src/services --format=prettier --lint=eslint
Observe that the request file used is request/core/request.ts instead of the specified src/services/request.ts.

OpenAPI spec file
If possible, please upload the OpenAPI spec file.
Expected behavior
The command should use the specified custom request file path (src/services/request.ts) instead of defaulting to request/core/request.ts.
Screenshots
If applicable, add screenshots or logs to help explain your problem.

OS: macOS
Version: v1.4.0

Additional context
The specified command is:
"codegen": "openapi-rq -i ./openapi.json -c axios --request=./src/services/request.ts -o ./src/services --format=prettier --lint=eslint"
Even if I specify a different file path for the request option, it defaults to using request/core/request.ts.
Thank you to the maintainer for this awesome and helpful library! It has been incredibly useful in our projects.

The custom request file is copied from the provided request file location to request/core/request.ts
When you want to edit the request file you must rerun the generator for the updates to be copied into the generated files.
I wouldn’t manually edit the generated files, unless you are committing them.


[MP Blog] Announcing an update for the Eclipse MicroProfile Course Bundle

Hi all,
I've just updated my free Eclipse MicroProfile course bundle to now support the latest MicroProfile version and created a blog post to announce it: https://microprofile.io/?p=6534&preview=true
Kind regards,
Phil

Hi Phil.
I will take a look at the blog and put together a SM push this week

@rieckpil do you mean your e-book is now 3.3 updated?
if yes, then we need to update the current bi-monthly push with new design that says 3.3. and version 2. do confirm, i will recommend small adjustments to current media description but up to you to adjust it. Amazing turnaround.

Thanks for your quick reply and help @rstjames and @aeiras.
Yes, the e-book is now 3.3 updated.
What about the following adjustment to the tweet?

#MicroservicesEnthusiasts, Philip Riecks @rieckpil updated the comprehensive & free Course Bundle (E-Book, Video Course, Cheat Sheet) to kickstart your #MicroProfile success using the latest MicroProfile version 3.3. LINK_HERE
#opensource #MicroProfileeBOOK

Looking forward to hearing your feedback

reads perfect!
@rstjames confirmed that current e-book pushes need to be deleted and the updated version needs to be put forth. Say, until the end of Q3 this time around.
my only addition to your description @rieckpil is to add before your name #MicroProfiler Philip...
@rstjames thanks for closing ticket and choosing to help once again upon ticket arrival. :)

@rieckpil
I updated the top image for the blog. I suggest to set the blog post a bit differently and maybe post it as a normal blog post and have an solid overview what is in the bundle and put a link to it instead of the redirect set up on the MP site. I can set up the link as a big button so people dont miss it. This makes it a bit easier to host it on the MP site.
Also why is your old post "pending" I would not change or mess this that link because if anyone sees an old tweet with the link you want them to still be able to get to your site not get a 404. Make sense?
If you agree please let me know when this is done and I will push it all out.

Thanks for the ideas @rstjames. I'll update it and create a proper blog post on microprofile.io instead of the redirect link and ping you once it's ready 👍
Which old post do you mean? The one with the re-direct link or Tutorials on each Eclipse MicroProfile Specification
I somehow can't edit my old posts:

@rieckpil maybe easiest thing is to have your old blog redirect to you new blog post. Thoughts?
If you think this works I will take care of this when you are done with the blog post.

Stand by & allow me to look into Wordpress with my admin access to find issue & fix it before adjusting 1st blog! 💛

Thanks for the ideas @rstjames. I'll update it and create a proper blog post on microprofile.io instead of the redirect link and ping you once it's ready 👍
Which old post do you mean? The one with the re-direct link or Tutorials on each Eclipse MicroProfile Specification
I somehow can't edit my old posts:

@rieckpil,
Your blog is no published and you & Ryan can adjust its header banner & content as stated on previews exchanges +1!
I finally understand what you meant by not being able to adjust your blog, I assume you meant you were not able to publish your blog. That is normal as only MP Editors can push content from contributors.
Please note that this update is a continuation of the introduction of your valuable e-book, as such I have adjusted its title, see below. :) Super fun stuff to see such a force of nature with content update Phil. you rock!

@rstjames,
Phil needs to adjust the content of the blog as you suggested, excellent feedback by the way.
Could you help him get a nice blog banner too? awesomeness via open channels. #oss!

thanks for publishing it @aeiras.
As it is now published I can't this post anymore (you see this screenshot I sent before - after publishing its read-only). I wanted to incorporate @rstjames feedback and changing the blog post from a link to post type while adjusting the content.
Can I send you the final version and you change the content & post-type for me @aeiras?
Thanks in advance,
Phil

The final content of the blog post could be something like:
The Eclipse MicroProfile Course Bundle contains the most comprehensive e-book and video course to start learning Eclipse MicroProfile. It covers all twelve Eclipse MicroProfile specifications with hands-on examples and is constantly updated with each new MicroProfile release.
Not only does it help MicroProfile newcomers, but it also helps seasoned Java EE and Jakarta EE developers to enrich their existing applications with missing features.
While working with this Course Bundle, you’ll:

get hands-on experience with each MicroProfile specification
write cloud-native applications with ease
learn how to add missing parts (e.g. Open API support, JWT Auth, Resiliency, etc.) to an existing Enterprise Java application

The Course Bundle was recently updated to now cover Eclipse MicroProfile 3.3. With this minor release, the e-book now covers:

Eclipse MicroProfile Config 1.3
Eclipse Fault Tolerance 2.1
Eclipse MicroProfile Health 2.2
Eclipse MicroProfile Metrics 2.3
Eclipse MicroProfile Rest Client 1.4

For two new features of this MicroProfile 3.3 release (@SimplyTimed and CDI support in ClientHeadersFactory), there is now also a dedicated video course section available.
GET ACCESS TO IT HERE (CTA Button)

@phillip-kruger

Blog has been updated. Touched up the formate a little bit.
Eclipse MicroProfile release 3.2 blog has a redirect to the new blog post.

Tweet:
#MicroservicesEnthusiasts, Philip Riecks @rieckpil UPDATED the comprehensive & free Course Bundle (E-Book, Video Course, Cheat Sheet) to kickstart your #MicroProfile success using the latest @MicroProfileIO version 3.3. https://bit.ly/3bq6e5I
#opensource #MicroProfileeBOOK
I will push it on SM today if all looks good to you. Please Confirm.

I think you mean other Phillip :)  @rieckpil  ^^^

thanks for your effort @rstjames. Blog post looks great 👍 You can start tweeting, thank you very much 🙇♂

@rieckpil all set.

Hola @rieckpil,
Yesterday with @rstjames, we discovered an issue listed here that affects the media post on your e-book. https://github.com/eclipse/microprofile-marketing/issues/305
We have fixed the issue & your e-book will be once again published via the MP twitter media as it was done before.
Thank you for being that amazing!

@aeiras @rieckpil All updated and scheduled to go out.

Amazing turnaround Ryan! you are AMAZING :) THANK YOU!


Improve searching on embedded assets

I'm bumping up against this on a project. I need to be able to query for a certain type of asset, and this is not currently possible as it stands.
I think it'd be great if Embedded Assets had a table that stored some of the metadata from the embedded asset to query against.
Right now I plan to walk over the json files to search, but that is not very sustainable.


Replace mico-spinner to nanospinner

nanospinner weigh less than others (13кб)

Release as part of 2.10.6


Impossible to reach the desired temp

Hi there,
I tested my Lepton Flir 3.5mm with the soft get Thermal and I cannot reach the dynamic range that was advertised by the manufacturer.
Indeed, this camera should be able to detect temperatures up to 400°C but the limit of the visualisation with this software in both high and low gain is of 181.35°C.
I just want to now if this issue is due to the software or to the camera that cannot transmit a higher level of intensity via the board.
Have a nice day,
Maxime B.

Looks like the low gain command isnt working.
181 C is the max for high gain. It sounds like it's most likely the
software. I dont have a fix for you unfortunately. Kurt will have to help
with that.
I suggest trying the windows flir lepton software for testing. You can also
try this repo instead https://github.com/Kheirlb/purethermal1-uvc-capture
which should also let you change from high to low. If it works, this will
confirm if it's a software issue at least.
Karl
On Tue, Sep 3, 2019, 9:19 AM peterbonnesoeur<EMAIL_ADDRESS>wrote:

Hi there,
I tested my Lepton Flir 3.5mm with the soft get Thermal and I cannot reach
the dynamic range that was advertised by the manufacturer.
Indeed, this camera should be able to detect temperatures up to 400°C but
the limit of the visualisation with this software in both high and low gain
is of 181.35°C.
I just want to now if this issue is due to the software or to the camera
that cannot transmit a higher level of intensity via the board.
Have a nice day,
Maxime B.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/groupgets/GetThermal/issues/24?email_source=notifications&email_token=AFPP7Y2FY4NTK7BLZFPEPKDQH2FAVA5CNFSM4ITIJNR2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HJA7YDA,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFPP7Y5Q7ZUWSD7UUOZKPZ3QH2FAVANCNFSM4ITIJNRQ
.

Has this issue been resolved?


#UseHook On is rejected

#UseHook On is valid code in v1.1, but is rejected as invalid by the converter.
#UseHook On should be treated the same as #UseHook without an argument (as per documentation).

This only happens when I run v2converter.ahk though... Not sure what's up.

Interesting. There should be no differences between using v2convertor or
QuickConverter. Will have to look into it. Thanks for the report
On Sat, Feb 24, 2024, 3:53 PM Irfan Adilovic @.***>
wrote:

This only happens when I run v2converter.ahk though... Not sure what's up.
—
Reply to this email directly, view it on GitHub
https://github.com/mmikeww/AHK-v2-script-converter/issues/142#issuecomment-1962729167,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOQBDFJZDTOZEPI4UQNUTTYVJHMHAVCNFSM6AAAAABDYGYXYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSG4ZDSMJWG4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Both scripts convert #UseHook On to use #UseHook true (valid code)
Could you share your script?


Selective semi-colon issues when submitting queries

This might be a mapd-connector issue, but from the MapD Explorer window in Jupyterlab occasionally an error will get thrown when the query has a semi-colon at the end:
Works:
select member_birth_year, count(*) as records from fordgobike_tripdata_v2 group by 1;

Throws error:
select member_birth_year, count(*) as records from fordgobike_tripdata_v2 group by 1 order by 1;

Uncaught (in promise) Error: Exception: Parse failed: Encountered ";" at line 1, column 96.
Was expecting one of:
    <EOF>
    "LIMIT" ...
    "OFFSET" ...
    "FETCH" ...
    "," ...
    "ASC" ...
    "DESC" ...
    "NULLS" ...
    "NOT" ...

    "BETWEEN" ...
    "LIKE" ...
    "SIMILAR" ...
    "+" ...
    "-" ...
    "*" ...
    "/" ...
    "%" ...
    "||" ...
    "AND" ...
    "OR" ...
    "IS" ...
    "MEMBER" ...
    "SUBMULTISET" ...
    "CONTAINS" ...
    "OVERLAPS" ...
    "EQUALS" ...
    "PRECEDES" ...
    "SUCCEEDS" ...
    "IMMEDIATELY" ...
    "MULTISET" ...
    "[" ...

    at s (browser-connector.js:1737)
    at browser-connector.js:1760
    at eval (eval at e.exports (browser-connector.js:83), <anonymous>:10142:7)
    at XMLHttpRequest.eval (eval at e.exports (browser-connector.js:83), <anonymous>:349:25)

As a quick hack, might just chomp the ; off the end of the string since it's not needed, but should probably figure out the real issue long-term

Just as a note, I performed the same query (with just a table name substitution) using the latest browser mapd-connector against a latest Core server and cannot reproduce the error with the semicolon (that is, it works fine).


JupyterHub Auth with Environment Variables

Automatically login students using env var tokens (which are fetched by JupyterHub when the user logs in)
No rush on this - as its going to be a few weeks before they deploy this.

I have a couple questions:

What happens if the OKPY_ACCESS_TOKEN is expired? We should refresh, but right now it'll just return the expired token.
If both OKPY_ACCESS_TOKEN and a stored token exists, which one should we use? One possibility would be the one that expires latest, but there might be a simpler solution.

I think it would be easiest to answer those questions if the code was moved to assignment.authenticate.

A possible non-notebook use case: using ok-client for scripting (if you have an access token but don't want to log in interactively)

What happens if the OKPY_ACCESS_TOKEN is expired? We should refresh, but right now it'll just return the expired token.

Right now the function checks to make sure the token actually returns an email before saving the token in the environment variable - so it's no big deal if it's expired.

If both OKPY_ACCESS_TOKEN and a stored token exists, which one should we use? One possibility would be the one that expires latest, but there might be a simpler solution.

Not sure - I think checking for an existing token before using OKPY_ACCESS_TOKEN might make the code more complicated. I'll think about this some more.

@epai  - do you think you could handle some of the comments for this PR? It'd be good to have this merged by summer so datascience folks can use this

Yeah, I'll take a look after my finals end on Tuesday


Invertibility of Glow

Hi Yujin,
Thanks for the nice implementation. From your reconstruction demo, it seems that the implementation cannot guarantee full invertibility of glow. Could you comment on it?
Best,
Kede

Hi, @makedede .
I want to figure out what do you mean with invertibility.

If you mean invertibility as the ability to reconstruct original input, you can't reconstruct the input exactly with only final z ~ N(0,1) (which is the method used in my demo). Because, there are several Split2d layers which early output z  ~ N(0,1). You have to keep recording of all early-outputed z from Split2d layers to fully reconstruct original input.
If you mean invertibility as the bijectiveness of modules, I think each module,  asking reverse input in my code, is reversable.

Thanks for your reply. I’ve sent you an email to your qq account. Would you please to take a look at it? Many thanks.

Then I understand that you only make use of the coarsest scale of z for reconstruction and the blurry images make sense to me.

Hi,  @chaiyujin,
Thanks for the reimplement of glow in pytorch.
I think it is not bad to keep the same dimensions of Z as the input image. Reducing dimensions results in information loss.
Your code is very helpful, thanks again, anyway.

@plutoyuxie
Thanks for your attention.
What do you mean about "keep the same dimensions of Z as the input image"?
If the input image is (3, H, W), what the shape of Z in your case?

Hi, @chaiyujin,
Thanks for your response.
In your case, the input image shape is (3, 64, 64) and L = 3.
The total dimensions of z should be equal to 3X64X64=12,288.
After discarding the first part of z (6, 32, 32)  and the second part of z (12, 16, 16),
only 1/4 dimensions of z (48, 8, 8) left.
That leads to bad reconstruction result because of some information loss.
I think it is kind of limitation for flow-based models. The large dimensions of z seem to be wasteful but necessary.


openscad: Add binary

Description
Link the openscad binary analogous to this PR in homebrew-cask so it can be used as CLI.
Verification

[x] The submission is for a stable version or documented exception. This is a nightly snapshot and therefore possibly unstable.
[x] brew audit --strict --cask --online openscad-snapshot is error-free.
[x] brew style --fix openscad-snapshot reports no offenses.
[x] openscad --version works after installing the cask with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source openscad-snapshot.

I followed How to Open a Homebrew Pull Request — Homebrew Documentation. brew tests failed around 50 tests. I don't think this change could have possibly caused them.
Furthermore brew test openscad-snapshot cannot find the formula. I assume the test command only works with homebrew/core then?

gh did not create this PR in the original repo.


bax2bam needs to be on github

The point of #61 is absent without a way to marshall data into ccs-compatible form.

@bnbowman , since I think you had expressed interest in becoming the point-man for bax2bam, I think ti would make sense for you to look into this.  Because of internal dependencies, this is going to have to be done using git-p4, we can't move the canonical repo to git.  Let me email you my slide deck on how to use git-p4 in our environment.

@dalexander I am also very interested in testing the bax2bam. Can you also email me the software? Thanks!

it is! now included with blasr


prefixfree generates errors when viewing a local file

When viewing files locally that use prefixfree, I see errors similar to the following:
XMLHttpRequest cannot load file:///Users/jessamyn/Development/llc-intro-to-jquery/framework/css/theme-llc.css. Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https, chrome-extension-resource.
I recognize that I can work around this by specifying data-noprefix but I would prefer not to have to do so.

FAQ, third question. http://leaverou.github.io/prefixfree/#faq


view article functionality (after save)

https://aws-amplify.github.io/docs/js/storage#get

generate a preview url for the article and the article index.
**the storage folder should contain all the necessary css & js files to serve the preview with the correct styling.


RedisAdvancedClusterAsyncCommands does not include transactional commands

Rocking Lettuce 5.0.0 here.
The transactional commands are not included in the RedisAdvancedClusterAsyncCommands interface but are included in the corresponding class, RedisAdvancedClusterAsyncCommandsImpl.
I believe RedisClusterAsyncCommands should extend RedisTransactionalAsyncCommands.

Thanks for raising this issue. Transactional commands are not included in RedisAdvancedClusterAsyncCommands by design. Transactions are not possible in Redis Cluster from a cluster perspective. Transactions are only possible from a node perspective:
StatefulRedisClusterConnection<String, String> connection = …;

StatefulRedisConnection<String, String> nodeConnection = connection.getConnection(host, port);

RedisAsyncCommands<String, String> async = nodeConnection.async();

async.multi();

// …

RedisFuture<TransactionResult> exec = async.exec();

It boils down to the absence of a distributed transaction notion in Redis.
References:

StackOverflow: Is there any Redis client (Java prefered) which supports transactions on Redis cluster?
Blog: Redis Transactions

Thanks, that makes sense given the design.


Teleport working not correctly

The character is lightning sorc. At first, the teleport to the Pindel works fine. But after the occurrence of this message in the log " Doing runs: odict_keys(['run_pindle']) ", teleportation shifts indoors from the Pendel to the left and up. Not much, but it leads to the death of the character, as teleportation takes place in a crowd of monsters.
Can I remove this procedure or disable it?

probably your cast rate isnt 11ish or your missing a prebuff hotkey. Or your running vm with not enough resources.

probably your cast rate isnt 11ish or your missing a prebuff hotkey. Or your running vm with not enough resources.

TNX!!! cast rate - 8! Changed to 11 and voila!

I would say check the fcr breakpoints for your correct cast rate frame and change it as indicated. @D2RLegit Sounds right with what has been stated.


cat

pull celestia/cmwaters CAT mempool implementation into cometbft
feat: mempools should correctly respect time based TTL (#1088)

@faddat seems like a good approach!

There are still some tests that aren't passing, if you'd like to help :).

Closing in favor of #1472. We have been in touch with Celestia team regarding CAT mempool performance profile and have been doing investigations for a few weeks. Let's continue in the other PR.


chore: freeze azlinux images until 1.29 previews

What type of PR is this?
AzureLinux images are not going to be consumed until 1.29 previews. This PR freezes AzureLinux images to the last publish image and also stops building it for weekly VHD building pipelines.

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.095%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2255

Relevant Lines:
2851

💛  - Coveralls


Create pods!

It's alive! Pods are creating automatically and running through the pipeline!

It's alive! Pods are creating automatically and running through the pipeline!


Convert Lab* to RGB

Find a package that converts LAB color codes into RGB.
Wikipedia article on L*a*b*: https://en.wikipedia.org/wiki/CIELAB_color_space
"The lightness value, L*, also referred to as "Lstar," defines black at 0 and white at 100. The a* axis is relative to the green–red opponent colors, with negative values toward green and positive values toward red. The b* axis represents the blue–yellow opponents, with negative numbers toward blue and positive toward yellow."
"The a and b* axes are unbounded, and depending on the reference white they can easily exceed ±150 to cover the human gamut. Nevertheless, software implementations often clamp these values for practical reasons. For instance, if integer math is being used it is common to clamp a* and b* in the range of −128 to 127."*
Possible libraries achieving that:

https://www.npmjs.com/package/rgb-lab
https://www.npmjs.com/package/colord : a lot of downloads and in typescript

In the end, we use the colord library and convert the colors like this:
import { colord, extend } from 'colord';
import labPlugin from 'colord/plugins/lab';

extend([labPlugin]);

console.log(colord({ r: 255, g: 0, b: 0 }).toLab());
console.log(colord({ l: 50, a: -128, b: -128 }).toRgb());


[Bug] :  episodes dont play and cannot skip intro when some do i cant go to a certain part of a episode or it will forever load

Device information
-Tachiyomi version: <IP_ADDRESS>

Android version: 12
-Samsung galaxy S21 5G

Source information

Name:  animepahe
Extension version: English 12.13

Steps to reproduce
1 episode didn't play
2 episode should have played

I hope this gets fixed