feat(Message): add different types

ChatMessage
This PR aims to add different types of messages for the chat. We should support the following types:

control messages

bubble messages (this is currently the implementation of the ChatMessage itself)

divider messages

TODO

[ ] Conformance test
[ ] Minimal doc site example
[ ] Stardust base theme
[ ] Teams Light theme
[ ] Teams Dark theme
[ ] Teams Contrast theme
[ ] Confirm RTL usage
[ ] W3 accessibility check
[ ] Stardust accessibility check
[ ] Update glossary props table
[ ] Update the CHANGELOG.md

I have three proposals and would like to hear your opinion on.
API Proposal 1
type: 'bubble' | 'control' | 'divider'
The type property will be used for rendering one of the three different components. The properties will remain the same, we would just add additional icon shorthand for the control component.
Usage:
<ChatMessage type='bubble' author={...} ... />
<ChatMessage type='control' icon={...} ... />
<ChatMessage type='divider' content={'Yesterday'} ... />

This would be easy for use, but not much open for fully customizing the generated component.
API Proposal 2
bubble: shorthand for the Message.Bubble component
control: shorthand for the Message.Control component
divider: shorthand for the Message.Divider component
Open for customization and consistent with the other usages of the shorthand. One problem is that, a message cannot be at the same time two of these types, so we will kind a have to decide behind the scene which would take precedence.
Usage
<ChatMessage bubble={{author: 'John Doe', ...}} ... />
<ChatMessage control={{icon: {{...}}, content='Some content...'}} ... />
<ChatMessage divider={{content: '.Yesterday}}... />

API Proposal 3
This proposal is combination of the previous two. We should use the type for deciding which component will be rendered, but will use the shorthand for generating the component itself.
type: 'bubble' | 'control' | 'divider'
bubble: shorthand for the Message.Bubble component
control: shorthand for the Message.Control component
divider: shorthand for the Message.Divider component
<ChatMessage type='bubble' bubble={{author: 'John Doe', ...}} ... />
<ChatMessage type='control'  control={{icon: {{...}}, content='Some content...'}} ... />
<ChatMessage type='divider' divider={{content: '.Yesterday}}... />

From those options that are provided would rather votew against the third one, due to that it opens way to (accidentally) do things improperly on the consumer's side:
<ChatMessage type='divider' bubble={{author: 'John Doe', ...}} ... />

Speaking of the rest two - the safest approach would be to start with just different type values being supported (no shorthands for now), and see how it will go afterwards; although I should notice that I do really like a shorthand-based approach suggested as well

Got it, I agree it's the easiest from user perspective as well. Will start with the Proposal 1, we can easily migrate to 2 if we see it would be needed.

Actually, let's go directly with the Proposal 2, because even now I can see some problems with allowing the user to define different types for the Divider for example. I would like to avoid having to change the whole API rather soon...

I'd go also with the second proposal.
One thing though: why do we need the divider type of the message? Can't divider be simply a divider and used where needed?

Not really in this moment, as the messages are collection shorthand inside the chat component, so we aren't really capable of adding some other component in between..

To be more specific the chat definition would look something like this:
<Chat items={[
    {{bubble: {avatar: {...}, content: '...'}}},
    {{bubble: {avatar: {...}, content: '...'}}},
    {{control: {icon: {...}, content: 'Meeting started...'}}}, //now that I think about this, maybe the better naming would be action
    {{divider: {content: 'Today'}}},
    {{bubble: {avatar: {...}, content: '...'}}},
  ]}

So the general change is that, the chat messages will be transformed to chat items, which can be messages, dividers, action etc..

Still work in progress, but this is how it currently looks:

And here is the code for that example:
import React from 'react'

import { Chat } from '@stardust-ui/react'

const janeAvatar = {
  src: 'public/images/avatar/small/ade.jpg',
  status: {
    color: 'green',
    icon: 'check',
    title: 'Available',
  },
}

const items = [
  {
    key: 1,
    bubble: { content: 'Hello', author: 'John Doe', timestamp: 'Yesterday, 10:15 PM', mine: true },
  },
  {
    key: 2,
    bubble: {
      content: 'Hi',
      author: 'Jane Doe',
      timestamp: 'Yesterday, 10:15 PM',
      avatar: janeAvatar,
    },
  },
  {
    key: 3,
    bubble: {
      content: 'Would you like to grab a lunch?',
      author: 'John Doe',
      timestamp: 'Yesterday, 10:16 PM',
      mine: true,
    },
  },
  {
    key: 4,
    bubble: {
      content: "Sure! Let's try the new place downtown",
      author: 'Jane Doe',
      timestamp: 'Yesterday, 10:16 PM',
      avatar: janeAvatar,
    },
  },
  { key: 5, divider: { content: 'Today', type: 'primary', important: 'true' } },
  {
    key: 6,
    bubble: {
      content: "Let's have a call",
      author: 'John Doe',
      timestamp: 'Today, 11:15 PM',
      mine: true,
    },
  },
  {
    key: 7,
    action: { icon: 'record', content: 'Meeting started', timestamp: "'Today, 11:15 PM'" },
  },
]

const ChatExampleShorthand = () => <Chat items={items} />

export default ChatExampleShorthand

@kuzhelov @alinais please give me your feedback about the API.

We have some doubts whether the Divider should be define as an attribute to the ChatItem, as it is not really a message in the context of the Chat component. We can provide props like lastRead and firstMessageOnDay together with some shorthand for defining the Divider before or after the message. Although this provide more 'chat like' API, my worries are that those are kind a restrictive to just these two scenarios. We could define before and after dividers too, but that would be too much generic for the ChatItem (not much different than the List). I would like to hear your opinion on this, and what makes more sense from user's perspective.

We have some doubts whether the Divider should be define as an attribute to the ChatItem, as it is not really a message in the context of the Chat component. We can provide props like lastRead and firstMessageOnDay together with some shorthand for defining the Divider before or after the message. Although this provide more 'chat like' API, my worries are that those are kind a restrictive to just these two scenarios. We could define before and after dividers too, but that would be too much generic for the ChatItem (not much different than the List). I would like to hear your opinion on this, and what makes more sense from user's perspective.

I like an idea to make a divider as an attribute to a message. But maybe, to avoid restrictions just to those 2 mentioned scenarios (often chats have 'yesterday' and 'someday' dividers), we can provide just props before and after, and a user will define the content ('last read', 'yesterday', 'today', etc).

Recapping our offline chat about Stardust's guidelines for creating components and APIs:

Syntax is evil, avoid creating it at all costs.  Prefer patterns, composition, and reuse instead.
Stardust will not contain any original components, component slots, or layouts.
Stardust only creates components and APIs when we can find a concept in widespread usage in many well-known UIs.
Never invent a component or API based on a single use case or our feelings.

The conclusion to the Chat API for now was to provide mostly or only the children API for now.  As we have more time and use cases to standardize on we will.  Right now, the correct abstraction is not clear so we'll let users compose their own abstractions using our lower level components for Chat.

So, after the discussions we had, the components are updated to provide the user to use the Chat with different components that should be used as children of the Chat.Item component that represents just a row inside the Chat. For example, here is what the code would look like for rendering the Chat provided in the picture.

import React from 'react'

import { Chat, Divider } from '@stardust-ui/react'

const janeAvatar = {
  src: 'public/images/avatar/small/ade.jpg',
  status: {
    color: 'green',
    icon: 'check',
    title: 'Available',
  },
}

const ChatExample = () => (
  <Chat>
    <Chat.Item>
      <Chat.Message content="Hello" author="John Doe" timestamp="Yesterday, 10:15 PM" mine />
    </Chat.Item>
    <Chat.Item>
      <Chat.Message
        content="Hi"
        author="Jane Doe"
        timestamp="Yesterday, 10:15 PM"
        avatar={janeAvatar}
      />
    </Chat.Item>
    <Chat.Item>
      <Chat.Message
        content="Would you like to grab a lunch?"
        author="John Doe"
        timestamp="Yesterday, 10:16 PM"
        mine
      />
    </Chat.Item>
    <Chat.Item>
      <Chat.Message
        content="Sure! Let's try the new place downtown"
        author="Jane Doe"
        timestamp="Yesterday, 10:15 PM"
        avatar={janeAvatar}
      />
    </Chat.Item>
    <Chat.Item>
      <Divider content="Today" type="primary" important />
    </Chat.Item>
    <Chat.Item>
      <Chat.Message
        content="Let's have a call"
        author="John Doe"
        timestamp="Today, 11:15 PM"
        mine
      />
    </Chat.Item>
  </Chat>
)

export default ChatExample

For generating the Action messages, the user can use the Accordion component, or the Chat.Message component with providing transparent background and using the avatar as an icon. Moreover, the Divider can be used together with the Chat.Message, or as a separate Chat.Item. More example will be provided in the Chat Pane Prototype.
Question
One issue we have now, after removing the usage of the slots inside the styles property is that I had to add the mainStyle and avatarStyle properties inside the Chat.Message, because the layout is not a shorthand, and we cannot pass these styles any other way. Am I missing something, do we have another way to pass these?

Note, we'll remove the temporary styles properties in this PR.   Once the Layout is fixed, https://github.com/stardust-ui/react/pull/236, we can use the shorthand styles properly for the Layout areas.

This is ready for final review @levithomason @kuzhelov @miroslavstastny


Fdroid repo error with new update

Something wrong with the repo. I get ..."repo failed to verify" . Tried to remove and add repo again but same error. Linage Oreo, Magisk and Xposed

Repo is working fine for me. I just upgraded microg using F-Droid 1.3.1.
URL: https://microg.org/fdroid/repo
Fingerprint: 9B D0 67 27 E6 27 96 C0 13 0E B6 DA B3 9B 73 15 74 51 58 2C BD 13 8E 86 C4 68 AC C3 95 D1 41 65


Doesn't work in src/main/kotlin

After AGP 7.0 we can use src/main/kotlin as kotlin sourceSet by default, but the kotlinter doesn't work in this location.
See JLLeitschuh/ktlint-gradle#524.

Indeed, thanks for the report. Being addressed in #229


no smart cropping in 11?

Version 3.x introduced smarty entropy based cropping, but this support appears to have gone in 11.x?
Would be great to have this functionality restored.
(Or I'm being stupid, but I'm getting Unrecognized crop option: smart and looking in processors.py shows no sign of smart crop)

Yes its gone :/ also there is no processors.py :). Patches are welcome!


Privacy manifest

This repo needs to provide a privacy manifest, as described here: https://developer.apple.com/news/?id=av1nevon, thank a lot.

Hello,
This is very important for us too.
At WWDC23 Apple announced that apps and SDKs that make use of certain "required reason" APIs etc will need to provide a privacy manifest.
Does Device need to include this manifest?
Is this update on the roadmap for the team? I appreciate that enforcement won't happen until Spring 2024 but I wanted to make contact so that we can plan our own app releases to take this into account.
Here’s some useful references:
https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests
https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api
https://developer.apple.com/videos/play/wwdc2023/10060/
Thanks

bump

@tristanhimmelman any plans to include a privacy manifest? Apple are starting to enforce it.
Thank you

@tristanhimmelman just bumping thread so as not to forget about Privacy Manifest (if needed). Thanks.


Is there any way to deserialize to Box<dyn Any>?

Is there any way to deserialize a self-describing format to Box<dyn Any>?
This would be useful if I have an unspecialized struct like
#[derive(Serialize, Deserialize)]
struct Entry {
    r#type: String,
    payload: Box<dyn Any>
}

where the payload can be anything depending on the r#type.

Maybe something like this:
use serde::{Deserialize, Deserializer};
use serde_json::Value;
use std::any::Any;

#[derive(Deserialize, Debug)]
pub struct Entry {
    pub r#type: String,
    #[serde(deserialize_with = "any_value")]
    pub payload: Box<dyn Any>,
}

fn any_value<'de, D>(deserializer: D) -> Result<Box<dyn Any>, D::Error>
where
    D: Deserializer<'de>,
{
    let value = Value::deserialize(deserializer)?;
    Ok(Box::new(value))
}

fn main() {
    let j = r#"
    {"type": "array", "payload": []}
    "#;

    println!("{:#?}", serde_json::from_str::<Entry>(j).unwrap());
}

Also take a look at https://github.com/dtolnay/typetag which is possibly more appropriate for your use case than dyn Any.

Oh, apparently I didn't phrase my question very well – I was wondering how I could extend my own Serializer/Deserializer to allow the work with type-erased objects. However I found a way using a custom visitor and erased-serde: https://github.com/KizzyCode/serde_asn1_der-rust/blob/master/src/any.rs
Ty 😊


fix workload spell error

@tengqm

rebase first, please

@zhangxiaoyu-zidif The conflict has been resolved.

/lgtm

/assign @bradtopol

/lgtm cancel

ok, thanks!


Grafana setup ignoring defined certificates which cause errors in dashboard

Bug Report
What happened:
ceph-ansible is skipping the copy of defined certificates which causes errors in performance dashboards. The http_addr setting is also an issue, since it picks the IP of the server, but certificates are issued for a domain. So if I head to performance dashboards with generated Grafana certs, I get the error: Error code: MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT. If i replace the certificates manually, most of dashboards are working, but

Dashboard continuously pops the red error "500 - Internal Server Error"
Pools performance dashboard is broken and I additionally get "TypeError: l.c[t.type] is undefined" error
ceph-mgr log still reports an error:

SSLError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=3000): Max retries exceeded with url: /api/dashboards/uid/z99hzWtmk (Caused by SSLError(CertificateError("hostname '<IP_ADDRESS>' doesn't match '*.dc1.redacted.com'",),))

Setting the http_addr manually to domain also doesn't help =/

How to reproduce it (minimal and precise):
Just run the rolling update playbook with dashboard settings and certificates defined.
inventory
[mons]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[osds]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[mgrs]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[rgws]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[nfss]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[grafana-server]
ceph1-40-10.dc1.redacted.com
ceph1-40-11.dc1.redacted.com
ceph1-40-12.dc1.redacted.com

[ceph:children]
mons
osds
mgrs
rgws
nfss
grafana-server

snippet from dashboard config in group vars (everything else is default)
dashboard_enabled: True
dashboard_protocol: https
dashboard_port: 8443
dashboard_admin_user: admin
dashboard_admin_password: *****************************
dashboard_crt: '/etc/ssl/certs/companycert.pem'
dashboard_key: '/etc/ssl/private/companycert-key.pem'
node_exporter_port: 9100
grafana_admin_user: admin
grafana_admin_password: **********************
grafana_crt: '/etc/ssl/certs/companycert.pem'
grafana_key: '/etc/ssl/private/companycert-key.pem'

How the grafana.ini looks like after deployment
[users]
default_theme = light

[auth.anonymous]
# enable anonymous access
enabled = true

# specify organization name that should be used for unauthenticated users
org_name = Main Org.

# specify role for unauthenticated users
org_role = Viewer

[server]
cert_file = /etc/grafana/ceph-dashboard.crt      <--- those were generated by ansible-run
cert_key = /etc/grafana/ceph-dashboard.key    <--- those were generated by ansible-run
domain = ceph1-40-10.dc1.redacted.com
protocol = https
http_port = 3000
http_addr = <IP_ADDRESS>

[security]
admin_user = admin
admin_password = *********************
allow_embedding = True

grafana snippet from ansible log
2020-02-12 12:17:36,082 p=24266 u=root |  TASK [ceph-grafana : ship systemd service] *************************************************************************************************
2020-02-12 12:17:36,083 p=24266 u=root |  Wednesday 12 February 2020  12:17:36 +0100 (0:00:00.731)       0:30:21.922 ****
2020-02-12 12:17:36,558 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:36,704 p=24266 u=root |  TASK [ceph-grafana : start the grafana-server service] *************************************************************************************
2020-02-12 12:17:36,704 p=24266 u=root |  Wednesday 12 February 2020  12:17:36 +0100 (0:00:00.621)       0:30:22.543 ****
2020-02-12 12:17:37,201 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:37,352 p=24266 u=root |  TASK [ceph-grafana : include configure_grafana.yml] ****************************************************************************************
2020-02-12 12:17:37,352 p=24266 u=root |  Wednesday 12 February 2020  12:17:37 +0100 (0:00:00.647)       0:30:23.191 ****
2020-02-12 12:17:37,649 p=24266 u=root |  included: /etc/ansible/ceph-ansible/roles/ceph-grafana/tasks/configure_grafana.yml for ceph1-40-10.dc1.redacted.com
2020-02-12 12:17:37,802 p=24266 u=root |  TASK [ceph-grafana : install ceph-grafana-dashboards package on RedHat or SUSE] ************************************************************
2020-02-12 12:17:37,803 p=24266 u=root |  Wednesday 12 February 2020  12:17:37 +0100 (0:00:00.450)       0:30:23.642 ****
2020-02-12 12:17:37,909 p=24266 u=root |  skipping: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:38,058 p=24266 u=root |  TASK [ceph-grafana : make sure grafana is down] ********************************************************************************************
2020-02-12 12:17:38,059 p=24266 u=root |  Wednesday 12 February 2020  12:17:38 +0100 (0:00:00.255)       0:30:23.898 ****
2020-02-12 12:17:38,756 p=24266 u=root |  changed: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:38,914 p=24266 u=root |  TASK [ceph-grafana : wait for grafana to be stopped] ***************************************************************************************
2020-02-12 12:17:38,914 p=24266 u=root |  Wednesday 12 February 2020  12:17:38 +0100 (0:00:00.855)       0:30:24.753 ****
2020-02-12 12:17:39,612 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:39,703 p=24266 u=root |  TASK [ceph-grafana : make sure grafana configuration directories exist] ********************************************************************
2020-02-12 12:17:39,703 p=24266 u=root |  Wednesday 12 February 2020  12:17:39 +0100 (0:00:00.789)       0:30:25.542 ****
2020-02-12 12:17:39,923 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=/etc/grafana/dashboards/ceph-dashboard)
2020-02-12 12:17:40,154 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=/etc/grafana/provisioning/datasources)
2020-02-12 12:17:40,406 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=/etc/grafana/provisioning/dashboards)
2020-02-12 12:17:40,665 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=/etc/grafana/provisioning/notifiers)
2020-02-12 12:17:40,823 p=24266 u=root |  TASK [ceph-grafana : download ceph grafana dashboards] *************************************************************************************
2020-02-12 12:17:40,823 p=24266 u=root |  Wednesday 12 February 2020  12:17:40 +0100 (0:00:01.119)       0:30:26.662 ****
2020-02-12 12:17:42,240 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=ceph-cluster.json)
2020-02-12 12:17:42,992 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=cephfs-overview.json)
2020-02-12 12:17:43,830 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=host-details.json)
2020-02-12 12:17:44,611 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=hosts-overview.json)
2020-02-12 12:17:45,384 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=osd-device-details.json)
2020-02-12 12:17:46,332 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=osds-overview.json)
2020-02-12 12:17:47,209 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=pool-detail.json)
2020-02-12 12:17:47,988 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=pool-overview.json)
2020-02-12 12:17:48,784 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=radosgw-detail.json)
2020-02-12 12:17:49,598 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=radosgw-overview.json)
2020-02-12 12:17:50,381 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com] => (item=rbd-overview.json)
2020-02-12 12:17:50,562 p=24266 u=root |  TASK [ceph-grafana : write grafana.ini] ****************************************************************************************************
2020-02-12 12:17:50,563 p=24266 u=root |  Wednesday 12 February 2020  12:17:50 +0100 (0:00:09.739)       0:30:36.402 ****
2020-02-12 12:17:51,136 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:51,290 p=24266 u=root |  TASK [ceph-grafana : write datasources provisioning config file] ***************************************************************************
2020-02-12 12:17:51,290 p=24266 u=root |  Wednesday 12 February 2020  12:17:51 +0100 (0:00:00.727)       0:30:37.129 ****
2020-02-12 12:17:51,780 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:51,932 p=24266 u=root |  TASK [ceph-grafana : Write dashboards provisioning config file] ****************************************************************************
2020-02-12 12:17:51,932 p=24266 u=root |  Wednesday 12 February 2020  12:17:51 +0100 (0:00:00.641)       0:30:37.771 ****
2020-02-12 12:17:52,424 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:52,579 p=24266 u=root |  TASK [ceph-grafana : copy grafana SSL certificate file] ************************************************************************************
2020-02-12 12:17:52,580 p=24266 u=root |  Wednesday 12 February 2020  12:17:52 +0100 (0:00:00.647)       0:30:38.419 ****
2020-02-12 12:17:52,672 p=24266 u=root |  skipping: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:52,822 p=24266 u=root |  TASK [ceph-grafana : copy grafana SSL certificate key] *************************************************************************************
2020-02-12 12:17:52,822 p=24266 u=root |  Wednesday 12 February 2020  12:17:52 +0100 (0:00:00.242)       0:30:38.661 ****
2020-02-12 12:17:52,915 p=24266 u=root |  skipping: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:53,061 p=24266 u=root |  TASK [ceph-grafana : generate a Self Signed OpenSSL certificate for dashboard] *************************************************************
2020-02-12 12:17:53,062 p=24266 u=root |  Wednesday 12 February 2020  12:17:53 +0100 (0:00:00.239)       0:30:38.901 ****
2020-02-12 12:17:53,302 p=24266 u=root |  changed: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:53,452 p=24266 u=root |  TASK [ceph-grafana : enable and start grafana] *********************************************************************************************
2020-02-12 12:17:53,452 p=24266 u=root |  Wednesday 12 February 2020  12:17:53 +0100 (0:00:00.390)       0:30:39.291 ****
2020-02-12 12:17:53,946 p=24266 u=root |  changed: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:54,095 p=24266 u=root |  TASK [ceph-grafana : wait for grafana to start] ********************************************************************************************
2020-02-12 12:17:54,095 p=24266 u=root |  Wednesday 12 February 2020  12:17:54 +0100 (0:00:00.642)       0:30:39.934 ****
2020-02-12 12:17:58,350 p=24266 u=root |  ok: [ceph1-40-10.dc1.redacted.com]
2020-02-12 12:17:58,509 p=24266 u=root |  TASK [set ceph grafana install 'Complete'] *************************************************************************************************

Additional informations:

Copying the same certificate for dashboard works fine, so it's not an issue with certificates itself.

Environment:

OS (e.g. from /etc/os-release): Ubuntu 18.04.3 LTS
Kernel (e.g. uname -a): Linux ceph1-40-10 5.0.0-16-generic
Docker version if applicable (e.g. docker version): 18.09.7
Ansible version (e.g. ansible-playbook --version): ansible-playbook 2.8.6
ceph-ansible version (e.g. git head or tag or stable branch): stable-4.0
Ceph version (e.g. ceph -v): ceph version 14.2.7 (3d58626ebeec02d8385a4cefb92c6cbc3a45bfe8) nautilus (stable)

I think I've spotted the crt/key issue when provided custom values.
For the other issue about http_addr IP vs FQDN this might take more time to investigate and provide something.
As a temp workaround could you try running ceph dashboard set-grafana-api-ssl-verify False and see if it helps ?

@dsavineau I also found this command in the docs and tried it out. It turns out, my ceph dashboard doesn't have this command. Any ideas why?
tab-tab result:
root@ceph1-40-10~# ceph dashboard set-
set-alertmanager-api-host                  set-ganesha-clusters-rados-pool-namespace  set-grafana-update-dashboards              set-prometheus-api-host                    set-rgw-api-host                           set-rgw-api-ssl-verify
set-audit-api-enabled                      set-grafana-api-password                   set-iscsi-api-ssl-verification             set-rest-requests-timeout                  set-rgw-api-port                           set-rgw-api-user-id
set-audit-api-log-payload                  set-grafana-api-url                        set-jwt-token-ttl                          set-rgw-api-access-key                     set-rgw-api-scheme                         set-ssl-certificate
set-enable-browsable-api                   set-grafana-api-username                   set-login-credentials                      set-rgw-api-admin-resource                 set-rgw-api-secret-key                     set-ssl-certificate-key

@dsavineau I also found this command in the docs and tried it out. It turns out, my ceph dashboard doesn't have this command. Any ideas why?

No I also saw this in the documentation but this seems to not be present in the ceph dashboard code :(

I spoke too quickly, the option is present indeed
https://github.com/ceph/ceph/blob/nautilus/src/pybind/mgr/dashboard/settings.py#L42
Even if it's not present in the auto completion can you give it a try ? (if not already done)

@dsavineau Already tried it. Spooky :)
root@ceph1-40-10~# ceph dashboard set-grafana-api-ssl-verify False
no valid command found; 10 closest matches:
dashboard set-grafana-api-username <value>
dashboard get-audit-api-log-payload
dashboard get-grafana-api-password
dashboard get-grafana-api-username
dashboard set-rgw-api-access-key <value>
dashboard reset-enable-browsable-api
dashboard set-rgw-api-host <value>
dashboard set-grafana-api-password <value>
dashboard get-rgw-api-access-key
dashboard get-rgw-api-host
Error EINVAL: invalid command

@epuertat valid ceph dashboard bug ?
That's weird because the other ssl verify options (set-iscsi-api-ssl-verification and set-rgw-api-ssl-verify) are declared the same way but are present

@styleart are you using containerized ceph setup ? (ie: containerized_deployment: true)
On Centos 7 and Nautilus 14.2.7, the command is present
# ceph --version
ceph version 14.2.7 (3d58626ebeec02d8385a4cefb92c6cbc3a45bfe8) nautilus (stable)
# ceph dashboard set-grafana-api-ssl-verify
Invalid command: missing required parameter value(<string>)
dashboard set-grafana-api-ssl-verify <value> :  Set the GRAFANA_API_SSL_VERIFY option value
Error EINVAL: invalid command

@dsavineau nope, it's not containerized.

On Centos 7 and Nautilus 14.2.7, the command is present

@styleart Ok I've probably messed up with my different ceph environments because my previous comment is wrong
In fact the set-grafana-api-ssl-verify has been added to nautilus but after 14.2.7 [1]
That's why we can see it in [2] but not in [3]
So the command isn't present in 14.2.7 (it will be present probably in 14.2.8)
And I was looking at the ceph dashboard documentation from master [4]
[1] https://github.com/ceph/ceph/commit/a83e839b215f3ee89dfc6b249d4bd1db3798068d
[2] https://github.com/ceph/ceph/blob/nautilus/src/pybind/mgr/dashboard/settings.py#L42
[3] https://github.com/ceph/ceph/blob/v14.2.7/src/pybind/mgr/dashboard/settings.py#L38-L42
[4] https://docs.ceph.com/docs/master/mgr/dashboard/#enabling-the-embedding-of-grafana-dashboards

@dsavineau aaah, alright, that makes sense! Well, then I'll wait until the 14.2.8 release and try out if it works. Btw, the command is also present in current nautilus dashboard docs: https://docs.ceph.com/docs/nautilus/mgr/dashboard/
Thank you for the awesome feedback! :)


What does one need for FCC? (Suggestion for improving the Field Guide)

After reading the field guide there was only one question that I didn't get answer for what do I need? What knowledge do I need to start FCC? I kinda suspect that none, but this should be included in Field Guide. (I am really sorry for confusion guys)

Basically nothing, you will learn anything you need to know when you start the program. So, yeah, nothing.

Like @ckent20 said. FreeCodeCamp is supposed to allow people with no knowledge of programming to get up and running, whether you do that by just tackling the challenges or if you need more help from the chatroom.

Mmb tMf x%mfMdmkf x xbxfTaFf@MfAfafa dwfF dmhfFfMfFf zmffafFM DFtatatata
sfFMtafagz ffff ztafF xtaFBdfTabs "% \√¢°€
On Jul 19, 2015 6:22 PM, "Joshua Rodriguez"<EMAIL_ADDRESS>wrote:

Basically nothing, you will learn anything you need to know when you start
the program. So, yeah, nothing.
—
Reply to this email directly or view it on GitHub
https://github.com/FreeCodeCamp/freecodecamp/issues/1336#issuecomment-122726016
.

@paulacolburm Whats going on with that comment?

Ok, guys, sorry for not expressing myself clearly. This question was supposed to be a suggestion to improve the Field Guide by adding this question as an article.

@BerkeleyTrue what is that label for?

the label just means It'll be dealt with but no one is assigned to it yet

When blocked, it means there are no actions to be taken on an issue to progress it along. Until you clarified, it seemed like you where asking the question, "What do I need to get started", which @ckent20 and @ahstro answered. If you hadn't responded in some amount of time this issue would have been closed.
Now, with clarification, we know there is an action to be taken, mainly to discuss with the community if a new field guide is needed.

Closing as stale, you can add a page on the wiki for this if it doesn't exist yet


Add small dataset for Postgres and Kafka

In order to showcase what info we are able to extract, we can setup a a demo postgres and kafka dump.
This task should take care of:

Create a demo postgres dataset.
Create a demo kafka dataset.
Depending on the size, setup Git LFS (although better to avoid it as much as possible).
Add section to README so people can access it easily.

We can create the datasets from the dev testing network as are small enough and it includes most of the usecases.

I'm going to close this issue as is not necessary at all (you can easily generate a dataset with DEV network)


ERROR: libsmartredis failed to build and install

I am trying to execute the following commands to install SmartRedis for my project
cd $ROOTDIR
git clone https://github.com/CrayLabs/SmartRedis.git
cd SmartRedis
git checkout v0.2.0
make lib -j
export SMARTREDIS_DIR=$(pwd)

But I am getting the following error when make lib -j is executed:
make[3]: *** [CMakeFiles/smartredisPy.dir/build.make:76: CMakeFiles/smartredisPy.dir/src/python/src/pyclient.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: *** [CMakeFiles/smartredisPy.dir/build.make:328: CMakeFiles/smartredisPy.dir/src/python/bindings/bind.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:156: CMakeFiles/smartredisPy.dir/all] Error 2
make[1]: *** [Makefile:136: all] Error 2
ERROR: libsmartredis failed to build and install
make: *** [Makefile:29: lib] Error 1

If more details are necessary please let me know.

Hi, thanks for reaching out to us!
I would like to start by asking if you are tied to version 0.2 of SmartRedis -- we are currently on version 0.4.2 and expect to release v0.5 shortly. There have been extensive updates to both the libary and the build system since we released v0.2 and both are substantially more robust now.
With that said, if you need to stay with v0.2, I can start with an observation that where build parallelism is supported, we already internally use the -j flag. So rerunning your build without it may make a difference. Also, I notice that you aren't building the dependencies before attempting to build the library. While this step is no longer needed in the current library, with v0.2, you'll have to make deps before you can make lib.
Please let us know if this helps. If you are still encountering trouble, please let me know the following:

Platform/OS
Compiler and version
Longer tail of the output

Thanks,
Bill

@billschereriii Thank you for your response.
Yes I am tied to version 0.2 because it is specified by the developers of the end software package I am working with.
I did a make clean followed by make deps and make lib resulting in the same error.

Platform/OS: x86_64 GNU/Linux (openSUSE Leap 15.4)
Compiler: gcc-7, g++-7, gfortran-7
Please find attached the documents with detailed output (make_deps.txt and make_lib.txt).
make_deps.txt
make_lib.txt

Looking forward to your reply. Thanks !!

Hi,
Thank you for the additional information.
I took a close look at your output files. I have a guess as to what might be causing the problem, but am not 100% sure. We do not regularly test with GCC v7 -- usually we require v8 through v11 (inclusive). I am guessing that the problem you are seeing is (partially) a reflection of a change in the way namespaces were handled between GCC version 7 and GCC version 8.
Specifically, if you look at include/pyclient.h, you'll see that the py namespace is declared inside the SmartRedis namespace, but if you look at include/pydataset.h, you'll see that it is declared outside the SmartRedis namespace. This inconsistency is actually still in the code as of this writing (though I will shortly put in a PR to unify the treatment).
I see three options for you at this point:

Speak with the developers of the end software package you am working with to ask them to upgrade to a more current version of SmartRedis. We would be happy to speak with them as well if you are able/willing to share contact information.
Upgrade GCC to version 8, 9, or 10. This will match the version we tested with at the time.
Failing all else, you could try manually updating pyclient.h in your download to reverse lines 44 and 46 (thereby moving the py namespace declaration outside of the SmartRedis namespace). If you do try this, please let us know if it resolved your problem or if there were additional problems that came about afterwards. We have not tested with GCC v7, so cannot promise that this will help.

I would like to hear how things go for you; please send an update when you get a chance!
Cheers,
Bill

Hi,
Thank you for your suggestions. I want to go ahead with the 2nd option which is to use version 11 of the gcc compiler.
Currently, I have access to versions 7 and 11. While the version 7 is symbolically linked as follows:
/usr/bin/gcc-7 -> gcc
/usr/bin/g++-7 -> g++
/usr/bin/gfortran-7 -> gfortran

Now I want to know how to tell make deps and make lib commands to use gcc-11, g++-11 and gfortran-11 compilers instead of the older version? The upgraded versions are as:
/usr/bin/gcc-11
/usr/bin/g++-11
/usr/bin/gfortran-11

Should I change set(CMAKE_C_COMPILER "/usr/bin/cc") to set(CMAKE_C_COMPILER "/usr/bin/gcc-11") in the CMakeCCompiler.cmake file?
(or)
Can I set the runtime variable export CC=/usr/bin/gcc-11 ?
Please let me know about the modifications that are required to use the upgraded versions instead of the older versions.
Thanks.

I would go with the export CC=/usr/bin/gcc-11 approach.
There shouldn't be any other modifications needed to use the more modern compilers.
Cheers,
Bill


[Fix][UI Next][V1.0.0-Alpha]: Fix the dag map display uncorrect afte…

Purpose of the pull request
This pull request fixs the dag map display uncorrect after changing the pre tasks.
Releated issues
close #8877

Codecov Report

Merging #8895 (c4b0d30) into dev (db64dca) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

:exclamation: Current head c4b0d30 differs from pull request most recent head 68ddaff. Consider uploading reports for the commit 68ddaff to get more accurate results

@@             Coverage Diff              @@
##                dev    #8895      +/-   ##
============================================
- Coverage     45.30%   45.27%   -0.03%
+ Complexity     4082     4081       -1
============================================
  Files           702      702
  Lines         27177    27186       +9
  Branches       2907     2908       +1
============================================
- Hits          12312    12309       -3
- Misses        13704    13715      +11
- Partials       1161     1162       +1

Impacted Files
Coverage Δ

...nscheduler/api/controller/ResourcesController.java
57.89% <0.00%> (-1.04%)
:arrow_down:

...heduler/api/service/impl/ResourcesServiceImpl.java
53.66% <0.00%> (-0.68%)
:arrow_down:

...e/dolphinscheduler/remote/NettyRemotingClient.java
50.70% <0.00%> (-2.82%)
:arrow_down:

...dolphinscheduler/remote/future/ResponseFuture.java
83.60% <0.00%> (+1.63%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db64dca...68ddaff. Read the comment docs.


How to loop over definition list items (dl)

How would I loop over definition list items?

Binding to rv-each to dl-element would repeat the dl element itself and binding to dt would only loop over dt.
Guess, I would have to write a custom binder for that or is there a better solution?

Is it unsemantic to use divs to wrap each pair?

Yep, that's what the specification says. I've turned it into a list now, so I can loop over the li-element.

Having the same problem. Have anyone figured out a good way?


⚠ Bofrost.es has degraded performance

In 47b5504, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1430 ms

Resolved: Bofrost.es performance has improved in 79bedf0 after 27 minutes.


Graph: Relationship Color vs Relationship Width

Hi,
I have one graph where I would like to colour the relationships and modify their width with variables.
If I only do one of the two, it works fine.
If I add a variable for "Relationship Color Property" and another for "Relationship Width Property", the width property seems to be ignored.
Here is the graph without the colour variable:

And here with the colour (no other changes made):

Hi @JoridK ,
Just tried this out locally on a virtual graph:
WITH
    apoc.create.vNode(["Person"], {name:"bob"}) as a,
    apoc.create.vNode(["Person"], {name:"alice"}) as b
RETURN a,b,apoc.create.vRelationship(a,"LIKES",{amount: 10, class: "blue"},b)

Then with the following settings:

It seems to work for me, perhaps it has something to do with the data in Neo4j? Could you share your query and the data in the database?

It works for me now as well, I think I configured the variable for the width wrongly. It was at 0 for that connection, even though it should not have been. What is a bit odd is that it was larger when there was no color (width was still at zero...) but I cannot replicate that either. So I think this issue is obsolete unless I run into it again. Thanks for looking into it :)

Hi @JoridK , great! just cleaning up the backlog so closing this issue.
Have a good week,
Niels


zopen query --upgradeable is throwing error

zopen query --upgradeable is throwing error when you try to use it with or without package option
$ zopen query --upgradeable
Package                   Installed                 Available                 Latest Tag
jq: error: Could not open file : EDC5129I No such file or directory.
***ERROR: Unable to retrieve remote information

$ zopen query --upgradeable vim
Package                   Installed                 Available                 Latest Tag
jq: error: Could not open file : EDC5129I No such file or directory.
***ERROR: Unable to retrieve remote information

$ zopen query vim --upgradeable
Package                   Installed                 Available                 Latest Tag
jq: error: Could not open file : EDC5129I No such file or directory.
***ERROR: Unable to retrieve remote information

This is still broken but it fails in a different way. Now it just doesn't list anything, e.g.
$ zopen query --upgradeable
Package                     Installed                   Available                   Latest Tag
$ zopen query --upgradeable vim
Package                     Installed                   Available                   Latest Tag

@DevonianTeuchter fyi

I have improved the help and removed the broken verbs. Here is the new help. Will open defects for other items:
bin/zopen-query --help
zopen-query -  a utility for z/OS Open Tools to query packages and repos.
Usage: zopen-query [OPTION] [VERB] [PACKAGE]
  VERB is the action to take, which is one of
    --list, --remote-search, --installed
  PACKAGE is a package, specified for --remote-search
Verbs:
  --list           list all available z/OS Open Tools.
  --remote-search  regex match package against available z/OS Open Tools
  -i, --installed  list installed z/OS Open Tools.
Options:
  -v               run in verbose mode.
  -d, --details    include full details for listings.
  --no-header,     suppress the header for the output.
  --no-version,    suppress version information, return package names.
  --help           print this help
  --version        print version


FiberPhotometryTable being added as acquisition

Is acquisition the best place to store FiberPhotometryTable?
https://github.com/catalystneuro/ndx-fiber-photometry/blob/2e80a7ac040ddca2873cce03aceaabd7c816a5f6/src/pynwb/tests/integration/test_fiber_photometry.py#L216
@alessandratrapani @pauladkisson @weiglszonja

Is acquisition the best place to store FiberPhotometryTable?

Yeah, maybe add it to LabMetaData? Like so:
fiber_photometry = NWBGroupSpec(
        neurodata_type_def='FiberPhotometry',
        neurodata_type_inc='LabMetaData',
        name='fiber_photometry',
        doc='all Fiber Photometry metadata',
        groups=[
            NWBGroupSpec(
                name='fiber_photometry_table',
                neurodata_type_inc='FiberPhotometryTable',
                ...
            ),
        ]
    )

Or maybe under a fiber photometry processing module?

Is acquisition the best place to store FiberPhotometryTable?

Yeah, maybe add it to LabMetaData? Like so:
fiber_photometry = NWBGroupSpec(
        neurodata_type_def='FiberPhotometry',
        neurodata_type_inc='LabMetaData',
        name='fiber_photometry',
        doc='all Fiber Photometry metadata',
        groups=[
            NWBGroupSpec(
                name='fiber_photometry_table',
                neurodata_type_inc='FiberPhotometryTable',
                ...
            ),
        ]
    )

Just to propagate, during the meeting today we discussed this should be the way to go.


Try to speed up large training jobs

Motivation:

Training a CoralNet classifier on 10,000s of images has a train-time measured in hours. This is true for both the MLP / EfficientNet case and the LR / VGG16 case, although MLP / EfficientNet is the faster case (Edit: actually it's not faster; see next post).

As we get larger sources on CoralNet, or as we start to explore having super-classifiers trained on multiple sources' data, the long train times will become more of a concern. Mostly for usability, but eventually also server load at a certain point.

We could encourage more folks to switch from VGG16 to EfficientNet if we improve train time + allow more frequent training in the latter case. (Since VGG16 is deprecated and seems more of a pain to work with, I assume trying to speed that up is not worth it.)

David said an idea from 2020 was to switch from logistic regression to an MLP when the size is large (don't know exact threshold). I'm not sure if this is the actual logic in train_utils.py et al. at the moment; it looks to me like it just uses MLP if EfficientNet, and LR if VGG16? But I could be wrong. Either way, it looks to be using MLP for large training jobs (assuming EfficientNet), which is the concern this issue is highlighting.
Other notes from David:

Training the MLP just uses sklearn MLPClassifier on CPU. We didn't explore training time for this, and except for learning rate, the parameters are vanilla.

Speeding up MLP training would involve:

Some benchmark problems.
Benchmarking existing code
Benchmarking existing code on some different instances (Maybe we can get a reasonable speedup by just using faster machines).
Stopping criteria for training.
Pytorch or fastai on GPU -- sklearn is pure CPU. A two layer MLP is not very big generally, but our input is 1280 and the hidden size is 100, which means > 100k weights for first layer (second layer is smaller) So, not tiny.

Don't need to deliver a research paper on this. Just some systematic experimentation to improve speed while maintaining the same level accuracy and having confidence that it's delivering.

Our slowest training times for EfficientNet sources:

s372: 35k train images * 10 pts, 7h45m
s3058: 20k train images * 5 pts, 4h56m
s3363: 14k train images * 50 pts, 4h23m
s3371: 14k train images * 10 pts, 4h15m
s3411: 14k train images * 50 pts, 3h57m

Our slowest (completed) training times for VGG16 sources, with dates included to show that some have been using CoralNet 1.0 specs:

s1052: 40k train images * 30 pts, 15h08m, 2020/10
s295: 57k train images * 10 pts, 14h54m, 2019/11
s1374: 16k train images * 30 pts, 9h35m, 2021/03
s1656: 16k train images * 50 pts, 9h14m, 2021/09
s371: 25k train images * 20 pts, 7h39m, 2020/04

So, VGG16 sources' trainings took somewhere between a factor of 1 to 2 longer than EfficientNet sources' trainings, given the same number of train images. Overall, both can get slow.
Additionally, as of https://github.com/beijbom/coralnet/pull/409 (merged into production September 2021) we were able to see trainings that started but never completed (as Classifiers stuck in PENDING status). When I checked last month (December 2022) there were stuck trainings in 3 sources: 1052, 1374, and 1656, all VGG16. I don't know if this means those stuck trainings were going even longer than 15 hours, but it seems to be a problem correlated with long training times.

For the record, you should multiply number of  images times pts since
that’s the number of examples used to train the classifier.  The biggest
one has 1.2M examples.
D
On Fri, Jan 20, 2023 at 4:43 PM StephenChan @.***>
wrote:

Our slowest training times for EfficientNet sources:

s372: 35k train images * 10 pts, 7h45m
s3058: 20k train images * 5 pts, 4h56m
s3363: 14k train images * 50 pts, 4h23m
s3371: 14k train images * 10 pts, 4h15m
s3411: 14k train images * 50 pts, 3h57m

Our slowest (completed) training times for VGG16 sources, with dates
included to show that some have been using CoralNet 1.0 specs:

s1052: 40k train images * 30 pts, 15h08m, 2020/10
s295: 57k train images * 10 pts, 14h54m, 2019/11
s1374: 16k train images * 30 pts, 9h35m, 2021/03
s1656: 16k train images * 50 pts, 9h14m, 2021/09
s371: 25k train images * 20 pts, 7h39m, 2020/04

So, VGG16 sources' trainings took somewhere between a factor of 1 to 2
longer than EfficientNet sources' trainings, given the same number of train
images. Overall, both can get slow.
Additionally, as of beijbom/coralnet#409
https://urldefense.com/v3/__https://github.com/beijbom/coralnet/pull/409__;!!Mih3wA!DWVkWTa4cfGhFud3Qt6k9ENzYbLupWYWbxiv7Ov-x0XuoWusXeNvbMlPijKXvnYV94o0JB7prOjzISWlDAuFBwW1$
(merged into production September 2021) we were able to see trainings that
started but never completed (as Classifiers stuck in PENDING status). When
I checked last month (December 2022) there were stuck trainings in 3
sources: 1052, 1374, and 1656, all VGG16. I don't know if this means those
stuck trainings were going even longer than 15 hours, but it seems to be a
problem correlated with long training times.
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/beijbom/pyspacer/issues/53*issuecomment-1399104362__;Iw!!Mih3wA!DWVkWTa4cfGhFud3Qt6k9ENzYbLupWYWbxiv7Ov-x0XuoWusXeNvbMlPijKXvnYV94o0JB7prOjzISWlDO8iw2jj$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABKA5AKSK5TCWRBDWKN7QUTWTMWKPANCNFSM6AAAAAAUA3E3KQ__;!!Mih3wA!DWVkWTa4cfGhFud3Qt6k9ENzYbLupWYWbxiv7Ov-x0XuoWusXeNvbMlPijKXvnYV94o0JB7prOjzISWlDDfs_T6g$
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Sent from Gmail Mobile

OK, edited to include that. Though note that the point count is just the source default, and may not necessarily be applied to all images in the source.

Ok. Good point. It’s really the total number of verified points. I was
assuming that all points in all images were verified, but the number of
images times points per image are proxies.
On Fri, Jan 20, 2023 at 9:21 PM StephenChan @.***>
wrote:

OK, edited to include that. Though note that the point count is just the
source default, and may not necessarily be applied to all images in the
source.
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/beijbom/pyspacer/issues/53*issuecomment-1399182776__;Iw!!Mih3wA!BnwAs-Iv47CJrhVaUNI0FZdwejpxi2btQ9k9P5P8RJjbWDXAkeWwGtCSRykKLWS9MAJLWgsB23lbQj--o5rKClce$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABKA5ALJWYIJSTVCQJ4W4F3WTNW6ZANCNFSM6AAAAAAUA3E3KQ__;!!Mih3wA!BnwAs-Iv47CJrhVaUNI0FZdwejpxi2btQ9k9P5P8RJjbWDXAkeWwGtCSRykKLWS9MAJLWgsB23lbQj--o4k6wfij$
.
You are receiving this because you commented.Message ID:
@.***>

--
Sent from Gmail Mobile

Those train-images counts only include images which have all points verified (confirmed). So that part has no ambiguity at least.
I was more pointing out that the source's default number of points may have changed during the source's lifetime, or maybe image-annotations were imported which didn't adhere to that default.

As of the past day, I can confirm that there isn't anything about the training code that makes large jobs impossible to complete (at least, large by coralnet standards). They just need to be allowed to run long enough, up to ~50 hours in the cases of sources 295 and 1656.
A few optimization ideas I have at the moment:

Cache feature vectors to local filesystem for quicker access in later training epochs. Though this needs enough filesystem space; 50k feature vectors = roughly 50 GB. Potentially relevant link here on specifying storage volumes for Batch jobs (we may want different volume sizes for different jobs).
Try to reuse S3 connections as long as possible instead of using one connection per feature vector download
Have the ability to reduce epoch count on the fly, based on amount of accuracy improvement on the reference set (is this valid or a sketchy optimization?)

 support for large training jobs

Once PR https://github.com/coralnet/pyspacer/pull/80 or equivalent is merged, think we'll have appreciable enough performance improvements to close this issue

And now that PR is merged.


[KED-1110] "src/getting_started/nodes/example.py" file not getting generated in hello world tutorial

Description
src/getting_started/nodes/example.py file not getting generated as per example pipeline tutorial
https://kedro.readthedocs.io/en/latest/02_getting_started/04_hello_world.html
kedro new --config config.yml
config.yml
output_dir: .
project_name: Getting Started
repo_name: getting-started
python_package: getting_started
include_example: true
Your Environment
Include as many relevant details about the environment in which you experienced the bug:

Kedro version used (pip show kedro or kedro -V):
Name: kedro
Version: 0.15.2
Summary: Kedro helps you build production-ready data and analytics pipelines
Home-page: https://github.com/quantumblacklabs/kedro
Author: QuantumBlack Labs
Author-email: None
License: Apache Software License (Apache 2.0)
Location: /home/ajit/miniconda3/envs/data/lib/python3.7/site-packages
Requires: requests, cookiecutter, click, memory-profiler, toposort, anyconfig, python-json-logger, xlsxwriter, PyYAML, s3fs, pip-tools, pyarrow, pandas, SQLAlchemy, tables, xlrd
Required-by:

Python version used (python -V):
Python 3.7.3

Operating system and version:
Ubuntu 18.04

Addressed in https://github.com/quantumblacklabs/kedro/commit/52632c5b4c28fc80464378c757e15abf82a23db9


Refactor: Use Rust 2018 edition

Refactored so that it compiles with edition = "2018"
Additionally changed assert!(x==y) to assert_eq!(x,y)
Added a ToDo regarding printing of strings to console
Tested on Windows if it can rusty-demo

CI test https://github.com/jschwe/rusty-hermit/runs/693909327 (ubuntu) also passed.

Closing this since commit e952096e9c748ce446bdaa5e3391489cc968cc60 also addressed this


Add global_yaml example

how would you know if it was working?

@nicks right now kubectl get pods or kubectl get deployments since currently the global YAML manifest is not represented in the HUD in any way. Making some stories for that now.
(I was hoping to get it for free by being able to treat a YAML manifest like a regular manifest for the most part)

i'm a little bit nervous about adding things to servantes that other people won't see immediately or remember to check. if you remember the original spec, this was something that we explicitly decided not to do.

Fair enough. I'll hold off on merging this until everything is in.


mirakc --version didn't show GIT_SHA

docker run --rm --entrypoint=mirakc mirakc/mirakc --version shows like this:
mirakc <version> VERGEN_IDEMPOTENT_OUTPUT

this is incorrect.
env!("VERGEN_GIT_SHA") seems not to work properly.

mirakc was built using docker build, but .git in the project root was not transferred because .dockerignore contains **/.git.
.git in the project root should be transferred.

20c325ff72aae47da3680432ccd9eeea55aec07f fixed this issue: https://github.com/mirakc/mirakc/actions/runs/10858583335/job/30136755659#step:10:142


Categorize tags into different types

Maybe categorize into types such as:

default type
module type
CCA type
...

Have to rush to implement this by Thursday because v1.4 is the alpha, but no need to polish it yet

I am not sure whether we should include this feature into our final product or not.

Resolved by #167


fix valid component IDs reference URL

Jira: KBC-XXX
Before asking for review make sure that:
Checklist

[x] New client method(s) has tests
[x] Apiary file is updated

Release

[x] I gave the PR a proper label:

major (BC break)
minor (new feature)
patch (backwards compatible fix)
no release (just test changes)

Pri review som objavil, ze v dokumentacii ten link neexistuje tak som tam dal aspon ten component-list


Don't test macOS in Travis

This is slowing down the entire Spyder organization because it's taking one hour or more for Travis to assign us the slots we need.
Besides, testing on macOS is almost equivalent to do it on Linux (at least for this simple library).

@goanpeca, this is driving me crazy. We need to stop testing in macOS with Travis (I know it's not a good practice, but we really need to be pragmatic here).

this is driving me crazy

Relax!


The Installation guidelines are missing for MS VS.

I have dont have rust environment. The setup required that I have Microsoft Visual Studio, and then ensure I have the C++ Build tools before proceeding. This step was missing in the instructions.

Indeed, this is something that I overlooked. The requirements for setting up rustup in a Windows environment are slightly different, due to Rust depending on the MSVC compiler when installing in Windows.
I will make a subsection to differentiate between Windows and Linux environments. We would need to wait for a Mac user to let us know how different the setup for rustup is...

I believe Mac users get stuck with the Terminal "app" which runs in bash shell. Basically the terminal is UNIX, and Linux inherits UNIX Compatibility - so should be same steps. :) I had a Mac before.


add ability to ignore unknown flags

It will be nice if we can add ability to ignore unknown flags. This can be useful when writing wrappers on other commands where exposing all downstream flags is un-necessary.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

requesting review from @eparis

pinging @eparis again

Pinging @eparis, please let me know if once a day ping is too much

ping @spf13 @eparis

ping @eparis
please continue to ignore if you are busy, i just want to make sure i don't forget about it.

How should this related to the *OnError defines?
https://github.com/spf13/pflag/pull/160/files#diff-1fdeef6cfb1d74886a385d13094d6cbcL117
I'm specifically wondering how Missing is today an error, which means it would be handled by the *OnError defines, but after this patch we have multiple error handle things...
Can you think of a good way to rationalize this into a single error handling logic?

I see what you are saying.
looking at this: https://github.com/spf13/pflag/blob/master/flag.go#L1087
i think with ContinueOnError, it will stop parsing flags as soon as it encounter an unknown flag, but let user decide what do to with error
with IgnoreUnknownFlags, it will ignore error encountered during parse while continuing to parse the other known flags.

Do you think IgnoreUnknownFlags is a better API? I think it makes sense to me.
I'm not arguing that the "ErrorHandling" stuff is a good idea, it's just the thing we have. And I don't want to make a second semi-related thing if we can help it....

if we add another ErrorHandling type as 'IgnoreUnknownFlags', what should be the behavior incase of any other error.
e.g. lets say user selected ErrorHandling to be of type IgnoreUnknownFlags, and flag parsing encountered a different error, what should we do with that error (panic, exit or continue?)

sorry if felt that way, but i am asking for suggestions here :)
this library is a master piece, and i want to keep it that way (even after my ugly contributions)

so kindly let me know what you think we should do here, and I will try to update the PR to reflect that.

pinging @eparis

thanks for the review @eparis
I have added the test case as suggested.

thanks a lot for merging eparis

I think we should probably include IgnoredFlags *flag.FlagSet field in command, that can be set when ignoredflags is on
Passing by default to run func may create prob again for user on what is child commands vs ignored flags

@rajatjindal but would that require the user to explicitly set IgnoredFlags in advance?

Yeah, i think so, otherwise the flag parsing will fail on it.
I think []string make sense

sorry i think i misread your comment. The user will just have to set a bool flag to ignore unknown flags, but ignored flags will be read and added to IgnoredFlags while parsing.

Making sure my thought was understood. I was saying that if we ignore a flag and arg (because the dev decided to set IgnoreUnknown) we should add those things to the args here: https://github.com/spf13/cobra/blob/master/command.go#L103  Today, I think, those args are just 'non-flag' args which were not consumed by cobra. If you set 'IgnoreUnknown' you program will start getting those 'ignored' bits along with the non-flag args. Now it would be your programs problem to parse and handle those 'args'.

I'm totally with @eparis, just unsure to be saying something nonsense, but passing everything back to the user in args makes sense to me.

I think thats fine too. args it is then :)

@rajatjindal , the last few messages in this discussion seem to indicate that any non-consumed flags should be left in the args array passed to Run.  That is not the behavior that I am observing.  I am currently using
	github.com/spf13/pflag v1.0.2 // indirect

I am writing a wrapper around kubectl logs:
var logsCmd = &cobra.Command{
	Use:   "logs [app name]",
	Short: "Fetch the logs of a container",
	Args:  cobra.MinimumNArgs(1),
	FParseErrWhitelist: cobra.FParseErrWhitelist{
		UnknownFlags: true,
	},
	RunE: func(cmd *cobra.Command, args []string) error {
		result, err := util.Runner{
			BufferStdout: true,
		}.Run("kubectl", []string{
			"get",
			"pod",
			"-l",
			fmt.Sprintf("app=%s", args[0]),
			"-o",
			"jsonpath={.items[0].metadata.name}",
		}...)
		if err != nil {
			return err
		}

		log.Debugf("running logs args [%v]", args)
		_, err = util.Runner{
			PrintStdout: true,
			PrintStderr: true,
		}.Run("kubectl",
			append([]string{
				"logs",
				result.Stdout.String(),
			}, args[1:]...)...)
		return err
	},
}

But when i use go run . logs foodb -f, the -f is not getting passed in args:
...
DEBU[0001] running logs args [[foodb]]
...

It appears your test code verifies that all the expected flags are parsed, and unexpected flags ignored, but it doesn't seem to be verifying that the unexpected args are present in the remaining arg list.  Also, this ignore feature never made its way into the cobra documentation, it was a bit of a pita to figure this out.  Might be a good idea to add it there.

Hi
Sorry for the pain it caused
I am in middle of relocating right now, i can try to fix this in about 2 weeks or if you want to open a PR that will be great

Totally agree, an ignored flag should have been kept in the argument list after flag parsing, because it is ignored. This is an important missing feature of this library which prevent developing a command wrapper (as said by @victorcoder) that has its own "global" arguments.
However this features is implemented as a kind of error to be ignored and not flags to be ignored. So I don't think it is worthwhile breaking by changing this behavior. There should be a real "IgnoreUnknownFlags" feature.

Is there any update on the ignored flag being passed back to the remaining arg list?

FWIW, using -- works as people expect with no additional options, including this configuration.
var execCmd = &cobra.Command{
	Use:   "exec -- [subcommand]",
	Short: "Spawn a subprocess",
	Args: cobra.MinimumNArgs(1),
	Run:  execRun,
}

func execRun(_ *cobra.Command, args []string) {
	execCmd := args[0]
	execArgs := args[1:]
	# ...
}

I agree it's not the ideal UX, but it gets the job done:
$ mycli exec -- ls -alh

Is there any update on the ignored flag being passed back to the remaining arg list?

I can't seem to find a way to access the ignore flags - was this ever addressed?

Is there any update on the ignored flag being passed back to the remaining arg list?

I can't seem to find a way to access the ignore flags - was this ever addressed?

cmd = &cobra.Command{
		......
		FParseErrWhitelist: cobra.FParseErrWhitelist{
			UnknownFlags: true,
		},
	}

Is there any update on the ignored flag being passed back to the remaining arg list?

I can't seem to find a way to access the ignore flags - was this ever addressed?

cmd = &cobra.Command{
		......
		FParseErrWhitelist: cobra.FParseErrWhitelist{
			UnknownFlags: true,
		},
	}

This allows the flag to be passed but how would I access the value?

https://github.com/spf13/pflag/issues/186 is an issue requesting that ignored flags be retained


Jukebox

Description

Test Instructions

Tested on emulator by spamming buttons
Ticket Links

Readiness Checklist

[ ] I have run make format to format the changes
[ ] I have compiled the firmware and the changes have no warnings
[ ] I have compiled the emulator and the changes have no warnings
[ ] I have run make cppcheck and checked that cppcheck_result.txt has no warnings for the changes
[ ] I have added doxygen comments to any code used by more than one Swadge mode. This includes /*! \file comments with Design Philosophy, Usage, and Example sections for new headers.
[ ] I have run make docs and checked that doxy_warnings.txt has no warnings for the new code

Not gonna merge this until it provably has no memory issues


snabbvmx ignores vlan settings in .conf file

Trying to get vlan support in snabbvmx working, based on the test case example https://github.com/Igalia/snabb/tree/lwaftr/src/program/snabbvmx/tests/end-to-end/data/vlan
Using the following config files:
snabbvmx-lwaftr-xe1.cfg:
return {
  lwaftr = "snabbvmx-lwaftr-xe1.conf",
  settings = {
    ingress_drop_action = "flush",
  },
  ipv6_interface = {
    ipv6_address = "",
    cache_refresh_interval = 1,
  },
  ipv4_interface = {
    ipv4_address = "<IP_ADDRESS>",
    cache_refresh_interval = 1,
  },
}

snabbvmx-lwaftr-xe1.conf:
hairpinning = false,
binding_table = binding_table_60k.txt.s,
aftr_ipv6_ip = 2001:db8::1,
aftr_ipv4_ip = <IP_ADDRESS>,
aftr_mac_inet_side = 02:b0:53:89:04:00,
inet_mac = 02:02:02:02:02:02,
aftr_mac_b4_side = 02:b0:53:89:04:00,
next_hop6_mac = 02:02:02:02:02:02,
ipv4_mtu = 9000,
ipv6_mtu = 9000,
vlan_tagging = true,
v4_vlan_tag = 100,
v6_vlan_tag = 100,

Running lwaftr standalone works as expected:
$ cat test_snabb_lwaftr_xe1.sh
#!/bin/bash
echo "launching snabb lwaftr on-a-stick on PCI 0000:04:00.0"
sudo usr/local/bin/snabb lwaftr run --conf snabbvmx-lwaftr-xe1.conf --on-a-stick 0000:04:00.0
mwiget@xeon:~/vmxlwaftr/tests/vlan-test$ ./test_snabb_lwaftr_xe1.sh
launching snabb lwaftr on-a-stick on PCI 0000:04:00.0
loading compiled binding table from ./binding_table_60k.txt.s.o
compiled binding table ./binding_table_60k.txt.s.o is up to date.

To generate vlan traffic, I've used this:
snabb packetblaster lwaftr --src_mac 02:02:02:02:02:02 --dst_mac 02:b0:53:89:04:00 --b4 2001:db8::40,<IP_ADDRESS>,1024 --aftr 2001:db8:ffff::100 --count 60000 --rate 3.1 --pci 0000:04:00.1 --vlan 100
packetblaster lwaftr: Sending 60000 clients at 3.100 MPPS to 0000:04:00.1

IPv6: 2001:db8::40 > 2001:db8:ffff::100: <IP_ADDRESS>:1024 > <IP_ADDRESS>:12345
      source IPv6 and source IPv4/Port adjusted per client
IPv6 packet sizes: 104,104,104,104,104,104,104,634,634,634,1540

IPv4: <IP_ADDRESS>:12345 > <IP_ADDRESS>:1024
      destination IPv4 and Port adjusted per client
IPv4 packet sizes: 64,64,64,64,64,64,64,594,594,594,1500
vlan set to 100
v6+v4: 0.000+1.766 = 1.766490 MPPS, 0.000+5.330 = 5.329517 Gbps, lost 41.569%
v6+v4: 0.000+3.099 = 3.098588 MPPS, 0.000+9.348 = 9.347615 Gbps, lost 0.008%
v6+v4: 0.000+3.099 = 3.098671 MPPS, 0.000+9.348 = 9.347739 Gbps, lost 0.007%

But when running snabbvmx, no traffic is processed:
$ cat test_snabb_lwaftr_xe1.sh
#!/bin/bash
echo "launching snabb lwaftr on-a-stick on PCI 0000:04:00.0"
sudo usr/local/bin/snabb lwaftr run --conf snabbvmx-lwaftr-xe1.conf --on-a-stick 0000:04:00.0

$ ./test_snabb_snabbvmx_xe1.sh
launching snabb snabbvmx on-a-stick on PCI 0000:04:00.0
Ring buffer size set to 2048
loading compiled binding table from ./binding_table_60k.txt.s.o
compiled binding table ./binding_table_60k.txt.s.o is up to date.
Hairpinning: no
nic_xe1 ether 02:b0:53:89:04:00
IPv6 fragmentation and reassembly: no
IPv4 fragmentation and reassembly: no
lwAFTR service: enabled
Running without VM (no vHostUser sock_path set)
nh_fwd6: cache_refresh_interval set to 0 seconds
nh_fwd6: static next_hop_mac 02:02:02:02:02:02
loading compiled binding table from ./binding_table_60k.txt.s.o
compiled binding table ./binding_table_60k.txt.s.o is up to date.
nh_fwd4: cache_refresh_interval set to 0 seconds
nh_fwd4: static next_hop_mac 02:02:02:02:02:02
Ingress drop monitor: flush (threshold: 100000 packets; wait: 20 seconds; interval: 1.00 seconds)

packetblaster shows no packets received:
$ ./run-gen5b.sh
ens6f1 at 0000:04:00.1 has , running at 3.1 gbps
sudo taskset -c 4 ../snabb/src/snabb packetblaster lwaftr --src_mac 02:02:02:02:02:02 --dst_mac 02:b0:53:89:04:00 --b4 2001:db8::40,<IP_ADDRESS>,1024 --aftr 2001:db8:ffff::100 --count 60000 --rate 3.1 --pci 0000:04:00.1 --vlan 100
packetblaster lwaftr: Sending 60000 clients at 3.100 MPPS to 0000:04:00.1

IPv6: 2001:db8::40 > 2001:db8:ffff::100: <IP_ADDRESS>:1024 > <IP_ADDRESS>:12345
      source IPv6 and source IPv4/Port adjusted per client
IPv6 packet sizes: 104,104,104,104,104,104,104,634,634,634,1540

IPv4: <IP_ADDRESS>:12345 > <IP_ADDRESS>:1024
      destination IPv4 and Port adjusted per client
IPv4 packet sizes: 64,64,64,64,64,64,64,594,594,594,1500
vlan set to 100
v6+v4: 0.000+0.000 = 0.000000 MPPS, 0.000+0.000 = 0.000000 Gbps, lost nan%
v6+v4: 0.000+0.000 = 0.000000 MPPS, 0.000+0.000 = 0.000000 Gbps, lost nan%

Sending untagged traffic to snabbvmx (with vlan set), processes traffic:
sudo taskset -c 4 ../snabb/src/snabb packetblaster lwaftr --src_mac 02:02:02:02:02:02 --dst_mac 02:b0:53:89:04:00 --b4 2001:db8::40,<IP_ADDRESS>,1024 --aftr 2001:db8:ffff::100 --count 60000 --rate 3.1 --pci 0000:04:00.1
packetblaster lwaftr: Sending 60000 clients at 3.100 MPPS to 0000:04:00.1

IPv6: 2001:db8::40 > 2001:db8:ffff::100: <IP_ADDRESS>:1024 > <IP_ADDRESS>:12345
      source IPv6 and source IPv4/Port adjusted per client
IPv6 packet sizes: 104,104,104,104,104,104,104,634,634,634,1540

IPv4: <IP_ADDRESS>:12345 > <IP_ADDRESS>:1024
      destination IPv4 and Port adjusted per client
IPv4 packet sizes: 64,64,64,64,64,64,64,594,594,594,1500
v6+v4: 0.834+0.833 = 1.666858 MPPS, 2.613+2.345 = 4.957152 Gbps, lost 91.443%
v6+v4: 1.550+1.550 = 3.100000 MPPS, 4.874+4.378 = 9.252662 Gbps, lost 0.000%
v6+v4: 1.550+1.550 = 3.100013 MPPS, 4.874+4.378 = 9.252686 Gbps, lost 0.000%

So it seems there is some configuration missing in the .cfg file to make this work, but that contradicts the vlan end-to-end testcase
https://github.com/Igalia/snabb/tree/lwaftr/src/program/snabbvmx/tests/end-to-end/data/vlan
Binding table used:
binding_table_60k.txt

I think I figured out what's going on.
VLAN tag values from lwAFTR's conf are only considered in SnabbVMX if V4 and V6 VLAN values are different. Cases:

lwAFTR in on-a-stick mode. It works, VLAN values are always considered.
SnabbVMX with Snabbvmx's conf VLAN not set but set in lwAFTR's conf with same V4 and V6 values (ignored).
SnabbVMX test case. It works because V4 and V6 VLAN values are different.


CoverCard's Link throws ChunkLoadError in Firefox

Hello 👋 Thanks for this great resource! I'll certainly recommend it to my coworkers, nice work 😊
That said, I've found a Firefox-only (Firefox Nightly, 73.0a1 (2019-12-19)) bug where CoverCards's CoverLoader triggers a ChunkLoadError[1] for some patterns[2]. I didn't manage to understand the exact reason, let alone come up with a fix, so I thought I'd leave a bug report.

1
The above error occurred in the <InnerLoadable> component:
    in InnerLoadable (created by Context.Consumer)
    in Unknown (created by ForwardRef)
    in ForwardRef (created by CoverCard)
    in a (created by LinkAnchor)
    in LinkAnchor (created by Context.Consumer)
    in Link (created by CoverCard)
    in CoverCard (created by HomePage)
    in div (created by HomePage)
    in section (created by HomePage)
    in div (created by HomePage)
    in div (created by Layout)
    in Layout (created by HomePage)
    in HomePage (created by App)
    in Route (created by App)
    in Switch (created by App)
    in Router (created by BrowserRouter)
    in BrowserRouter (created by App)
    in App

React will try to recreate this component tree from scratch using the error boundary you provided, CoverCard.

To get the following log, I needed to add a componentDidCatch in CoverCard:
ChunkLoadError: "Loading chunk patterns-accordion-Cover~patterns-card-Cover~patterns-centering-Cover~patterns-cookie-banner-Cover~p~e8feece5 failed.
(error: http://localhost:1234/patterns-accordion-Cover~patterns-card-Cover~patterns-centering-Cover~patterns-cookie-banner-Cover~p~e8feece5.c20cad6c14f01cd89b31.js)"
        requireEnsure
        webpackAsyncContext
        importAsync
        requireAsync
        loadAsync
        componentDidMount
        commitLifeCycles
        commitLayoutEffects
        callCallback
        invokeGuardedCallbackDev
        invokeGuardedCallback
        commitRootImpl
        unstable_runWithPriority
        runWithPriority$2
        commitRoot
        finishSyncRender
        performSyncWorkOnRoot
        scheduleUpdateOnFiber
        updateContainer
        legacyRenderSubtreeIntoContainer
        unbatchedUpdates
        legacyRenderSubtreeIntoContainer
        render
        b40e
        __webpack_require__
        2
        __webpack_require__
        checkDeferredModules
        webpackJsonpCallback
        <anonymous>

2
Holy grail, Same height columns, Sidebar, Split screen, Sticky footer, Sticky header, Drawer, Dropdown, Menu, Wizard, Input addon, Slider, Accordion, Card, Centering, Cookie banner, Drop area, Drop cap, Feature list, Media object, Pricing table, Questions and answers, Separator, Timeline, Video background, Modal

Outdated.


dyld: malformed mach-o image: symbol table underruns __LINKEDIT

Brew install on Mac Sierra 10.12.6 of the latest helm results in the following error when trying to use helm:
$ brew install kubernetes-helm
$ helm version
dyld: malformed mach-o image: symbol table underruns __LINKEDIT
Abort trap: 6

Some initial digging suggests this may be related to this issue:
https://github.com/haskell/cabal/issues/5220
But I'm not positive if that's the culprit here.
Output of helm version: 2.10.0
An earlier version (2.8.2) does not have this issue:
$ BREWURL=https://raw.githubusercontent.com/Homebrew/homebrew-core brew install {BREWURL}/993e941fb56b035ae3ab87c2b9081ec5ab46eeca/Formula/kubernetes-helm.rb
$ helm version
Client: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

Hi @CharlesBergmeier
I am not familiar with the brew issues and the core team does not produce the brew cask. @bacongobbler do you know the appropriate person to route this to?

Closing due to inactivity. Please open a new ticket against Helm 3 if this issue persists. Thanks!


Updated features

Coverage remained the same at 100.0% when pulling 5e084af11002ebde9e258b9979bb703e14776def on feature/Searchallfiles into 5f7e6c4639260e2f6f2c77e734111db8028b9ebc on master.

Coverage remained the same at 100.0% when pulling 775303f027afdf78d11e26461bb5519dc6f6528e on feature/Searchallfiles into 5f7e6c4639260e2f6f2c77e734111db8028b9ebc on master.


Add support for html summaries for Atom feeds

Refactors out the code that was used to process the Atom content and uses it for both the content and the summary. Also adds a type param to the summary function.

Coverage decreased (-0.4%) to 93.27% when pulling 8122fb7e67fce46d9de4fef5b1337754cd1e959e on pR0Ps:atom-summary-html into edd988f8a6d0ba693a5ea494dae7311fdc7f1019 on lkiesow:master.

Hi @pR0Ps, thanks for the contribution. It might take me a few days to take a look at this, but I sure will and will – assuming everything works as expected – include this in a new release soon after.


Select and introduce a JS/CSS package manager

Select a package manager for our JS/CSS dependencies. Rails will eventually be moving towards Yarn and Webpack in version 5.1.
Other options include npm, bower, etc.
We should also include something like Gulp or Webpack as well for optimization of these assets.
Here is an article explaining how to replace the asset pipeline with Webpack: http://samuelmullen.com/articles/replacing-the-rails-asset-pipeline-with-webpack-and-yarn/
And another explaining how Rails will be using Yarn and Webpack in v5.1.
http://samuelmullen.com/articles/embracing-change-rails51-adopts-yarn-webpack-and-the-js-ecosystem/
There is a Webpack gem (referenced in the article above) that would help us integrate this into the Rails 4.2.x ecosystem. Things get complicated when trying to mix Webpack and the asset pipeline.

We can also just handle the Webpack/Gulp tasks during deployment (especially in production). If we choose to move all of the CSS/JS out of the pipeline. I guess that will depend in part on how much Bootstrap CSS we need to override.

Closing this out because there is no UAT needed for this.


Rename IncludeRaiReason to IncludeRAIReason

Per initialism conventions in Go, RAI should be all capital. Due to autogeneration, IncludeRaiReason isn't following this convention.

Great suggestion! As is the case for https://github.com/googleapis/go-genai/issues/8, we'll discuss this proposal and respond back.

Completed, thanks for raising this issue.


Use background session

Is is possible to use background session on iOS with requests ? If not is it easy to add in ?

@jcampbell05 it is fully possible with a plugin. React Native does not support it out of the box.
https://github.com/ocetnik/react-native-background-timer

What I mean is to use NSURLSession background requests.

Then you have to code out yourself:)
Why not use fetch in js and save it?

@stoffern in this case its for allowying the user to use the action buttons for a notification. We need background requests so iOS accepts it.
In future would be cool with this was transparently used when iOS is in background.

@facebook-github-bot stack-overflow

Hey @jcampbell05 and thanks for posting this! @Kureev tells me this issue looks like a question that would be best asked on StackOverflow. StackOverflow is amazing for Q&A: it has a reputation system, voting, the ability to mark a question as answered. Because of the reputation system it is likely the community will see and answer your question there. This also helps us use the GitHub bug tracker for bugs only. Will close this as this is really a question that should be asked on SO.


Feature Request: Turn Dobot on and Off

Would it be possible to somehow add a function that can turn the dobot on and turn it off?

As far as I can see in the API spec (http://www.dobot.it/wp-content/uploads/2018/03/dobot-api-en.pdf) I don't think you can do this.
Personally I have a sonoff s20 wifi enabled plug flashed with tasmota firmware which I then send a post request to in order to turn my Dobot Magician on and off over the network.


⚠ Spryr has degraded performance

In 4444480, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 3034 ms

Resolved: Spryr performance has improved in 4e99aa4 after 7 minutes.


Primary Button color exploration

This pull request addresses some readability issues with $brand-green's color, especially on 1x displays. It's been flattened slightly and text-shadows have been removed for consistency with the rest of the buttons.
.btn-primary is now based off the $brand-green variable.
Visual changes are shown on the right.
Primary button

Primary button disabled

Ps. Ignore the extraneous commits. Things were getting a little wild over here.

It's been a while since this has been opened. How do we feel about it? Should we close this one out? If not, what needs to be changed?

I think any change at this point is going to get a bunch of push back. I'd like to wait while we work with the creative folks on some of their brand and color guidelines work.


Fixed bug from using File.exists?

@tomprats Have you managed to get this to work with ruby 3.3.4? I keep running into errors, solving them, then running into more errors. (i18n, template_to_s, etc.)

I've got it running with ruby 3.3.2 I believe. I don't think I ran into any more errors with this library


Wemo insight switch not working

Home Assistant release (hass --version):
0.38.4
Python release (python3 --version):
3.4.5
Component/platform:
wemo
Description of problem:
Wemo insight switch not working on latest version.
Status on webUI is "unavailable"

Problem-relevant configuration.yaml entries and steps to reproduce:
wemo:
static:
- <IP_ADDRESS>
Traceback (if applicable):
log.txt
Additional info:
1, port 8989/tcp was opened on the firewall.
2, Wemo insight has static IP assignment <IP_ADDRESS>
3, No error message in log.
4, Also tried on latest dev: 0.39.0 dev0 with no luck

My wemo's didn't work also with the last update. I tried a very simple configuration.yaml (with discovery) and the Wemo start working again. After that I reused my normal config file and the wemo start working again!

Hi Frekel, thanks for the reply.
This morning I tried to modify the config and it works!
The only thing I did is added another wemo into the list, no idea how it works though.

@SonireolxXx, if everything is working close the ticket.


bettercap: 2.28 -> 2.29

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/bettercap/bettercap/releases.
meta.description for bettercap is: "A man in the middle tool"
meta.homepage for bettercap is: "https://www.bettercap.org/"
meta.changelog for bettercap is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 2.29 with grep in /nix/store/d9ydkv5d4j0dq4pfwmdx0v2j2cxb8j62-bettercap-2.29

directory tree listing: https://gist.github.com/0171f85d8b12a1d0afdecac9e68dea3d

du listing: https://gist.github.com/6cc73f204552796be17ab16354b293c9

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
bettercap

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/d9ydkv5d4j0dq4pfwmdx0v2j2cxb8j62-bettercap-2.29 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bettercap https://github.com/r-ryantm/nixpkgs/archive/42d0ac20a51dc7c6bc8c4759805b6debf2339a02.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/d9ydkv5d4j0dq4pfwmdx0v2j2cxb8j62-bettercap-2.29
ls -la /nix/store/d9ydkv5d4j0dq4pfwmdx0v2j2cxb8j62-bettercap-2.29/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

bettercap

Maintainer pings
cc @y0no @kalbasit for testing.


Make RequestContext's RxJava auto inject hook strictly

Currently RequestContext RxJava hook pushes context to stack without
checking current context. Because armeria doesn't predict the case that
unrelated RequestContexts push on the same thread nestedly, it may cause
some unexpected problems.
e.g.

Current zipkin related module doesn't expect nested context, if there
are two different contexts called sequentially, the outer span's scope
won't be closed.

Application may utilize RequestContext#onChild to propagate
attribute from parent to child, so the 2nd context will see unrelated
request's context.

fix test, and I think we don't expect push has null context as parameter right?


Add Caveman to variant calling pipeline

Simpler, containerized implementation of CaVEMan caller:
https://github.com/cancerit/cgpCaVEManWrapper
singularity pull docker://quay.io/wtsicgp/pcap-core:5.4.2

More modular C-based implementation of CaVEMan caller:
https://github.com/cancerit/CaVEMan/blob/dev/setup.sh

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6097605/ caveman paper


Fix iteration over Row elements

This fixes a bug that leads to unexpected results when iterating over a Row (or over a Table containing Rows). Succinctly, this demonstrates the problem:
>>> import awkward
>>> awkward.fromiter(awkward.Table([0, 1]))
<JaggedArray [[[0 1]] [[0 1]]] at 0x7faad228ada0>

The single column gets duplicated, and this produces an extra dimension (a table of scalars should be 2D, not 3D). Basically, every element of every Row gets replaced with the entire corresponding column. I added a test that fails on this behavior and a [self._index] in Row.__iter__() to fix it. Now I get:
>>> import awkward
>>> awkward.fromiter(awkward.Table([0, 1]))
<JaggedArray [[0] [1]] at 0x7feec2a752b0>

@masonproffitt Do you understand why
                with self.assertRaises(TypeError, msg='Scalar row element should not have a length'):
                    len(element)
                with self.assertRaises(TypeError, msg='Scalar row element should not be iterable'):
                    iter(element)

passes when Row objects have a length (1) and are iterable (for one step)? I just wanted some clarity on that before merging the pull request.

The test is checking whether the elements of the row have a length or are iterable, not the row itself. In fact, I am indeed iterating over the two-element rows here. I would expect a one-element row to behave in the same way (be iterable and have a length of one).
For a more complicated Table (with actual non-scalar elements in the rows), these assertions would not necessarily hold, but the whole point is that this test fails on the given array without this fix. In other words, it appears like the scalar number elements themselves are iterable and have a length (because the whole column replaces each number). An extra dimension was popping in out of nowhere, and the test makes sure this doesn't happen.

Oh, I see. Okay that clears it up. I'll merge this.


US888096: Use docker image version management plugin

Ticket: https://internal.almoctane.com/ui/entity-navigation?p=131002/6001&entityType=work_item&id=888096
I temporarily reverted this to get it released for 24.2.3, this PR is putting it back again.

A developer build has not yet been created for this branch. Click here to go ahead and create the build...

CI Build Link:
https://sou-jenkins2.swinfra.net/job/CAFapi/job/CAFapi~opensuse-opensearch2-image~US888096~CI


Add ToNumericDataFrameTransformer

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (dcfa7dc) 98.49% compared to head (e90bdcc) 99.52%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #157      +/-   ##
==========================================
+ Coverage   98.49%   99.52%   +1.03%
==========================================
  Files          64       65       +1
  Lines        1457     1479      +22
  Branches      141      144       +3
==========================================
+ Hits         1435     1472      +37
+ Misses         22        7      -15

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


changed summary stat cards

I've changed the summary stats from averages to medians and updated the format of the cards. But I am still working on the histogram, so it is not ready for merging yet.

I love it!! Looks amazing. Just a few small things, could we change would be the tab name to "Categorical Data" and "Numerical Data", since average indicates its the mean. Also it might be better to add axis labels for the bar charts?

I love it!! Looks amazing. Just a few small things, could we change would be the tab name to "Categorical Data" and "Numerical Data", since average indicates its the mean. Also it might be better to add axis labels for the bar charts?

Yes sounds great, I will change that!


About - day 2 identification of the definition of a node graph

Consider those use-cases, where the node graph created from a service template deployment is persisted. I am aware that not all orchestrators have such a model, but some do.
In these cases, for day 2 scenarios you need to be able to come from the node graph and find the configuration of template objects that the node graph was created from. Using includes etc. that may in fact be a combination of versions of CSARs that were available at the time.
As discussed in today's language ad hoc meeting it makes sense to look for existing standards of configuration management for good ways of identifying configurations. We may want to move template_name and tempate_version to an about section.
For the cases were services are deployed based on one CSAR alone, the identification of the only feasible identification is presently the filesystem path and name of the CSAR, which I find not satisfactory.
ETSI clearly faced this problem and for a VNF TOSCA descriptor SOL001 they ended up defining template identification as properties:

descriptor_id: A globally unique string Identifier of this VNFD information element - typically a UUID. The VNFD Identifier shall be used as the unique identifier of the VNF Package that contains this VNFD. Any modification of the content of the VNFD or the VNF Package shall result in a new VNFD Identifier.
descriptor_version: A string that identifies the version of the VNFD.
provider A string defining the provider of the VNF and of the VNFD.
product_name: A string name to identify the VNF Product. Invariant for the VNF Product lifetime.
software_version`: Software version of the VNF. This is changed when there is any change to the software that is included in the VNF Package

The only one of these that ETSI specifies to be unique is the descriptor_id - and so when we get to day2 modifications of ETSI VNF descriptors, they define that a TOSCA template can define a policy to indicate that a node graph deployed from one descriptor_id can be migrated "up" or "down" to another descriptor_id - like this example:
tosca_definitions_version: tosca_simple_yaml_1_3
imports:
..
node_types:
  MyCompany.SunshineDB.2_0.2_0:
    derived_from: tosca.nodes.nfv.VNF
    properties:
      descriptor_id:
..
      default: ebc68e34-0cfa-40ba-8b45-9caa31f9dcb5       ###### My ID ( I am V2)
..
  interfaces:
    MyCompanyChangeCurrentVnfPackage:
      type: MyCompany.interfaces.nfv.ChangeCurrentVnfPackage   #### TOSCA interface ~= hook
..
interface_types:
  MyCompany.interfaces.nfv.ChangeCurrentVnfPackage:
    derived_from: tosca.interfaces.nfv.ChangeCurrentVnfPackage
    operations:
      change_from_version_1:
        description: operation for change from version 1 to 2
..
      change_to_version_1:
        description: operation for change from version 2 to 1
...
topology_template:
..
  node_templates:
    sunshine_db:
      type: MyCompany.SunshineDB.2_0.2_0
..
      interfaces:
        MyCompanyChangeCurrentVnfPackage:
          operations:
            change_from_version_1:
              implementation: change-from-version-1.workbook.mistral.yaml #### REAL script (this is mistral, could be shell or python)
            change_to_version_1:
              implementation: change-to-version-1.workbook.mistral.yaml
  policies:
..
  - change_from_version_1:
    type: tosca.policies.nfv.VnfPackageChange
    properties:
      selector:
        source_descriptor_id: b1bb0ce7-ebca-4fa7-95ed-4840d70a1177  ## vnfdid of the V1 package
        destination_descriptor_id: ebc68e34-0cfa-40ba-8b45-9caa31f9dcb5 ## My id again (I am V2)
        source_flavour_id: simple
      modification_qualifier: up        ### This is what tells VNFM that this is an upgrade
      additional_modification_description: ..
...
      triggers:
        - change_from_version_1:
          event: tosca.interfaces.nfv.Vnflcm.change_current_package_notification
          action: tosca.interfaces.nfv.MyCompanyChangeCurrentVnfPackage.change_from_version_1
  - change_to_version_1:
    type: tosca.policies.nfv.VnfPackageChange
    properties:
      selector:
        source_descriptor_id: ebc68e34-0cfa-40ba-8b45-9caa31f9dcb5      ### Ids reversed here
        destination_descriptor_id: b1bb0ce7-ebca-4fa7-95ed-4840d70a1177
        source_flavour_id: default
      modification_qualifier: down
....

What this is intended to mean is, for example, that a node graph deployed from the template with descriptor_id: b1bb0ce7-ebca-4fa7-95ed-4840d70a1177 can be up-graded to the template with descriptor_id: ebc68e34-0cfa-40ba-8b45-9caa31f9dcb5. As a minimum this requires that we can identify a template by the value of the descriptor_id in one of the nodes in the node graph.
I hope we agree that this approach is not what we should recommend.
The problem I see is that our TOSCA 2.0 specification does not provide a good alternative to the above.
In my world, I would have liked that at least there should be a well-defined template_name and template_version that was guaranteed to be unique. The current TOSCA 2.0 specification says:

Some service templates are designed to be referenced and reused by other service templates and have a lifecycle of their own.  Therefore, in these cases, a template_version value SHOULD be included and used in conjunction with a unique template_name value to enable lifecycle management of the service template and its contents.

My point here that referencability is not only for refences between templates, but also between the templates and the node graph, extending the need further than originally envisaged.
What would happen if we wrote "MUST" instead of "SHOULD"? This would give the template a unique name independent of where the CSAR happened to be placed in the file-system when it was unpacked.
I am probably overlooking a lot of additional complexity - so I won't say what "the solution" should be - only that when we leave open a hole in the language, the users get creative and misuse other constructs.

Our current section <IP_ADDRESS>.6.2 Grammer says template_version: <version> where in other places we have <string>. The version is defined in <IP_ADDRESS>.1 Grammar as the "TOSCA version": <major_version>.<minor_version>[.<fix_version>[.<qualifier>[-<build_version>] ] ]
Down to fix_version this format has the benefit of defining a clear way that an orchestrator can compare two versions to determine which one is newer than the other. The qualifier is not comparable.
Was this the intention?
What could an orchestrator do with that information? It could be given an order to upgrade a service to its latest version. That could make sense if the latest version were a mandatory security patch.
Personally, I think it would be useful to enforce this format - making name and version mandatory, because what happens today - as was mentioned in the meeting - is that people concoct arbitrary encodings of versioning into their CSAR names, node names, properties, etc.
I suggest that any standard - whatever we decide - is better than wild-west naming.

I completely agree. TOSCA should support (semantic) versioning of profiles and service templates.


small improvements to duckdb docs

I am unable to load data from the chess.com API to DuckDB after following the instructions in the docs. I get the following error when I try to run it:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/dlt/destinations/sql_client.py", line 138, in _wrap
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dlt/destinations/duckdb/sql_client.py", line 35, in open_connection
    self._conn.execute(f"SET {k} = '{v}'")
duckdb.BinderException: Binder Error: Catalog "chess_data" does not exist!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ty/Documents/dlthub/test-duckdb/chess.py", line 111, in <module>
    info = p.run(
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 99, in _wrap
    step_info = f(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 130, in _wrap
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 477, in run
    self.sync_destination(destination, dataset_name)
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 73, in _wrap
    rv = f(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 519, in sync_destination
    remote_state = self._restore_state_from_destination()
  File "/usr/local/lib/python3.10/site-packages/dlt/pipeline/pipeline.py", line 954, in _restore_state_from_destination
    job_client.sql_client.open_connection()
  File "/usr/local/lib/python3.10/site-packages/dlt/destinations/sql_client.py", line 140, in _wrap
    raise DestinationConnectionError(type(self).__name__, self.dataset_name, str(ex), ex)
dlt.destinations.exceptions.DestinationConnectionError: Connection with DuckDbSqlClient to dataset name chess_data failed. Please check if you configured the credentials at all and provided the right credentials values. You can be also denied access or your internet connection may be down. The actual reason given is: Binder Error: Catalog "chess_data" does not exist!

ducklabs released duckdb 0.7 yesterday where the programming interface was changed freely. the situation above is normal but was silently handled. now the exception they raise is different. I'll switch the duckdb to next version and try to release. you can also do pip install duckdb==0.6.1 which will downgrade

I'm merging the PR. I will fix it starting from devel