Improvements to Searching - Partial/Wildcard searches

Hi,
The search functionality could use some improvements by adding support for either wildcard or partial searches.
A few examples:

When searching for IP Addresses, it'd be useful to be able to enter a partial IP. For instance "192.168.10" in the search box. It should ideally treat that search as a wildcard. Either as "%192.168.10%" or "192.168.10%".

Similar, but for prefixes. It currently kinda works for /24 boundaries. For instance i can type "192.168.10" in the Prefix search bar, and it will return 192.168.10x.x and any child prefixes. However if i type "192.168" it returns nothing. This seems like weird behavior.

For both of the above, you could certainly search by specifying the parent prefix. But you don't always know what it is from memory. You could find it by drilling down the tree view of prefixes, but that's more steps.

This was covered in #787. Wildcards are of limited use when searching for IPs, as they only work along the dotted-decimal boundaries (8, 16, and 24 bits). Instead, you can search by covering prefix. For instance, instead of searching for "192.168.10," enter "<IP_ADDRESS>/24" in the parent prefix field.
Not only is this approach more efficient at the database level (as compared to casting all prefixes/IPs to strings), it decouples the matching logic from the dotted-decimal format. For example, I can search within <IP_ADDRESS>/21, which cannot be expressed using a simple wildcard (you'd have to resort to using a regular expression).

Fair enough.
Would it be possible to allow searching by parent prefixes that don't necessarily exist then?
For instance, say there's <IP_ADDRESS>/16 parent prefix. It has an immediate child /28 prefix (<IP_ADDRESS>/28 for instance). The current search behavior for the "parent prefix" field is that it will only return results for "<IP_ADDRESS>/16" and "<IP_ADDRESS>/28". If i try "<IP_ADDRESS>/22" for instance, it returns no results.
Being able to search with arbitrary prefix masks is useful when you don't know the parent prefix, or if you are wanting to quickly test which allocated prefixes would get covered by an ACL.
From an efficiency point of view it should be a cheap query.

Actually, the current behavior is different to what i just described.
So it works when searching with bigger masks, but not smaller ones.
For example, say you have this:
<IP_ADDRESS>/16
|---<IP_ADDRESS>/23
|---<IP_ADDRESS>/23
If i search for "<IP_ADDRESS>/20", it'll return both <IP_ADDRESS>/23 and <IP_ADDRESS>/23. That seems correct.
If i search for <IP_ADDRESS>/24, it returns no results. I can see why - strictly speaking it doesn't have a parent prefix that is in that range. However i think that the better behavior would be to display any allocated IP addresses that can be matched by a <IP_ADDRESS>/24 mask.
Thoughts?

Searching with a parent prefix of <IP_ADDRESS>/24 will return all IP addresses matched by that prefix. It will not return the prefix <IP_ADDRESS>/23, because <IP_ADDRESS> is not covered by <IP_ADDRESS>/24.

But the children IP Addresses kind of are. I guess it just depends on how you look at it. It makes sense to me that "<IP_ADDRESS>/24" should return any allocated IP addresses that are in 10.10.3.[0-255], regardless of which prefix they belong to.
Perhaps it'd make sense for the default search field to accept that type of search? Then its like "show me IP addresses that match this search criteria", rather than "show me IP addresses who have a parent subnet that falls within this search criteria".

Ok, I think I misunderstood what you were asking. NetBox treats prefixes and IP addresses entirely separately; you're just focused on IP addresses. If I understand correctly, the issue is that searching for <IP_ADDRESS>/24 won't return an IP address <IP_ADDRESS>/23, because its mask is larger than the /24.
This behavior is native to PostgreSQL, but I do see the value in allowing users to search for IPs without regard to their assigned masks. We'd need to read each INET type as though it has a /32 or /128 mask. This can be accomplished either by using set_masklen() or by recasting the host() form back to INET. The later is probably preferable as we don't have to worry selecting the correct mask length for the address family.
For example, this SQL query would include the IP address <IP_ADDRESS>/23:
SELECT * FROM "ipam_ipaddress" WHERE CAST(HOST("ipam_ipaddress"."address") AS INET) <<= '<IP_ADDRESS>/24';

Thanks for that stretch, champion.


Add requiresSQLCommentHint to UuidType to fix schema update

Fix problem with schema update or migration diff.
More info in https://github.com/ramsey/uuid-doctrine/issues/64
Also, pointed in https://github.com/ramsey/uuid-doctrine/pull/100#discussion_r472946672

@ramsey what do you think? can we merge it?

also interested, I'm facing the same issue in several projects

Just released this as part of version 1.8.0. Enjoy!


TServerSocket.cpp: Ensure the server is really listening

Currently the C++ server does not ensure that listening_ is true in isOpen(). I think this is incorrect because in multi-threaded environments, the socket may not be open or listening yet. Also, listening_ should only be set to true when the socket is fully set up and listening.
This PR adds a minor change: it respects listening_ for isOpen(), and sets listening_ to true only at the end of spcket initialization. In heavy stress tests, we found this makes the server method isOpen() more reliable.
Thanks for consideration.

[ ] Did you create an Apache Jira ticket?  (not required for trivial changes)
[ ] If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
[x] Did you squash your changes to a single commit?  (not required, but preferred)
[x] Did you do your best to avoid breaking changes?  If one was needed, did you label the Jira ticket with "Breaking-Change"?
[x] If your change does not involve any code, include [skip ci] anywhere in the commit message to free up build resources.

LGTM — eventhough, I think, people should pick the TNonBlockingServer for multithreaded environments, as it has at least some synchronization and is in general more suitable for threadpools and the like.

I'm very curious about this! We did not try it yet, was it a simple drop-in replacement for you? It uses libevent, correct? Did you compare it with the new fbthrift c++ server?

Checked, the build errors in Travis are all unrelated. Ready for review and merge...

was it a simple drop-in replacement for you?

Not sure about this...

It uses libevent, correct?

Better to check the source, but I think not. It just expects to be used from multiple threads, and does appropriate locking.

Did you compare it with the new fbthrift c++ server?

No; what's that?

Did you compare it with the new fbthrift c++ server?

No; what's that?

Apparently Facebook has continued development on thrift in a separate project. I think it has less features and supports less languages, but it seems to have a cool new async c++ server. I've always wondered if this server could not be backported to Apache thrift, but that seems to involve complicated discussion of licenses. But for end-users, things may be simpler: possibly the server could just be used with Apache thrift if one would manage to take it out and build it against Apache thrift. No idea if this is possible, though :-)
See https://github.com/facebook/fbthrift/blob/master/thrift/doc/Cpp2.md for more details.


Node not joining cluster: kubelet failure- missing /etc/kubernetes/pki/ca.crt

What happened:
Deployed an EKS cluster with 3 worker nodes via cloudformation. 2 nodes joined cluster, 1 did not.  Restarting the instance had no effect.  Terminating the instance once and letting the ASG spin up a new one had no effect.  Terminating the subsequent instance resulted in a new instance that did register with the cluster.
What you expected to happen:
Expected all 3 original nodes to join cluster.
How to reproduce it (as minimally and precisely as possible):
Unfortunately as described this appears to be sporadic.  Our cloud formation template is pretty standard.  Excerpt:
  ###
  # This Stack Provisions the EKS Cluster
  ###
  EKSCluster:
    Type: AWS::CloudFormation::Stack
    Properties:
      Parameters:
        MasterStackName: !Ref MasterStackName
      TemplateURL: './EKS-Cluster.yml'
      TimeoutInMinutes: '15'

  ###
  # This Stack Provisions the EKS Cluster
  ###
  EKSNodes:
    Type: AWS::CloudFormation::Stack
    DependsOn: EKSCluster
    Properties:
      Parameters:
        MasterStackName: !Ref MasterStackName
        NodeImageId: !Ref NodeImageId
        NodeInstanceType: !Ref NodeInstanceType
        NodeGroupName: !Ref NodeGroupName
        EKSClusterName: !Sub '${EKSCluster.Outputs.EKSClusterName}'
        KeyName: !Ref KeyName
        SQSInterfaceKMSKeyArn: !Ref SQSInterfaceKMSKeyArn
        NodeAutoScalingGroupMinSize: !Ref NodeAutoScalingGroupMinSize
        NodeAutoScalingGroupMaxSize: !Ref NodeAutoScalingGroupMaxSize
        EKSClusterControlPlaneSecurityGroup: !Sub '${EKSCluster.Outputs.EKSClusterControlPlaneSecurityGroup}'
        R53SubDomain: !Ref R53SubDomain
        R53RootDomain: !Ref R53RootDomain
        VpcId: !Ref VpcId
        EnvType: !Ref EnvType
        UseKeystoreBucket: !Ref UseKeystoreBucket
        KeystoreAccount: !Ref KeystoreAccount
        ClassB: !Ref ClassB
      TemplateURL: './EKS-Nodes.yml'
      TimeoutInMinutes: '15'

Cluster looks like:
  Cluster:
    Type: "AWS::EKS::Cluster"
    Properties:
      Version: "1.13"
      RoleArn: !GetAtt ClusterRole.Arn
      ResourcesVpcConfig:
        SecurityGroupIds:
          - !Ref ClusterControlPlaneSecurityGroup
        SubnetIds:
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetAPrivate"
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetBPrivate"
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetCPrivate"
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetAPublic"
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetBPublic"
          - Fn::ImportValue:
              !Sub "${MasterStackName}-SubnetCPublic"

Node group and launch looks like:
  NodeGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      DesiredCapacity: !If [ IsProdEnv, 5, 3 ]
      LaunchConfigurationName: !Ref NodeLaunchConfig
      MinSize: !Ref NodeAutoScalingGroupMinSize
      MaxSize: !Ref NodeAutoScalingGroupMaxSize
      VPCZoneIdentifier:
        - Fn::ImportValue:
            !Sub "${MasterStackName}-SubnetAPrivate"
        - Fn::ImportValue:
            !Sub "${MasterStackName}-SubnetBPrivate"
        - Fn::ImportValue:
            !Sub "${MasterStackName}-SubnetCPrivate"
      Tags:
      - Key: Name
        Value: !Sub "${EKSClusterName}-${NodeGroupName}-Node"
        PropagateAtLaunch: 'true'
      - Key: !Sub 'kubernetes.io/cluster/${EKSClusterName}'
        Value: 'owned'
        PropagateAtLaunch: 'true'
      - Key: k8s.io/cluster-autoscaler/enabled
        Value: ''
        PropagateAtLaunch: 'true'
      - Key: !Sub 'k8s.io/cluster-autoscaler/${EKSClusterName}'
        Value: ''
        PropagateAtLaunch: 'true'
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MaxBatchSize: 1
        MinInstancesInService: !If [ IsProdEnv, 5, 3 ]
        PauseTime: PT5M

  NodeLaunchConfig:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
      AssociatePublicIpAddress: 'false'
      IamInstanceProfile: !Ref NodeInstanceProfile
      ImageId: !Ref NodeImageId
      InstanceType: !Ref NodeInstanceType
      KeyName: !Ref KeyName
      SecurityGroups:
        - !Ref NodeSecurityGroup
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeSize: !Ref NodeVolumeSize
            VolumeType: gp2
            DeleteOnTermination: true
      UserData:
        Fn::Base64:
          !Sub |
            Content-Type: multipart/mixed; boundary="==BOUNDARY=="
            MIME-Version: 1.0
            --==BOUNDARY==
            Content-Type: text/cloud-boothook; charset="us-ascii"

            # Set the proxy hostname and port
            PROXY="proxy.${R53SubDomain}.${R53RootDomain}:3128"

            # Create the docker systemd directory
            mkdir -p /etc/systemd/system/docker.service.d

            # Configure yum to use the proxy
            cat << EOF >> /etc/yum.conf
            proxy=http://$PROXY
            EOF

            # Set the proxy for future processes, and use as an include file
            cat << EOF >> /etc/environment
            http_proxy=http://$PROXY
            https_proxy=http://$PROXY
            HTTP_PROXY=http://$PROXY
            HTTPS_PROXY=http://$PROXY
            no_proxy=10.${ClassB}.0.0/16,localhost,<IP_ADDRESS>,<IP_ADDRESS>,.internal
            NO_PROXY=10.${ClassB}.0.0/16,localhost,<IP_ADDRESS>,<IP_ADDRESS>,.internal
            EOF

            # Configure docker with the proxy
            cat << EOF >> /etc/systemd/system/docker.service.d/proxy.conf
            [Service]
            EnvironmentFile=/etc/environment
            EOF

            # Configure the kubelet with the proxy
            cat << EOF >> /etc/systemd/system/kubelet.service.d/proxy.conf
            [Service]
            EnvironmentFile=/etc/environment
            EOF

            --==BOUNDARY==
            Content-Type: text/x-shellscript; charset="us-ascii"

            #!/bin/bash
            set -o xtrace

            # Set the proxy variables before running the bootstrap.sh

             script
            set -a
            source /etc/environment

            /etc/eks/bootstrap.sh ${EKSClusterName} ${BootstrapArguments}
            /opt/aws/bin/cfn-signal --exit-code $? \
                      --stack  ${AWS::StackName} \
                      --resource NodeGroup  \
                      --region ${AWS::Region}

            --==BOUNDARY==--

So far this has happened 1 out of 4 attempts to stand up clusters with more than 1 node.
Anything else we need to know?:
kubelet logs show:
kubelet[12504]: F0327 21:30:29.616103   12504 server.go:244] unable to load client CA file /etc/kubernetes/pki/ca.crt: open /etc/kubernetes/pki/ca.crt: no such file or directory)

May be a false flag, but I also notice that the problem node had a cloud-init process that failed to complete.  It was still running stuck in the final module hours after the node came up when I went in to debug.
I've also posted against a similar issue here: https://forums.aws.amazon.com/thread.jspa?messageID=937703&#937703
Environment:

AWS Region: us-east1
Instance Type(s): t2-medium
EKS Platform version (use aws eks describe-cluster --name <name> --query cluster.platformVersion): eks.8
Kubernetes version (use aws eks describe-cluster --name <name> --query cluster.version): 1.13
AMI Version: amazon-eks-node-1.13-v20190701 (ami-0f2e8e5663e16b436)
Kernel (e.g. uname -a): Linux 4.14.128-112.105.amzn2.x86_64
Release information (run cat /etc/eks/release on a node):

BASE_AMI_ID="ami-052b172b0a9552df4"
BUILD_TIME="Mon Jul  1 21:38:37 UTC 2019"
BUILD_KERNEL="4.14.123-111.109.amzn2.x86_64"
ARCH="x86_64"

Faced similar issue.
Can confirm that it fails when cloud-init fails.
journalctl -u cloud-final.service |cat

# ... a custom bootstrap script successfully finished
Apr 16 03:23:28 host-name-eks-1111 cloud-init[4607]: Ncat: Connection refused.
Apr 16 03:23:28 host-name-eks-1111 cloud-init[4607]: Apr 16 03:23:28 cloud-init[4607]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/runcmd [1]
Apr 16 03:23:28 host-name-eks-1111 cloud-init[4607]: Apr 16 03:23:28 cloud-init[4607]: cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
Apr 16 03:23:28 host-name-eks-1111 cloud-init[4607]: Apr 16 03:23:28 cloud-init[4607]: util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]:
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]: #############################################################
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]: -----BEGIN SSH HOST KEY FINGERPRINTS-----
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]: 256 SHA256:dmw4QLBRS...ccLadpNXQs no comment (ECDSA)
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]: 256 SHA256:dnt+0Uby3...JUZ844 no comment (ED25519)
Apr 16 03:23:28 host-name-eks-1111 ec2[8894]: 2048 SHA256:zhvc...zB16peU no comment (RSA)
Apr 16 03:23:28 host-name-eks-1111 cloud-init[4607]: Cloud-init v. 19.3-43.amzn2 finished at Fri, 16 Apr 2021 03:23:28 +0000. Datasource DataSourceEc2.  Up 129.42 seconds
Apr 16 03:23:28 host-name-eks-1111 systemd[1]: cloud-final.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Apr 16 03:23:28 host-name-eks-1111 systemd[1]: Failed to start Execute cloud user/final scripts.
Apr 16 03:23:28 host-name-eks-1111 systemd[1]: Unit cloud-final.service entered failed state.
Apr 16 03:23:28 host-name-eks-1111 systemd[1]: cloud-final.service failed.

I'm facing the same problem using terraform to deploy my EKS cluster. Because of that, new nodes can't be registered in the cluster, and I'm currently trying to understand why we are facing this problem just right now


[Dialog] Fix double scroll issue

[x] I have followed (at least) the PR section of the contributing guide.

My first PR. Hopefully I didn't screw anything up :)
Closes #15951

@williamsdyyz It's a great first pull request on Material-UI 👌🏻. Thank you for working on it!


Expected table, got string error when calling vim.lsp.buf.definition()

Hi,
I'm running neovim NVIM v0.6.0-dev+1551-g0e22a40b6, and just updated a all of my plugins with Packer.
I am also using gopls, built-in LSP, and nvim-lspfuzzy. Previously, I would call vim.lsp.buf.definition() via a keybinding when hovering over a keyword in go. Often, there is only a single definition location, so my buffer would be moved automatically to that location. Further, if there were multiple definitions, I would get the fuzzy complete pop-up and could choose my definition.
For some reason, after upgrading, when I make a vim.lsp.buf.definition() call, I am getting the following error:
Error executing vim.schedule lua callback: vim/shared.lua:199: Expected table, got string
I also get this same error when calling vim.lsp.buf.implementation() and vim.lsp.buf.references().
I know that this is a nvim-lspfuzzy related issue, since when I disable nvim-lspfuzzy, these same function calls work as expected (except I don't get the pretty fuzzy pop-up!)
I am wondering if I am missing some configuration item here, or if this is an actual bug.
Please let me know if you would like additional information for re-producability.

Hello!
See this issue: #20
Basically you're in a detached commit between 0.5 and the commit 23fe6dba1 from Neovim which has introduced breaking change to the LSP handlers.
The plugin works for Neovim 0.5 and nightly/master (and additionally for all commits after 23fe6dba1) so I suggest you update Neovim. If you can't, you should checkout the old-handlers using in your plugin manager. Tell me if one of these solutions works for you.


[Feature] Add a few merchant useful features

Feature description
When merchants use Alby web version on mobile devices to receive from customers (e.g: in a restaurant or small shop) it may be interesting to have an operating mode when the employee is able to generate an invoice, check it was paid, but it unable to see accumulated funds. In order to withdraw funds the merchant would be required to enter a PIN, password or 2FA.
Describe the solution
Add an operating mode that would allow usages by merchants by locking some information that can be seen on the dashboard and asking for PIN, password or 2FA when withdrawing funds.
Describe alternatives
No response
Additional context
No response
Are you working on this?
No

Hi, thanks for your feedback!
We have an experimental PoS app which can be shared with employees without them having access to payments or the wallet balance: https://pos.albylabs.com/
Please let us know if this would be something you'd like developed further (e.g. with fiat support and other features)

Hi @rolznz
Thanks for the reply.
Cool that you have it on the radar and is already working on it.
One useful thing I mentioned in another issue is to have the ability to enter the fiat currency amount and get it automatically converted to satoshis before generating the invoice.
I guess it may not be ready yet, but I was not able to see anything in the Profile settings and also on the shopping cart.
Also I didn't fully understand where do I get the wallet URL to enter there. I was able to connect using the Browser Extension but that may not be the case if one is using a tablet of a mobile phone.
Regarding the overall PoS concept it may have its specific usage, but what I mentioned above it a little simpler that doesn't require shopping cart and items added beforehand.
Finally a recent history of the last confirmed transactions (expired after a configured time) may be useful to validate last payments.

Hi rolznz
No, I said the contrary, that in many usages the shopping cart may be unnecessary. I was trying to simply it and focus on the main points I mentioned in the first message which is allow the employee to generate an invoice, validate it was paid, see recent historic  but not the accumulated one.
Regarding the wallet URL I don't get where I extract it from either my Browser Extension or getably.com in order to enter in the PoS version and hand over the mobile device to an employee. I don't use Nostr Wallet Connect. Is there any way to extract that URL from getalby.com or Browser Extension ?

Hi @ffrediani
Inside the PoS on the top left there's a hamburger menu. You can click the "Share with a co-worker" option there and they can scan the QR code.

Yes, that is after I have logged into the PoS with my Alby Browser extension in my Desktop, but I didn't get where I can find the wallet URL from either the getalby.com or Browser Extension without having to login in to the PoS in the Desktop.

@ffrediani you can simply copy the link and send it to your other devices or any employees via whatever channel you like (message, email, etc.) so that all devices can run the same PoS

Sorry Roland I still don't get it. Which link ? From where ?
Are you talking about the Alby Page URL or another ? If so which one ?

@ffrediani
Open https://pos.albylabs.com/ and it should remember your wallet. Then you can just copy the URL from your browser (or hamburger menu -> share with a coworker -> copy the URL) and send it to all your devices.

Ok, so at least one time I must login using my wallet, from a Desktop using the Browser Extention in order to get the URL to pass over to other devices ?

@ffrediani yes, but it's not using the browser extension - it's using Nostr Wallet Connect to connect to your Alby Account. NWC enables you to access your wallet from any device without the need for the Alby browser extension.

Isn't there any way to not depend or need the Nostr Wallet for that ?
I guess many merchants will not be interested in Nostr and only on Alby for receiving their payments so the doubt if to use something either from getalby.com or the Browser Extension in order to be able to use the PoS interface.

@ffrediani it's just a communication protocol that allows for permissioned access to a wallet on all devices. Merchants don't even need to know it's powered by Nostr.
Without this protocol it's very difficult to do the permissioned access you are looking for.

Fine, but the point is simplicity. Merchants not needing to deal with yet another application for receiving payments.
As for now if they have the getalby.com installed they can already receive payments in a very simple way. What I am asking is for a way to be able to generate such URL from inside getalby.com web interface, pass over to the PoS devices to be used and not having to deal with NWC at all.

@ffrediani I think this is pretty much how this app works, I've recorded a video on how this process looks like:

Login to getalby.com
Visit pos.albylabs.com
Allow this app to request invoices, etc
Share the URL with coworkers

Screencast from 2024-05-23 20:13:56.webm
NWC is just the underlying protocol used, users will not even need to know how exactly that works.

Hi @reneaaron
Thanks for the time to record the video. I see know how to extract the URL to share with the devices.

Back to the main point and my first message of this Feature Request I think one the main points to add to the PoS interface is the ability to enter the amount in the local currency (as also in getalby.com interface).
Another thing that could be useful in certain situations is the employee or person operating the PoS to be able to check the last few transactions, in the case something happens and it is not possible to confirm it straight after it was paid (due to intermittent connection problems for example).
To make it useful for its propose a maximum time the person can see the transaction history could be configured by the manager beforehand in the getalby interface.

@ffrediani thanks for your suggestions! the "maximum time" would be difficult, but at least in the PoS app itself we could show only recent transactions. I doubt many employees would fork the code just to try to retrieve the full transaction list (we probably will not be able to implement such detailed permission handling)

Thanks for the reply @rolznz
Perhaps then a fixed maximum time (15 - 30 minutes) helps on situations where is not possible to confirm payment immediately for whatever reason, doesn't need to call the manager to validate and does not delay client departure.


GitHub: Invite member

Please invite these two Consortium members to the Catena - X NG organization.
GitHub User:
@jonbckr
@BenediktMuellerIML
Vouching person:
@evgenyvinokurov (Process Owner Standardization within Catena - X)

Hi @evgenyvinokurov,
could you verify the new users are part of Catena-X ?
Thanks in advance.

Hi @FaGru3n,
yes we are. Both of us work in the Traceability Use Case. Jon is from T-Systems and I'm from Fraunhofer IML.
Here are our Confluence Profiles as 'proof'
<EMAIL_ADDRESS><EMAIL_ADDRESS>

@BenediktMuellerIML thanks for mentioning your pages, process is a bit problem that was can not verify every new member since moved to public GitHub (basically could be anyone),
So for proof that you have a confluence page is unfortunalty not enough if the vouching person tells us. 😉
But because @evgenyvinokurov raised the issue we could be assume, he allready knows who should be invited and also a verification for the correct UserID's.

Perfect, thank you! 😊
From: Fabian Grün @.>
Sent: Dienstag, 1. August 2023 08:49
To: eclipse-tractusx/sig-infra @.>
Cc: Becker, Jon @.>; Mention @.>
Subject: Re: [eclipse-tractusx/sig-infra] GitHub: Invite member (Issue #205)
@BenediktMuellerIMLhttps://github.com/BenediktMuellerIML thanks for mentioning your pages, process is a bit problem that was can not verify every new member since moved to public GitHub (basically could be anyone),
So for proof that you have a confluence page is unfortunalty not enough if the vouching person tells us. 😉
But because @evgenyvinokurovhttps://github.com/evgenyvinokurov raised the issue we could be assume, he allready knows who should be invited and also a verification for the correct UserID's.
—
Reply to this email directly, view it on GitHubhttps://github.com/eclipse-tractusx/sig-infra/issues/205#issuecomment-1659672373, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A3ZNS2LVWQQSRDNPZTCQIADXTCREPANCNFSM6AAAAAA262RR64.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Perfect, thank you!  From: Fabian Grün @.> Sent: Dienstag, 1. August 2023 08:49 To: eclipse-tractusx/sig-infra @.> Cc: Becker, Jon @.>; Mention @.> Subject: Re: [eclipse-tractusx/sig-infra] GitHub: Invite member (Issue #205) @BenediktMuellerIMLhttps://github.com/BenediktMuellerIML thanks for mentioning your pages, process is a bit problem that was can not verify every new member since moved to public GitHub (basically could be anyone), So for proof that you have a confluence page is unfortunalty not enough if the vouching person tells us.  But because @evgenyvinokurovhttps://github.com/evgenyvinokurov raised the issue we could be assume, he allready knows who should be invited and also a verification for the correct UserID's. — Reply to this email directly, view it on GitHub<#205 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A3ZNS2LVWQQSRDNPZTCQIADXTCREPANCNFSM6AAAAAA262RR64. You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>
@jonbckr please keep in mind public crawler love company mail adresses, maybe you want to check out the GitHub Mobile App. Than you could answer directly in a proper format for issues and PR's 😉

@evgenyvinokurov invitation was sent, please add our new members to the desired gh-team for standardisation.

@evgenyvinokurov invitation was sent, please add our new members to the desired gh-team for standardisation.
Please verfiy and close the issue afterwards. Thanks in advance.

The two gentlemen are part of the Traceability Use Cases within the consortium.

would close the issue new members are allready in the GitHub Organisation
Cheers


Update The Handbook.md

I don’t think this is a meaningful change. Would be open to a PR spelling out “object oriented” instead of using the acronym though.

I don’t think this is a meaningful change. Would be open to a PR spelling out “object oriented” instead of using the acronym though.

I change that because of this:

page title and link title is not same,I think they should be same.

Oh, I see. Which title is more recent according to git blame?

Oh, I see. Which title is more recent according to git blame?

I can't distinguish which is recent, becaus they are same pull request and same time.
But according to article's content, I think it should be "TypeScript for Java/C# Programmers".
As for another, you can determine it.(I'm not good at English😅)


Bug fixes and enhancements

Bug fixes:

Translations of selections did not work (at all).
Accepting a translation with blank translation text named the address with "a".
Changing the encoding in the translation window failed with script errors.
Address failed to be named if the name already exists. (now avoids name collisions)

Enhancements:

Opens the translation window even if chardet fails to detect the encoding, allowing users to manually identify the correct encoding.
Added the following encodings: cp720, cp737, cp856, cp858, cp874, cp875, cp1006, koi8_u, utf_8_sig, palmos
Added the ability to overwrite an old translation with a new one.
Remembers the last successfully used encoding as a hint when chardet detection fails.

Tested with IDA Pro 6.6.

I like the changes- this does change the base requirements to Python 2.7 and IDA 6.3, which I don't think is onerous at this point (given that we're up to 6.7 now). I'll make the changes to the README and merge this change. Thanks!


Eliminate need to call CBase_Foo::pup in Foo::pup

Original issue: https://charm.cs.illinois.edu/redmine/issues/26

https://wiki.engr.illinois.edu/display/ppl/Incantationectomy
Have a look at branch phil/base_pup for work in progress. There's currently a merge conflict with mainline as a result of commit b201616fdf2fac821312ade8079f567831fd8458.

Original date: 2013-03-18 08:07:16

Bringing this old branch up to date. Encountered two challenges: templated chare code, and chare code in non-global namespaces. The revised version is on phil/base_pup_revised. The template thing seems to be dealt with, though it does require some finer-grain module definition at the moment.
The namespace issue is that the recursive pup template is in the global namespace, but its specialization is currently happening the namespace of the type it will operate on. It instead needs to be lifted out to the global namespace, with the relevant type names fully scoped. Right now, chare types defined within namespace blocks don't even know their enclosing block's name, so that needs to be pushed down the stack. Once that's done, scoping should be easy, though moving the declaration out to the global scope might be a bit trickier. Something like a scope-recursive genGlobalScopeCode(scopePrefix, decls, defs) routine might be necessary.

Original date: 2013-03-18 20:03:04

Namespace thing is dealt with, but the template issue is a bit tricker than I'd thought. Needs a bit more thought put into getting the solution to match the right principles.

Original date: 2013-03-20 02:57:59

Fully tested implementation of CBase::pup and __sdag_pup automation on branch phil/base_pup_revised. Checked that both the no-op and call removed cases work, with the built in test suites, ChaNGa, LeanMD, and randomAccess. Checked that the template-fu involved works at least as far back as gcc 4.4.
Will have to test on non-gcc compilers next. Some testing of ParFUM and threadCollide  in ck-libs/collide would be good, too. The implementation in AMPI and TCharm warrants particularly careful review.
Things that may still want updates are

various array maps in cklocation.{ci,C}
most load balancer strategies
many classes in {conv,ck}-com
ck-libs/{amr, ckmulticast, mblock, pose}
ck-libs/collide/threadCollide.* and the related examples in collide and mpi-charmcollide

The update would consist of declaring inheritance in the .ci file, inheriting from the generated CBase class in the class definition, and removing the extraneous base class pup calls. These should all be fine without update, but it's preferable that we not leave archaic examples lying around.

Original date: 2013-03-20 03:15:42

Clang 3.0 finds some trouble with constructor definitions of the CBase class in NDMeshChunkStreamer. When I comment that out, the remainder of the tests and examples (besides examples/charm++/hello/stl_array, which is known-broken on clang - cf #121) run successfully.
This might augur in favor of adding a pseudo-internal chare attribute [nocbase] to tell charmxi not to generate potentially-broken CBase classes that won't be used. The same would then apply to AMPI, TCharm, and some of the deeper internal entities. There might also be room for C++11 forwarding constructors to ease this, potentially.

Original date: 2013-03-20 06:21:52

All of the tests and examples build with GNU and XL (dec2011) compilers on Surveyor. Not trying to run tests there at the moment.

Original date: 2013-03-20 06:47:36

Builds of gemini_gni-crayxe with icc 13 compile and link all of the tests and examples as well.

Original date: 2013-03-20 21:29:10

Builds with GCC 4.2, Cray, and PGI compilers come back clean as well.

Original date: 2013-03-20 22:04:13

Clean builds for Mac gcc and clang (aside from documented unrelated clang issues).

Original date: 2013-03-21 18:06:44

Clean builds and links on Windows with Dexterity.

Original date: 2013-03-22 01:15:56

Ugh, this isn't quite ready yet.
The problem case is where we have the logical inheritance [internal group stuff] -> BaseLB -> CentralLB (and more things that inherit from CentralLB). Right now, the inheritance as written in *LB.h* is CBase_BaseLB -> BaseLB -> CentralLB -> [many other LB strategies]. What this means is that the call to obj->virtual_pup(p) will reach the implementation in CBase_BaseLB::virtual_pup`, which recursively pups up the inheritance hierarchy starting from BaseLB. Specifically, CentralLB and anything downstream of it will be missed. Absent the direct inheritance from BaseLB (using CBase_CentralLB instead), CBase_CentralLB::virtual_pup would start from CentralLB::pup and work its way up instead.
What this means is that old-style code that inherits multiple layers not through its appropriate CBase_Foo class will have broken PUP implementations, unless it's conscious of this and overrides virtual_pup explicitly.
We can implement a runtime check for this pathological structure pretty easily, but actually having it fixed as an API change that could break existing code. Outside our own codebase, I don't know how prevalent affected code is.

Original date: 2013-08-12 22:30:03

In order to smooth the transition path for the LB code, I want to merge the phil/inherit-constructors branch, which requires support for variadic templates. Discussions on whether the attendant effects of dropping support for older compilers (GCC < 4.3) are acceptable will have to follow.

Original date: 2013-08-20 14:39:41

The relevant systems this impacts are Ubuntu 8.04 installations, with GCC 4.2.4, ICC 11.1 and below, and Bluegene P installations with GCC 4.1.2. The consensus among those polled seemed to be that this would be fine. Will confirm more broadly, and then proceed.
To get ICC versions 12 and up working, one must specify a flag -gcc-version=440 (or a greater number). I guess we would be adding a version check and configuration flag to cc-icc.sh as well, then. Our own system with ICC (Order) has version 10, but we should be able to get an upgrade or do ICC tests elsewhere. Hopper only offers sufficiently-new versions. Taub has 11 and 13, though 11 is still default.
PGI apparently needs at least version 13 (12.9 fails, regardless of options chosen, while 13.3 works). It needs the option --gnu_version=40400 (or some greater number) or --c++11. Hopper, Titan, and Blue Waters all have multiple version-13 releases installed, though Titan still defaults to 12.10.
MS Visual C++ is fine at least from VS 2013 onwards. 2012 is unclear because of an unstated mapping between release year numbering an compiler version numbering.
From the documentation, Clang is fine with variadic templates from 2.9 onwards. I've compiled with 3.1 successfully.

Original date: 2013-08-30 01:33:42

Updated load balancers in the Charm++ tree to match the new semantics, and avoid the problem.
Was able to run ChaNGa's make test, but that doesn't say very much. Most of its load balancers have the same issue, and will need similar corrections. Updated MultistepLB and asked for testing from the cosmology subgroup.

Original date: 2014-09-08 20:09:54

Confirmed with Jim that we don't need to be too concerned about old compilers on NAMD's behalf.

Original date: 2015-01-21 05:04:35

Emailed core, pointing to the branch phil/cpp11 that has the necessary changes.

Original date: 2015-02-07 18:13:13

There's a hold-up to integrating this over NAMD compatibility. The new code requires that instantiated chare/message templates be fully defined where the old code didn't care, while NAMD's ComputeMoaMgr unconditionally instantiates MSA classes without including their definition. I've submitted a patch for NAMD, but older versions may lose backwards compatibility since they don't include this patch. I'll suggest that the patch also be included in updated source tarballs of NAMD 2.9 and 2.10 - 2.8 and older don't include the relevant code, and so are unaffected.


Nativescript with Angular 2: proper i18n approach?

From @andy-tai on September 21, 2016 7:35
Angular 2 has its own i18n functionality:
http://www.savethecode.com/angular2-i18n-native-support/
Will NativeScript adapt Angular's i18n for its own use?  Or the nativescript-i18n plugin is to be officially supported part of NS going forward?
Copied from original issue: NativeScript/NativeScript#2776

It looks like translations are compiled to TypeScript files, which should be perfectly usable in NativeScript apps. I haven't tried it yet, but my guess is that it's just a matter of tweaking the tools, and publishing an article in the docs.

We use https://github.com/NathanWalker/angular2-seed-advanced as a base for our project, with this we get ng2-translate support for both web, desktop and nativescript in the same codebase.
Translations are super easy, it's just an JSON-file with KEY => translated string.

While that seems working well, the choice of the Angular team matters greatly--they made a choice, and if you don't go with it, that will mean greater cost in keeping up to date and so on.

Some update on this guys ?, I feel the same as @andy-tai, but currently using ng2-translate.

Any news on this? I second @andy-tai on this one. ng2-translate is good as far as it goes, but I prefer the Angular team's solution. And it would be nice to see NativeScript stay close to the Angular team's decisions.


Better api

haven't started on readme yet...
design (this version untested with real buoy):

include/buoy_examples/controller.hpp (if we keep this idea, probably don't want it to be in the "examples" package):
-- templated base class exposing all interfaces (services/subscribers) to the buoy
-- utilizes CRTP static_cast to user-derived controller to use user callback for data subscribers
include/buoy_examples/torque_controller.hpp
-- declare user-derived controller with CRTP to base class
-- declare use of control policy member
-- declare user-defined power_callback and set_params
src/torque_controller.cpp
-- main, controller configuration, and definition of callback
include/buoy_examples/torque_control_policy.hpp
-- define control policy (conversion from motor rpm->winding current)
-- define set_params to load params for policy

python gain_scheduler_node.py & damping_selector.py have not been tested yet

I changed the base branch for this PR from main to the branch on #2 , so this PR only shows the diff from the other PR.

include/buoy_examples/controller.hpp (if we keep this idea, probably don't want it to be in the "examples" package):

One thing to consider is whether this base class is required to control the buoy, or recommended but not required, or just one example for how it could be done. Depending on the answer, we may want to create a new "controller base" package.
I could also see that header living within the buoy_msgs package itself too. A parallel would be how the sensor_msgs package provides not only the messages, but also convenient C++ APIs for working with those messages.

@andermi , let me know if you'd like help fixing conflicts and linters in this PR. Just making sure we don't duplicate work 😄

@chapulina , how do you get the linter to ignore using namespace std::chrono_literals; ?

colcon test passes:
Label Time Summary:
copyright     =   0.67 sec*proc (1 test)
cppcheck      =   0.21 sec*proc (1 test)
cpplint       =   0.34 sec*proc (1 test)
flake8        =   0.26 sec*proc (1 test)
lint_cmake    =   0.18 sec*proc (1 test)
linter        =   2.53 sec*proc (8 tests)
pep257        =   0.21 sec*proc (1 test)
uncrustify    =   0.21 sec*proc (1 test)
xmllint       =   0.46 sec*proc (1 test)


Create fully Dynamic Resources page for website

The resources page will be created using Django. There will be an easier method to post new resources using the Django admin UI.

I might be able to help in this issue. But currently I just have some of the knowledge of back-end development!


Allow to set extra environment variables on manager DaemonSet

Describe what this PR does
This PR adds the possibility of set extra environment variables in the manager DaemonSet.
This allows for example to support corporate proxy when downloading images
with ImagePullJob setting the HTTP_PROXY, HTTPS_PROXY, NO_PROXY env vars.
Does this pull request fix one issue?
Fixes https://github.com/openkruise/kruise/issues/1120
Special notes for reviews
This is my first PR on this project. I tried to follow the contributing guidelines checklist, bumping also a new beta version for changes. Let me know if it's preferable to move the PR to an existing version.
Note on DCO:
If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.
Checklist:

[ ] I have bumped the chart version according to versioning
[x] I have updated the chart changelog with all the changes that come with this pull request according to changelog.
[x] Any new values are backwards compatible and/or have sensible default.
[x] I have signed off all my commits as required by DCO.

Changes are automatically published when merged to master. They are not published on branches.

@mgeri tks, can you immediate modify in kruise 1.3.0 version ?

@zmberg sure, pushed the commit with changes in kruise 1.3.0 version

@mgeri I didn't express it clearly before. I think the EnvFrom field can be ignored for now. Is env enough at the current stage?

@zmberg Sorry, I did not understand. I agree. Pushed new changes. Thanks for the suggestions :-).

/lgtm

/approve


ceph mon's no longer start up on first ansible run

I've been failing around trying to solve my previous issue and my servers get to the stage where the playbooks try to 'start the monitor service' and this fails reliably every time on my servers.
I tried starting them manually (using systemctl start ceph-mon@host1.service), but to no avail.
However if I manually try the command that systemctl status<EMAIL_ADDRESS>suggests to me it tried to run, my monitor starts.
To shorten the pain i've felt over the last few hours trying to debug this, it turns out that the systemd config for the ceph monitors doesn't actually allow me to start the monitors up for 30 mins after a failure.
Once I realised that and editted<EMAIL_ADDRESS>reducing the StartLimitInterval setting to something much less than 30 mins, I can start the monitors using systemctl start without issue.
Now I am confused.... how on earth do I work out why the playbook cannot start my monitors on its first run? If I literally run them and let it fail, then edit the systemd file mentioned above reducing StartLimitInterval to 1, run systemctl daemon-reload, then start the monitors by hand myself, they start without issue.
Is this an issue with the ansible code or ceph itself?
I didn't have this problem 2-3 weeks ago, the monitors would start without issue on the first run of the playbook.
My system is a fully up to date Debian Jessie machine.

Do you have systemd logs? Ceph-mon logs? Just so we can understand the reason why the monitor didn't start.

That's what i've spent the best part of the last few hours trying to come up with. But I don't seem to get any actual useful logging in any of the log files.
The ceph mon logs just say this:
2017-02-03 15:55:13.532279 7fca0c21b600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:13.532287 7fca0c21b600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 34833
2017-02-03 15:55:13.911303 7f7af63bf600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:13.911316 7f7af63bf600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 34899
2017-02-03 15:55:14.108655 7f9e128e0600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:14.108669 7f9e128e0600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 34965
2017-02-03 15:55:14.450215 7f46b8fe1600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:14.450221 7f46b8fe1600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 34994
2017-02-03 15:55:14.699871 7f3ae7aab600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:14.699879 7f3ae7aab600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 35011
2017-02-03 15:55:14.948805 7f37b209c600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:14.948812 7f37b209c600  0 ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367), process ceph-mon, pid 35023
2017-02-03 15:55:30.511419 7f2b12efe600  0 set uid:gid to 64045:64045 (ceph:ceph)
2017-02-03 15:55:30.885159 7f2b12efe600  1 leveldb: Delete type=3 #1
I tried to work out how to make them give me something more useful, but mostly failed and then I realised that the systemd stuff I mentioned above.
Syslog says:
Feb  3 15:55:30 vm1 ansible-command: Invoked with creates=/var/lib/ceph/mon/ceph-vm1/keyring executable=None _uses_shell=False _raw_params=ceph-mon --cluster ceph --setuser ceph --setgroup ceph --mkfs -i vm1 --fsid 74dd3dec-4242-4810-8742-f4f6eec7a972 --keyring /var/lib/ceph/tmp/keyring.mon.vm1 removes=None warn=True chdir=None
Feb  3 15:55:31 vm1 ansible-systemd: Invoked with no_block=False name= enabled=True daemon_reload=False state=started user=False masked=None
Feb  3 15:55:31 vm1 systemd[1]: Starting Ceph cluster monitor daemon...
Feb  3 15:55:31 vm1 systemd[1]<EMAIL_ADDRESS>start request repeated too quickly, refusing to start.
Feb  3 15:55:31 vm1 systemd[1]: Failed to start Ceph cluster monitor daemon.
jounalctl -x tells me:
Feb 03 15:55:31 vm1 ansible-systemd[35730]: Invoked with no_block=False name= enabled=True daemon_reload=False state=started user=False masked=None
Feb 03 15:55:31 vm1 systemd[1]: Starting Ceph cluster monitor daemon...
-- Subject: Unit<EMAIL_ADDRESS>has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Unit<EMAIL_ADDRESS>has begun starting up.
Feb 03 15:55:31 vm1 systemd[1]<EMAIL_ADDRESS>start request repeated too quickly, refusing to start.
Feb 03 15:55:31 vm1 systemd[1]: Failed to start Ceph cluster monitor daemon.
-- Subject: Unit<EMAIL_ADDRESS>has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Unit<EMAIL_ADDRESS>has failed.
-- The result is failed.
I have no clue how to get better systemd logging, suggestions?

As far as I see, the mon logs say that the mon is up or was? So every time you bootstrap an initial mon, it fails, then later you can start it without apparent good reason and it works, right?

I'm not sure I agree with the first part of what you're saying, but I'm not very experienced with ceph.
The mon logs to me look like it tried to start 6 times in succession. After which I ended up with no mon process running.
systemctl (i've not touched the system since) says:
●<EMAIL_ADDRESS>- Ceph cluster monitor daemon
Loaded: loaded<EMAIL_ADDRESS>enabled)
Active: failed (Result: start-limit) since Fri 2017-02-03 15:55:15 GMT; 55min ago
Process: 35023 ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i --setuser ceph --setgroup ceph (code=exited, status=1/FAILURE)
Main PID: 35023 (code=exited, status=1/FAILURE)
Which tallies up with the timings of all but the last two entries in the mon logs.
The last two entries in the mon logs do tally up with the journalctl -x and syslog outputs though. Both of which suggest the mon failed to start.
I ended up with no mon process.
The latter part of what you say is correct though.
If I wait beyond the amount of time set in<EMAIL_ADDRESS>for the StartLimitInterval (default seems to be 30mins). And I try to start, the mon starts fine.
If I manually hack the StartLimitInterval setting in<EMAIL_ADDRESS>to be 1 and perform systemctl daemon-reload, then try to start the mon immediately it also starts.
I have not been able to work out why it doesn't start when ansible tries to start it. But then can be started almost immediately afterwards with the above hack.

@ashak can you start from scratch, run purge-cluster.yml, then comment out these lines: https://github.com/ceph/ceph-ansible/blob/master/roles/ceph-mon/tasks/main.yml#L5-L6 the playbook should fail after this, which is normal. Then try to run the monitor with systemd. See the result.
If this doesn't work, repeat the same operation and try to run the monitor manually from the command line and see if this works.

Running purge-cluster.yml failed on my hosts at:
TASK [include_vars] ***
fatal: [<IP_ADDRESS>]: FAILED! => {"ansible_facts": {}, "changed": false, "failed": true, "message": "Unable to find 'roles/ceph-common/defaults/main.yml' in expected paths."}
roles/ceph-common/defaults/main.yml definitely exists in my copy of the ceph-ansible git repo?!
I finished cleaning stuff up myself manually as much as possible with:
apt-get autoremove --purge ceph ceph-common; rm -r /etc/ceph /var/lib/ceph /etc/default/ceph
Then I commented out the suggested lines and ran the playbook again as requested.
It doesn't fail (at least it hadn't after a number of minutes). It sat at TASK [ceph-mon : collect admin and bootstrap keys (for or after kraken release)]  taking forever... Which I see is the first task after the commented out task.
I left ansible waiting and tried to start ceph using systemctl<EMAIL_ADDRESS>The results are as before. It doesn't start. None of the logs contain anything different to the logs in the second post on this thread. I'm sure there must be something I can do to obtain more verbose logging somewhere, but i've not managed to work that out yet. Suggestions welcome.
I purged again (with the same problem as before. Cleaned up manually, as before. And tried the second suggestion.
As with the first test, It sat at the same task waiting again.
I tried to start the mon manually from the command line which worked fine.
Interestingly, whilst waiting for it to get to the same place as before, I also kept looking at the status of<EMAIL_ADDRESS>on one of my hosts. Interestingly, after ceph was installed, but before ansible had got anywhere near to trying to start the mon, the status output changed to
root@vm1:~# systemctl status<EMAIL_ADDRESS>●<EMAIL_ADDRESS>- Ceph cluster monitor daemon
Loaded: loaded<EMAIL_ADDRESS>enabled)
Active: failed (Result: start-limit) since Mon 2017-02-06 14:14:16 GMT; 10s ago
Process: 59963 ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i --setuser ceph --setgroup ceph (code=exited, status=1/FAILURE)
Main PID: 59963 (code=exited, status=1/FAILURE)
Suggesting to me that after install something is already trying to start my mon's and that's failing. This would definitely explain why ansible itself is then unable to start them due to the 30 min restart limit in the systemd config for mon services.
Perhaps this has to do with me being unable to purge everything properly?
I'd rebuild my test hosts entirely and try it out with a clean slate (at least until I could get purge-cluster.yml working correctly), but I can't do that with these hosts until i'm onsite, which is now 11 days time.

To run the purge playbook, first copy it to the root of the repo then run it.
The waiting for keys is expected, at this point you should kill ansible and try to run the monitor manually with systemd.

Purge works much better now!
Will report back on running the tests again shortly.

OK,
Purging the cluster appears to solve my problem.
With the two lines commented out I can start the mons correctly.
I then put them back in and tried again, my mons start up properly.
Clearly I had something still lying around and my attempt at purging all the things wasn't good enough.
Apologies for wasting time with this. Back to the original issue


skaffold build|run fails in Cloud Shell with api-server

In the directory point-of-sale-app, when running skaffold build in Cloud Shell, I get this error on the api-server:
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] abm-edge-usecase-workload                                          [pom]
[INFO] service-sdk                                                        [jar]
[INFO] ui                                                                 [jar]
[INFO] api-server                                                         [jar]
[INFO]
[INFO] ----------------< com.google.abm-edge:abm-edge-usecase >----------------
[INFO] Building abm-edge-usecase-workload 0.1.0-SNAPSHOT                  [1/4]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- jib-maven-plugin:3.2.0:_skaffold-fail-if-jib-out-of-date (default-cli) @ abm-edge-usecase ---
[INFO]
[INFO] ----------------< com.google.abm-edge:abm-edge-usecase >----------------
[INFO] Building abm-edge-usecase-workload 0.1.0-SNAPSHOT                  [2/4]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- jib-maven-plugin:3.2.0:build (default-cli) @ abm-edge-usecase ---
[INFO] Skipping containerization of this module (not specified in jib.containerize)
[INFO]
[INFO] ------------------< com.google.abm-edge:service-sdk >-------------------
[INFO] Building service-sdk 0.1.0-SNAPSHOT                                [3/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ service-sdk ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /home/mabenoit/point-of-sale/point-of-sale-app/service-sdk/src/main/resources
[INFO] skip non existing resourceDirectory /home/mabenoit/point-of-sale/point-of-sale-app/service-sdk/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ service-sdk ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ service-sdk ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /home/mabenoit/point-of-sale/point-of-sale-app/service-sdk/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ service-sdk ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ service-sdk ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ service-sdk ---
[INFO]
[INFO] --- jib-maven-plugin:3.2.0:build (default-cli) @ service-sdk ---
[INFO] Skipping containerization of this module (not specified in jib.containerize)
[INFO]
[INFO] -----------------------< com.google.abm-edge:ui >-----------------------
[INFO] Building ui 0.1.0-SNAPSHOT                                         [4/4]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- frontend-maven-plugin:1.12.1:install-node-and-npm (install node and npm) @ ui ---
[INFO] Node v16.13.2 is already installed.
[INFO]
[INFO] --- frontend-maven-plugin:1.12.1:npm (npm install) @ ui ---
[INFO] Running 'npm install' in /home/mabenoit/point-of-sale/point-of-sale-app/ui
[INFO] npm ERR! code ENOTEMPTY
[INFO] npm ERR! syscall rename
[INFO] npm ERR! path /home/mabenoit/point-of-sale/point-of-sale-app/ui/node_modules/ansi-escapes
[INFO] npm ERR! dest /home/mabenoit/point-of-sale/point-of-sale-app/ui/node_modules/.ansi-escapes-XHR7Ud8o
[INFO] npm ERR! errno -39
[INFO] npm ERR! ENOTEMPTY: directory not empty, rename '/home/mabenoit/point-of-sale/point-of-sale-app/ui/node_modules/ansi-escapes' -> '/home/mabenoit/point-of-sale/point-of-sale-app/ui/node_modules/.ansi-escapes-XHR7Ud8o'
[INFO]
[INFO] npm ERR!  SUCCESS [  0.016 s]
[INFO] service-sdk ........................................ SUCCESS [  0.707 s]
[INFO] ui ................................................. FAILURE [  4.825 s]
[INFO] api-server ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.966 s
[INFO] Finished at: 2022-01-23T05:16:38Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.12.1:npm (npm install) on project ui: Failed to run task: 'npm install' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 217 (Exit value: 217) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :ui
maven build failed: exit status 1

It seems to complain while doing npm install, not sure if I'm missing anything to do in Cloud Shell before running skaffold build?

cooling down from our OSLO issues

@mathieu-benoit is it still active case?

Closed as an expected flaky test.


[BUG] dry_run require parameters

Describe the bug
If I'm not wrong I can pipeline.dry_run without any parameter as in pipeline.run. However, this is not allowed.
To Reproduce
Code to reproduce
with Pipeline(name="mistral-pipe", description="A pipeline to generate and score a distiset") as pipeline:

    load_dataset = LoadDataFromDicts(
        name= "load_dataset", data=[{"input": "test"}],
    )
distiset = pipeline.dry_run()

Expected behaviour
A clear and concise description of what you expected to happen.
Screenshots

Desktop (please complete the following information):

Package version: 1.1.0
Python version: Python 3.10.12

Additional context
Add any other context about the problem here.

Thanks for reporting @sdiazlor, will tackle this later today.


Support different auth types in backend

Add support for other auth types.
Adds HTTP Basic auth.
Needed for metrics and other endpoint types that don't use OAuth2

LGTM, merging this before PR https://github.com/cloudfoundry-incubator/stratos/pull/1617 because it has the Migrator changes


Language switcher not working with subfolders

On http://testing.88k.com.tw, I have the default language set to Chinese and the English language in as a subfolder. I set up siteurls as such
'siteUrl' => array(
         'zh' => 'http://testing.88k.com.tw/',
         'en' => 'http://testing.88k.com.tw/en/',
        )

Perhaps your plugin doesn't handle this or am I missing something?

That looks like it should work... What URLs are your links actually generating?
Can you paste the Twig snippet you're using with the Language Link code?

Did you work this out? If this is no longer an issue, I will close this thread.


feat(support): improve helpers

This pull request adds some improvements to ArrayHelper and StringHelper:
ArrayHelper

Added a explode constructor for better chaining (I wanted to use that in #513)
Added support for no callback on the filter method—by default, it filters out false and null values (strictly)
Added a dd method to help with debugging

StringHelper

Added Stringable support where it made sense
Added toString for better chaining (__toString is ugly, (string) feels bad)
Renamed ld to dd (I learned that we could ignore a line with phpstan, and dd is more intuitive to reach for than ld)

Nice PR! I'm going to give @brendt the honor of reviewing this one because there some stylistic choices that had his influence here. 🙂

Looks good! I don't know about dd though, I've grown used to ld by now, but I don't have a strong opinion on it. Maybe we can simply provide both? Anyway, happy to merge this, and feel free to share your opinions on it afterwards


RTC GPIO deep sleep ESP32S3 (IDFGH-6711)

I wrote this here, because our offical contact William  檀军华 is on holiday and referred me here.
This topic is important, we are about to start mass production.
We want to use 3 Pins to wakeup from deep sleep, Pin 8 going to 1, Pin 20 and 21 going to 0.
There is no API to do that. But what is worse is that the datasheet does not contain the information about ext1 and ext0 source as wakeup. the chapter about RTC_CNTL_EXT_WAKEUP is completely missing in 0.3: https://www.espressif.com/sites/default/files/documentation/esp32-s3_technical_reference_manual_en.pdf
For S3 there is no config like : SOC_GPIO_SUPPORT_DEEPSLEEP_WAKEUP
I cannot use use esp_sleep_enable_ext1_wakeup, because the mode is set once and we have inverted pins.
I cannot use esp_sleep_enable_ext0_wakeup, it overwrite the pin is selected.
I have already looked deep into hal, at a level that users should not use, but find no solution.

Hi @hardyacht ,
For your case, you may need to use EXT0 and EXT1 together, with different triggering level.
I have tried that on master, both ext0 and ext1 is working properly. (However, the logic of EXT1 seems to be ANY_HIGH or ANY_LOW, unlike the enum name, I'm checking this).
You may need to properly configure the internal pullup/downs if you don't have external ones to make the pins stable during deepsleep. See https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/sleep_modes.html#external-wakeup-ext0 for

Thanks for reporting, sorry for slow turnaround, fix is available https://github.com/espressif/esp-idf/commit/5a2f672d5aa60ed6286bf232afed804d2f4c1c67, feel free to reopen.

@Alvin1Zhang in which official release will this fix be included?

Thanks for reporting and the notes, the fix will be included in the forthcoming 5.0, and will back port to release/4.3 and release/4.4, thanks.

Is there a time when a new tagged release is planned for the 4.4?


WIP: script for Shango  in local LAN

as we don't have something for fixed ip yet, this only does settings for local

[ ] check wallet unlock
[ ] add button to display qr code for connection in menu?
#44

did not run it on fresh install yet.

exited to play with that next week :)


initial horrible implementation of search on 404

add search div on 404
append ?q=path to url to prepopulate the search box with the path for the 404

Not working for me, doing some horrible redirects.

closing this as @Ancairon put me to shame :)


Support for pamqp>=2.0.0

Now that pamqp 2.0.0 is out, but setup.py still states that it's unsupported, I was wondering if there are any known issues.

Any status on this? Trying to use pytest-rabbitmq and aio-pika together, but this package is holding me back!

I'm in the process of readying pamqp 2.2.0 and will update rabbitpy when that drops.

Hey @gmr 👋  Thanks for the quick reply!
I just wrote a quick-and-dirty rabbitmq pytest fixture that uses subprocess and docker under the hood, so no worries. I would definitely prefer to use pytest-rabbitmq though.
That sounds great. Looking forward to it! 😄

2.0.0 was released and pushed to pypi today. It's worth noting that it has a breaking behavior of changing the type of field-table values from bytes to str.


actual SpeciesFeature support

resolves #72
Just starting to sketch this out...

Going to drop some other thoughts here as they come up...
Fairly minor point, but a change that I think could help clarify things for some people: should we change the "callable" action of an FD on a structure to be raw rather than encoded value? Since it's very unlikely anybody will use that signature to actually encode things anyway, it’s probably more useful/intuitive to be able to get a human-readable value out of that, especially since we will now have more separated notions of computing a value and encoding it since the computation can be less trivial than a lookup table.

We also would need to featurize more than just atoms too.

For Weave yeah, we need BondFeatureDescriptor actually implemented too, but let's do one thing at a time. ;) Once we sort this stuff out, that one should fall into place rather cleanly.

Are we removing the AbstractAtoms at
https://github.com/Chemellia/ChemistryFeaturization.jl/blob/aa7d310bc427cb6a2f20e6e5d09ae0bfe37baea8/src/features/features.jl#L32
?

I guess that's a fair point; if we do what I suggested, we would have to remove that dispatch of encode...which I guess might actually get used internally sometimes...in either case, I think it would be useful to explicitly have a function to get the value of a feature vs. the encoded value since in many cases they're different. @thazhemadam, thoughts?

What is the API to add a new feature that you're envisioning? Currently something like struct MGDegree <: ChemistryFeaturization.FeatureDescriptor.AbstractAtomFeatureDescriptor end is needed, which can then be called via encode(fd, ::GraphMol) to return the matrix, which can then be used elsewhere. I'm not sure CF can control what can go inside the encode since the feature descriptor could come from anywhere and only work with types not in control of CF. MolecularGraph is only the first, and might be one of the more forgiving ones.

What I'm envisioning is that we hopefully avoid defining lots of new structs (maybe this is personal preference, but I kind of hate reading code with lots of narrow-use-case types, especially if I have to go to some other file to track down what they actually are), so the goal would be to do all of these as instances of SpeciesFeatureDescriptor...I'm thinking to start with I'll create a bunch of "built-in" things along the lines of what we did for EFD's, but rather than a lookup table of values, it's a lookup table of functions. This hopefully will also give a template to users/developers of how they could implement another one themselves if needed by providing the relevant function.

Removing special dispatches as much as possible is a good thing. If the method exists, it will get called anyway, if it doesn't it will clearly say why via a MethodError (and also indicate what needs to be defined), and Julia will always specialise for the type, whether we type it or not.

hate reading code with lots of narrow-use-case types

I tend to agree. We can also dispatch on functions directly, as I had mentioned. I think there might be a case for cleaning up some of the types in CF too.

it's a lookup table of functions

so... dispatch? It would be much faster considering dispatch is resolved at compile time (as far as possible, with runtime dispatch being pretty fast anyway), and lookup will happen at runtime and will always have additional overhead (which is much more than the runtime dispatch).

it's a lookup table of functions

so... dispatch? It would be much faster considering dispatch is resolved at compile time (as far as possible, with runtime dispatch being pretty fast anyway), and lookup will happen at runtime and will always have additional overhead (which is much more than the runtime dispatch).

Okay, haha, fair point, but if we do it that way then (SpeciesFeatureDescriptor becomes an abstract type, and also) the user has to define a new struct for any new feature they want to encode. I guess this is probably okay as long as we provide very clear instructions in the docs, I just feel like it could seem intimidating for some people.

Oh, I guess the function dispatch could be a plausible "intermediate" solution to creating a new type for every feature, as you said.

Fairly minor point, but a change that I think could help clarify things for some people: should we change the "callable" action of an FD on a structure to be raw rather than encoded value? Since it's very unlikely anybody will use that signature to actually encode things anyway, it’s probably more useful/intuitive to be able to get a human-readable value out of that, especially since we will now have more separated notions of computing a value and encoding it since the computation can be less trivial than a lookup table.

So instead of the encoded value, this would return the value that would be encoded? I think that makes more sense than the current behaviour tbh.

Okay, haha, fair point, but if we do it that way then (SpeciesFeatureDescriptor becomes an abstract type, and also) the user has to define a new struct for any new feature they want to encode. I guess this is probably okay as long as we provide very clear instructions in the docs, I just feel like it could seem intimidating for some people.

I think SpeciesFD should be an abstract type.
If we need a lookup table of functions to decide what we're dispatching on, those types are probably better off as separate types, with SpeciesFD as their super-type. The "lookup table of functions" idea sounds to me like we're trying to do what the compiler should be doing, but very poorly in comparison.

I think separating the original value vs the encoded value is a good thing. The way to handle it would usually be a function that looks like encode(f, x) where f can generically operate on the input and always return the encoded value (whose return type can be made a requirement for the time being), and maybe have a separate real_value(x::AbstractFD) =  x.value (say).
We aren't "removing the dispatch" by getting rid of the type in that statement. Notice that it isn't doing anything special that any other feature wouldn't be expected to support anyway. Removing the type only means we are able to input things like GraphMol and output an AtomGraph or FeaturizedAtoms.

Might help to see the type hierarchy we have for the FeatureDescriptors.
Julia> print_tree(ChemistryFeaturization.AbstractType.AbstractFeatureDescriptor)
AbstractFeatureDescriptor
├─ AbstractAtomFeatureDescriptor
│  ├─ SpeciesFeatureDescriptor
│  ├─ ElementFeatureDescriptor
│  └─ MGDegree
├─ AbstractEnvironmentFeatureDescriptor
│  └─ OrbitalFeatureDescriptor
└─ AbstractPairFeatureDescriptor
   ├─ BondFeatureDescriptor
   │  ├─ BondType
   │  ├─ InRing
   │  └─ IsConjugated
   └─ PairFeatureDescriptor

The question becomes, what interface would a new feature be expected to follow, and where in this tree would such a feature fit in with the right tradeoffs. Making new features as instances of existing FeatureDescriptors with a hook out for adding a completely separated system for encoding would be fine too I think. Notice that MGDegree could also be expressed as an instance of SpeciesFeatureDescriptor.

We definitely need a way to keep the number of types getting out of hand.

Another thing I thought was: If we need parameters, handling vectors, or anything of the sort, using a function can become somewhat illegible. Consider writing a feature as a function that requires additional arguments and parameters than the current system would allow without types.
function myfeature(x, extra_arg) ... end

To supply the additional arg in the FeatureDescriptor would look like
julia> SpeciesFeatureDescriptor(x -> myfeature(x, some_arg), ...)
SpeciesFeatureDescriptor{#47}(#47, ...)

losing the ability to "see" what the FD is.
Types don't have that issue, of course, but its a minor point I thought I'd share.

Okay, so it seems there's consensus on changing the "callable" syntax of FD's in general so I'll for sure do that.
In terms of how to implement species FD, I don't think we've come to a 💯  clear consensus. There seem to be a few options:

SFD becomes an abstract type, every feature is a concrete subtype --> my sense is we're leaning against this because it leads to a clunky proliferation of types, and also more work for a user who wants to add a new feature to support
what I had originally suggested, wherein SFD has a field that stores the function, and also takes a type parameter describing what "origins" of atoms objects it can act on (this dispatch is fairly clean because you just match the type parameters)
@DhairyaLGandhi's suggestion, where the function itself becomes a type parameter (possibly in addition to the type parameter I described above, which is also necessary IMO). Then we can dispatch on both the "matching" of underlying structure representation and also on the evaluation of the function. AFAICT, this is basically an entirely superior version of option 2 (unless, I suppose, you dislike types with multiple parameters).

So if this sounds reasonable, I'll push forward with option 3? In terms of actual implementation, how does that work? e.g., will it work to do (where A is the atoms type and F the function to compute the feature's value)
struct SpeciesFeatureDescriptor{A,F} <: AbstractAtomFeatureDescriptor
    # existing stuff that doesn't make explicit reference to A or F
end

and then would I be able to still access F to call it? Or do I need fields inside the struct that also store these? If not, it seems impossible to get away from the thing @DhairyaLGandhi had discouraged, namely, using type parameters in constructors...

Somewhat distinct discussion, relevant for docs and also for eventual names of fields, etc.: What is clear terminology for distinguishing values vs. encoded values of features? And similarly, the functions the compute those things, which might be even trickier to figure out, since I don't think something like raw_value vs. encoded_value is all that bad, and encode_f for the encoder is fine, but then what do you call the function that gets the raw_value in the first place?!? #ntih amirite @thazhemadam? 😭

What's the callable syntax now and the updated version?
Generally if you want a type param in the constructor, it's alright but couldn't that also be gotten via a constructor that accepts a type or can infer the type? If there isn't a preferred solution then I'd suggest with having the type-param-in-constructor so as to not halt progress

Yes, concrete types can have type params that the abstract type doesn't have, and you can access them like usual. Storing it as a function would be okay too.

Okay so in 7d08383 I made a couple changes summarized in the commit message, but I wanted to make a couple more comments here...
First, a short code snippet demo of the changed syntax to distinguish get_value (which is now what is called by the callable syntax on an FD) vs. encode:
julia> ag = AtomGraph(Float32.([0 1; 1 0]), ["H", "O"])
AtomGraph{SimpleWeightedGraph{Int64, Float32}}  with 2 nodes, 1 edges
        atoms: ["H", "O"]

julia> fd(ag)
2-element Vector{String}:
 "s"
 "p"

julia> encode(fd, ag)
4×2 Matrix{Float64}:
 1.0  0.0
 0.0  1.0
 0.0  0.0
 0.0  0.0

I like this, personally. 👍
Second, in thinking about this, I've realized that as currently implemented, OneHotOneCold is not actually functionally all that different from @thazhemadam's SimpleCodec, since the actual onehot/onecold logic is all in EFD-specific utility functions. This commit is the first step towards separating that, but I'd like to finish moving that logic over there in a subsequent commit if you guys think it makes sense. Basically, the only things that need to be FD-specific are the computation of the value (now separated by the get_value function) and also figuring out what the bins are (we already have the EFD-specific get_bins function). Once you have those two things, the actual onehot encode/onecold decode are totally generic, and we should be able to move them all over there. I'll do that it another commit so hopefully if we want to revert one/both of these it won't be too hard to disentangle.
THINGS STILL TO DO!

[ ] actual SFD-related things (this discussion has led to a couple commits that have actually only changed EFD stuff, but I think it's important groundwork)
[ ] fix tests, update changelog and docs (didn't want to do this until you guys were on board with these changes...the tests will fail hardcore at the moment because some functions have changed names 😆 )

Does this mean fd(ag) == get_value(ag)?

Does this mean fd(ag) == get_value(ag)?

Yup!

Okay, 61a3abe now has the additional changes I was alluding to...
The basic concept is to continue the separation so that calculation of a feature’s value happens entirely within the EFD, and the codec only handles encoding that to onehot or decoding it back. I’ve made some syntax changes in line with this…
We previously had a callable syntax on codecs. I’ve removed this because I don’t think it’s necessary and I do think it’s confusing. We now explicitly call encode(codec, val) or decode(codec, val). The loose philosophy here is “FD’s act on Atoms, Codecs act on values.” One exception is that I did keep encode(fd, atoms) and decode(fd, val) as basically aliases to the codec analogues, as I think these are useful and not confusing.
For now, the logic for how to combine the encoded vectors for individual AFD’s when encoding an entire atoms object is fixed to just be hcat. We may want to think at some point about generalizing that, but it will become trickier to define the decoding in that case.
@thazhemadam, eager for your thoughts on any/all of this!

Separate minor but annoying issue: I've somehow screwed up the scope of the get_value function so that it's accessible to internal functions in the package without issue, but when I do using ChemistryFeaturization in the REPL, I have to explicitly invoke it with FeatureDescriptor.get_value AND I CAN'T FIGURE OUT WHY (the main module ChemistryFeaturization.jl has export get_value in it, what else do I need?!?). @DhairyaLGandhi, are you a scope wizard in addition to an AD wizard? 🧙

Also: so proud that as of now, this PR is a net decrease in lines of code. 💃

Second, in thinking about this, I've realized that as currently implemented, OneHotOneCold is not actually functionally all that different from @thazhemadam's SimpleCodec, since the actual onehot/onecold logic is all in EFD-specific utility functions.

I'm not sure what exactly this means. 😅  SimpleCodec is basically a skeleton, with just the encoding and decoding functions coupled together (which as of now we expect every Codec to have at the very least). The only difference is that OneHotOneCold actually has additional attributes that are also required for actually encoding/decoding using the encode_f/decode_f field.

Basically, the only things that need to be FD-specific are the computation of the value (now separated by the get_value function) and also figuring out what the bins are (we already have the EFD-specific get_bins function).

I assume you mean this with respect to onehot-onecold only? I assume one might need codec schemes that may be based on some other attributes of the data being represented by get_value itself. (however, I can't think of any, so it could possibly be me being overly cautious too lol).

We previously had a callable syntax on codecs. I’ve removed this because I don’t think it’s necessary and I do think it’s confusing. We now explicitly call encode(codec, val) or decode(codec, val). The loose philosophy here is “FD’s act on Atoms, Codecs act on values.” One exception is that I did keep encode(fd, atoms) and decode(fd, val) as basically aliases to the codec analogues, as I think these are useful and not confusing.

This sounds like a good idea to me.
One reason why I don't like the current callable syntax is that Codecs facilitate two things at once - encoding and decoding, and having a single callable syntax that determines if it has to encode or decode depending on parameters always has been kind of clunky imo. The benefit of having it is that we can say "take this codec, and apply its codec-y logic on this atoms object for this feature descriptor". It's more restrictive, and users would be less prone to making silly mixups with the types, which I reckon we would still be able to enforce using the analogous encode and decode as necessary.
I'm on board with these changes. Assuming I've understood this correctly, it's worth taking a moment here to also see that this shifts the responsibility of actually encoding and decoding to the encode and decode functions. It would also fundamentally change the purpose of a Codec from "what does the encoding and decoding" to "what helps you do the encoding and decoding".

Separate minor but annoying issue: I've somehow screwed up the scope of the get_value function so that it's accessible to internal functions in the package without issue, but when I do using ChemistryFeaturization in the REPL, I have to explicitly invoke it with FeatureDescriptor.get_value AND I CAN'T FIGURE OUT WHY (the main module ChemistryFeaturization.jl has export get_value in it, what else do I need?!?)

I'd be curious to know if there's a (more?) idiomatic way of going about this. IIRC, I've done this only in a "weave-in, weave-out"  way by importing it from the main module and exporting it from the submodule. (for instance, see  7e6ad96859f401972408488f201621b05db020d9)

Re scoping - yeah. There's two things needed to fix this. First is that you should export get_value from the FeatureDescriptor module. Second, you need to actually call using .FeatureDescriptor in src/ChemistryFeaturization.jl without which, get_value was never included in the scope of ChemistryFeaturization the main module. Currently, only the ElementFeatureDescriptor and FeatureDescriptor (the symbol) are.

Maybe we can overload Base.values for get_value?

As nice as this sounds, I don't think it's a good fit. Base.values takes in an iterator and returns the value represented at that position in the data structure, which isn't logically the same thing as get_values does.

Finally getting back to this stuff after being tied up with some other things the past ~week and a half...first, to @thazhemadam's comment:

Second, in thinking about this, I've realized that as currently implemented, OneHotOneCold is not actually functionally all that different from @thazhemadam's SimpleCodec, since the actual onehot/onecold logic is all in EFD-specific utility functions.

I'm not sure what exactly this means. 😅 SimpleCodec is basically a skeleton, with just the encoding and decoding functions coupled together (which as of now we expect every Codec to have at the very least). The only difference is that OneHotOneCold actually has additional attributes that are also required for actually encoding/decoding using the encode_f/decode_f field.

Based on the rest of your comments, I think you ended up figuring out what I meant even if I didn't explain it all that clearly, but I'll try again here just in case: Basically, we had this thing called OneHotOneCold and we called it a Codec, which I would intuitively interpret to mean that it handles encoding/decoding of the one-hot/one-cold variety. However, what it actually had was just generic encode_f and decode_f functions, and the fact that they happen to do one-hot/one-cold encoding/decoding wasn't enforced within the codec itself, but rather by EFD-specific functions that were fed into it. With the changes I'm making here, the encode/decode logic is moved into the codec itself, which both makes more intuitive sense to me and feels like a cleaner separation of concerns all-around. Is that clearer? 😆

Basically, the only things that need to be FD-specific are the computation of the value (now separated by the get_value function) and also figuring out what the bins are (we already have the EFD-specific get_bins function).

I assume you mean this with respect to onehot-onecold only? I assume one might need codec schemes that may be based on some other attributes of the data being represented by get_value itself. (however, I can't think of any, so it could possibly be me being overly cautious too lol).

In this case, I did mean OHOC only, but to me the fact that we've separated out the codec at all sort of implies a thesis that that separation is in general robust, and given I also can't think of any, I would argue we don't need to worry about it right now. #YAGNI

and re @DhairyaLGandhi's comment...

Re scoping - yeah. There's two things needed to fix this. First is that you should export get_value from the FeatureDescriptor module. Second, you need to actually call using .FeatureDescriptor in src/ChemistryFeaturization.jl without which, get_value was never included in the scope of ChemistryFeaturization the main module. Currently, only the ElementFeatureDescriptor and FeatureDescriptor (the symbol) are.
Re - idiomatic way: Yeah we usually don't have these many modules that are all intertwined and interdependent, there really isn't much benefit. You want them to be as self contained as possible. It causes the all the using statements in the files and confusion about what gets exported and to where. Also, I'd generally try to be cautious about what I export. exporting get_value , encode, decode etc seem like they could cause namespace collisions with other packages and their purpose can't directly be tied to CF because the names are too generic. Maybe we can overload Base.values for get_value?

Thanks for the scoping fix...I will someday get all that through my head and be able to it right the first (or, like, fifth) time.
Regarding the namespace conflicts, that's a reasonable point. My first thought was to add feature into the names e.g. encode_feature, but that's a bit confusing since it works for individual FD's but also featurizations, so I don't think that would work, though it would probably be okay for get_feature_value. We could obviously just not export them so you either explicitly import the function yourself as a user or always call ChemistryFeaturization.encode. I personally don't love that as it seems inconvenient, but it is probably the safest option. I agree with @thazhemadam on the Base.values not really being a good fit for that case...

Since it seems that there's general consensus on the changes being made here, my next plan is to update tests so that they pass with this stuff, and then start actually populating the SFD utils with some MolecularGraph functions so we can (finally!) start playing with those more "idiomatically."

Phew, that was fun. Note to @thazhemadam to do a review specifically on the orbitalfeature-related changes I made.

but that's a bit confusing since it works for individual FD's but also featurizations

Seems reasonable to me, what would the source of confusion be?
Re values there's usually a bit of give in the interpretation of what what the APIs do in different contexts. Think of names(DataFrame) vs names(Module), but avoiding it for conceptual clarity is fair game.

Okay, starting to actually populate the SpeciesFeature stuff. See last commit message for some notes, but also here's the test code to see what's broken:
using MolecularGraph, ChemistryFeaturization
caffeine = smilestomol("CN1C=NC2=C1C(=O)N(C(=O)N2C)C")
ag = AtomGraph(caffeine)
sfd = SpeciesFeatureDescriptor("isaromatic")
get_value(sfd, ag) # this doesn't work, because the type parameters in the next line don't match...
typeof.([ag, sfd])

Specifically, here's the result:
julia> typeof.([ag, sfd])
2-element Vector{DataType}:
 AtomGraph{GraphMol{SmilesAtom, SmilesBond}}
 SpeciesFeatureDescriptor{GraphMol}

I assume there's gotta be some way to get those to register as the same thing (i.e. ignore the type parameter of GraphMol), but I'm not sure of the syntax to do it...

I'll track this down. I think it might have something to do with version incompatibilities introduced in https://github.com/Chemellia/ChemistryFeaturization.jl/pull/118/commits/b3d1c905ab71b006ab35ce0dcf0e4d1ed57d7260 (I remember running into something similar a while ago when I tried to up this locally), but I'll know for sure when I bisect this.

so maybe this is a LightGraphs vs. Graphs thing...

I don't think that's likely the issue here as LightGraphs should still work properly.

Were we able to track down the failures


Grav-v1.0.10 : Blank Screen on apache and 500 Internal Server Error on caddy server

my localhost on Arch linux with Grav-v1.0.10
Apache/2.4.20 (Unix)
PHP 7.0.5
mode_rewrite enable

Result : Blank screen

Caddy 0.8.2
PHP-FPM 7.0.5
with rewrite

Result : 500 Internal Server Error

Caddyfile
localhost:8080
gzip
errors error.log
fastcgi / /var/run/php-fpm/php-fpm.sock php

# Begin - Security
# deny all direct access for these folders
rewrite {
    r       /(.git|cache|bin|logs|backups|tests)/.*$
    status  403
}
# deny running scripts inside core system folders
rewrite {
    r       /(system|vendor)/.*\.(txt|xml|md|html|yaml|php|pl|py|cgi|twig|sh|bat)$
    status  403
}
# deny running scripts inside user folder
rewrite {
    r       /user/.*\.(txt|md|yaml|php|pl|py|cgi|twig|sh|bat)$
    status  403
}
# deny access to specific files in the root folder
rewrite {
    r       /(LICENSE.txt|composer.lock|composer.json|nginx.conf|web.config|htaccess.txt|\.htaccess)
    status  403
}
## End - Security

# global rewrite should come last.
rewrite {
    to  {path} {path}/ /index.php?_url={uri}
}
![screenshot from 2016-04-23 23-04-21](https://cloud.githubusercontent.com/assets/11581180/14762571/d3c0cd3c-09a7-11e6-93a9-b5351c6f444c.png)

any help, please?

does your caddy sever error log give you any detail?

These errors can be caused by missing PHP libraries.  not familiar with ArchLinux, but on Ubuntu/Debian they break out the PHP packages, so you need to ensure you have the critical ones that are usually part of the standard PHP build.
https://learn.getgrav.org/basics/requirements#php-requirements

Re-check my php-modules
PHP 7.0.5 , PHP-FPM 7.0.5
GD 2.1.1  enabled
curl 7.48.0 enabled
OpenSSL/1.0.2g
Zip 1.13.2
mbstring enabled / libmbfl version 1.3.2 / oniguruma version 5.9.6
xml 2.9.3 active

Caddy Error logs
24/Apr/2016:21:54:28 +0700 [ERROR 500 /] PHP message: PHP Fatal error:  Uncaught RuntimeException: Opening file for writing failed on error fopen(/srv/http/user/config/security.yaml): failed to open stream: Permission denied in /srv/http/vendor/rockettheme/toolbox/File/src/File.php:202
Stack trace:
#0 /srv/http/vendor/rockettheme/toolbox/File/src/File.php(316): RocketTheme\Toolbox\File\File->lock()
#1 /srv/http/system/src/Grav/Common/Config/Setup.php(250): RocketTheme\Toolbox\File\File->save(Array)
#2 /srv/http/system/src/Grav/Common/Config/Setup.php(175): Grav\Common\Config\Setup->check(Object(RocketTheme\Toolbox\ResourceLocator\UniformResourceLocator))
#3 /srv/http/system/src/Grav/Common/Grav.php(213): Grav\Common\Config\Setup->init()
#4 /srv/http/index.php(38): Grav\Common\Grav->process()
#5 {main}
  thrown in /srv/http/vendor/rockettheme/toolbox/File/src/File.php on li...

Permission denied ??? how to fixed, please?
This command  not work!
chgrp -R http .
find . -type f | xargs chmod 664
find . -type d | xargs chmod 775
find . -type d | xargs chmod +s
umask 0002

Well it seems like you might of already read this: https://learn.getgrav.org/troubleshooting/permissions
But the problem lies in your files have permissions such that the webserver is unable to write to them.  When I run php-fpm i like to setup the pool so that it runs with the same user/group that own the files.  You can see how I do this in this DigitalOcean writeup that uses Nginx and PHP-FPM: https://learn.getgrav.org/hosting/vps/digitalocean#configure-nginx-connection-pool
Most hosting providers use SuPHP to achieve the same result, but if you do this, you will never have any permissions issues ever again :)

BTW, you don't have to use your own user, you can create another unprivileged user that you use for both PHP and files access.  However, most setups have webserver/php running with a user that has no shell, so you can't login or su to it.

fixed with
chown http:http .
chown -R http:http *
find . -type f | xargs chmod 664
find . -type d | xargs chmod 775
find . -type d | xargs chmod +s
umask 0002

I am use Arch Linux, another distro maybe different..

You'll just need to make sure than you always chown the files to http:http if you edit/create from the terminal/ftp.  That's why i prefer to use a user that can login.


Added some new configuration options.

I have created a branch with some new configuration options which were useful for a project I was working on. The current master branch always displays the day view by showing the hours 00:00 -> 23:00. If the control is being used as strictly a time picker then sometimes it would be nice to restrict this to a set of working hours (e.g. 08:00 -> 17:00). To facilitate that, I have added some new options:

dayViewHoursStart - The hour to start the day view at. e.g. 8 will show 08:00 as the first hour in the day view popup. (default 0)
dayViewHoursEnd - The hour to end the day view at. e.g. 17 will show 17:00 as the last hour in the day view popup. (default 23)
showHeaderView - Whether or not to show the date/navigation header in the popup. This can be hidden via CSS anyway but it is good to be able to configure it. (default true)

A screenshot with dayViewHoursStart: 8 and dayViewHoursEnd: 17:

A screenshot with showHeaderView: false:

Thanks @barriestewart for this proposal, I really like it. I would just have changed the name of the properties in hourStart, hourEnd, and showHeader. Using the same concept, I will add the equivalent properties for the Hour View and the Minute View, I mean minuteStart, minuteEnd, secondStart and secondEnd. Anyway, let's go for it, just make these little changes and let's merge the pr.
--
Hi @fchesaru, your issue is due to the locale you use. Anyway, you can change the format of the minutes in the Hour View using the MomentPickerProvider and setting the minutes-format property, like this:
angular
    .module('myApp', ['moment-picker'])
    .config(['momentPickerProvider', function (momentPickerProvider) {
        momentPickerProvider.options({
            minutesFormat: "HH:mm"
        });
    }]);

Ok @indrimuska would you like me to just change the names of those 3 parameters in my branch then you will add the necessary changes for minuteStart, minuteEnd etc.?

Yes, please. If you want, or you don't have time, I can merge the branch as-is and then I will change them by myself. Just let me know what you prefer :)

FYI, v0.6.8 is now out with these changes.
Thank you @barriestewart for spending your time in this project! :)

@indrimuska, thank you for the tip! I was not sending the application locale to the directive, so the default (US) was always applied.  Keep up the good work!

@indrimuska Sorry I was away on holiday. Thanks for merging these changes and making your additions.


Cannot change modes between VISUAL and VISUAL LINE

I think this is a issue related to #465, I can't change modes between VISUAL and VISUAL LINE.
That means while I'm in VISUAL mode by v key, I can't switch to VISUAL LINE mode by uppercase V, or vice versa.

Sorry for this mess, I didn't noticed this mess, just wanted to fix my commit log, sorry I should learn git.


[BUG] Ugly references to relative path

** Which version the bug has been detected **
Luos engine 2.6.0
Describe the bug
It's not a bug, but it's a bug for our eyes.
In all platformio projects (including other luos repos), libs are references as relative path such as ../../../.. etc...
These relative paths should be replaced by an absolute one. It will be easier to maintain.
*It was an idea of @K0rdan : thank you dude !
We could add variables in python pre-script "LUOS_ENGINE", "TOOLS_SERVICES", "WHATEVER_YOU_WANT" to point on abolute path and then use those references in platformio.ini

Fixed in luos_engine 2.6.0