VirtualBox nixos guest missing 3D acceleration

In NixOS 16.03 I enable VirtualBox guest additions (virtualization.virtualbox.guest.enable = true) and I properly get Direct Rendering in X.org but it still uses the swrast GL code. In VirtualBox I do enable "3D acceleration" but I guess there should be some userlib in the guest that makes GL go through the vm for the acceleration.
Does anybody know how to get this 3d acceleration?
Thank you.

Did you check if guest additions are correctly installed and enabled?

Sure, they are installed. I also made the guest nixos module put virtualbox_dri.so to /run/opengl-driver/lib/dri, but that did not help.
My X log only says:
vboxvideo: kernel driver found, not loading.
modeset(0): using drv /dev/dri/card0
...
AIGLX: Screen 0 is not DRI2 compatible
AIGLX: reverting to software rendering

I tried with 'modeset' in cmdline too, no difference.

Ok, I have a dirty patch that makes it work. Basically, libGL.so.1 should point to VBoxOGL.so, instead of the mesa one. Then, acceleration works.
But in my machine, Optimus (Intel+NVidia), I fail to make the acceleration go through NVidia and I only get the Intel acceleration. Like this poor guy: http://superuser.com/questions/779070/use-nvidia-gpu-from-virtualbox  (but it is another topic).
We should think how to add a nixos option that makes the libGL.so.1 override in opengl-drivers/lib

btw, here is the source from where I got the idea of the symlink: https://bugs.archlinux.org/task/49752

I was able to get it to work using the link you provided:
LD_PRELOAD=/nix/store/jbyg9nzcg405qw8byjg5p35fnhkknivz-system-path/lib/VBoxOGL.so glxinfo | grep -i render
Any ideas how to make this the default using my configuration.nix?
thanks!

I think that #9415 may be somewhat related.

Also #1608 and #5051

I'm having the same issue. I can get good output from glxinfo in a similar way to @husnoo, but I am not sure where to go from here.

I'd been meaning to submit a PR for this, but unfortunately I'm no longer on VirtualBox.
Last time I tried it this expression in my config worked more or less:
{ pkgs
, config
, ...
}:

let
  makePackage = p: p.buildEnv {
    name = "mesa+vbox+txc-${p.mesa_drivers.version}";
    paths =
      let version = "5.1.12";
          virtualboxGuestAdditions = p.linuxPackages.virtualboxGuestAdditions.overrideDerivation (oldAttrs: {
              /* src = p.fetchurl { */
              /*   url = "http://download.virtualbox.org/virtualbox/${version}/VBoxGuestAdditions_${version}.iso"; */
              /*   sha256 = "08a3vycj7yw6ihb3myi9yhag1ivpwdl9m5c33viw66lpmdf2d80k"; */
              /* }; */
              buildCommand = lib.traceShowVal oldAttrs.buildCommand +
                ''
                  ln -s $out/lib/VBoxOGL.so $out/lib/libGL.so
                  ln -s $out/lib/VBoxOGL.so $out/lib/libGL.so.1
                  ln -s $out/lib/VBoxOGL.so $out/lib/libGL.so.1.2.0
                  # TODO: in the next version add EGL?
                  # patchelf --set-rpath ${lib.makeLibraryPath (with p; with p.xorg; [ "$out" dbus libXcomposite libXdamage libXext libXfixes ])} lib/VBoxEGL.so
                  # cp -v lib/VBoxEGL.so $out/lib
                  # ln -s $out/lib/VBoxEGL.so $out/lib/libEGL.so
                  # ln -s $out/lib/VBoxEGL.so $out/lib/libEGL.so.1
                  # ln -s $out/lib/VBoxEGL.so $out/lib/libEGL.so.1.2.0
                '';
            });
      in
        [ virtualboxGuestAdditions # for the DRI driver
          # p.mesa_drivers
          # p.mesa_drivers.out # mainly for libGL # (TODO: replace?)
          (if config.hardware.opengl.s3tcSupport then p.libtxc_dxtn else p.libtxc_dxtn_s2tc)
        ];
  };
in
{
  hardware = {
    opengl = {
      /* FIXME: See https://github.com/NixOS/nixpkgs/issues/5051#issuecomment-64393958 */
      package = makePackage pkgs;
      /* extraPackages = [pkgs.linuxPackages.virtualboxGuestAdditions]; */
    };
  };
}

I hope it'll be helpful for someone that wants to submit a proper PR.

@rehno-lindeque, I am relatively new to nixos, just trying to get it to do what I want for the second time (tried a while ago). When I simply paste your configuration snippet from above in my configuration.nix,
and call nixos-rebuild, it complains that the variable lib is undefined.
Any hint what I am doing wrong? Thanks!

@cgo: typically one adds lib to the parameter list at the start, but it's also accessible e.g. as pkgs.stdenv.lib.

BTW, setting videoDrivers = [ "virtualbox" ]; on the guest nixos config is supposed to do all this automatically.  Now I see this setting is even done by virtualisation.virtualbox.guest.x11 = true;   If this doesn't work, it's a bug somewhere in the implementation.  (I've never tested this myself.)

BTW, setting videoDrivers = [ "virtualbox" ]; on the guest nixos config is supposed to do all this automatically.

@vcunat @cgo To clarify, the above is a makes use of virtualbox's VBoxOGL.so for hardware acceleration. Without this you get software-only rendering for 3D graphics. Never-the-less I wouldn't recommend this ugly hack unless you specifically need it or want to submit a PR.

any movement on this?

This should get fixed by https://github.com/NixOS/nixpkgs/pull/60985.

This should get fixed by #60985.

I don't think so. The problem is that the 3D support in VirtualBox additions relies on programs loading VBoxOGL.so in place of libGL.so, it (presumably) doesn't play with libglvnd. The current additions package doesn't provide the symlinks from libGL.so to VBoxOGL.so which supposedly fix this issue. Actually, #60985 will make it a bit harder to get this working once LD_LIBRARY_PATH no longer contains /run/opengl-driver/lib (one could still set it manually though).
But since the Linux 3D support on VirtualBox has a questionable support status and is known to have issues, I would be reluctant to make it enabled automatically even if we support it.

If virtualbox would support libglvnd, this would have fixed it in a nice way.
But unfortunately you're right. Seems adding libglvnd and/or fixing some of the unstable 3d guest support is very much down on Oracles priority list: https://www.virtualbox.org/pipermail/vbox-dev/2017-June/014561.html
We should probably add some documentation on how to load VBoxOGL.so manually, but seems using 3d acceleration from a Linux guest currently isn't really advisable…

@flokli @ambrop72  In the last two comments it looked like the #60985 would NOT solve the issue, does this mean that it has been fixed or that this has been closed accidentally?

@zarelit as explained in the mail linked in my comment, 3d guest support is pretty much broken in virtualbox upstream too, and fixing that is far down on Oracle's priority list:

In fact, as soon as I get round to it I will disable 3D support for
Fedora guests altogether, as it is not reliable.

On top of that, it's probably not really possible to run it in NixOS either, due to it not supporting libglvnd.
I guess this would need to be fixed upstream first.
I opened https://github.com/NixOS/nixpkgs/pull/61944 which changes virtualboxGuestAdditions to not install VBoxOGL.so, with a comment on why.

@flokli  thank you for the clarification on for your work on this!

just to clarify, there's a working solution for this, but we're not going to make it easy for ideological reasons? did I understand this right?

@husnoo The reasons are mostly technical, given even upstream considers their driver to be unstable and are thinking loudly about disabling it entirely.
This could be made "easy" by passing in the VBoxOGL driver (together with the required libGL.so symlinks) through hardware.opengl.extraPackages, behind a virtualisation.virtualbox.guest.enableHardwareAcceleration option. I'd be reluctant to unconditionally enable it on virtualisation.virtualbox.guest.enable together with the other additions, given its unstable nature (we could flip the default if/when things improve).
I was just assuming there wasn't too much need in supporting that, given nobody stepped up with a PR after a possible solution was sketched in March 2017.
I'd be happy to review a PR adding this option, though ;-)


Fix #2245: Set the max file size of a submission to be configurable

Assumed 1MB = 1000000 Bytes (similar to past behaviour)

@binuri Please address the Hound style comments

@binuri Can you please add the new key for all three environments? Currently, it's only in development.rb.


Shubham/props issue fixes

Following changes completed for all the components

Default props collision issue fixed
Updated react hooks import
CSS padding class updated in menuList.css

Hi @shubhamkhunt04,
Thanks for this PR for the improvement of React app.
But these PR contains a lot of changes which is for code formatting and it's making really hard to check the real code changes in the first run.
Would you like to make new PR with only functionality improvements and leave formatting aside? then we'll definitely check and merge it to our codebase.

Hi @ravisojitra ,
It would be great and code auto-formatting prevention for future PR if you can set a prettier file in the client package.


[instrumentation http]: Missing http.host attribute on timed out requests

Please answer these questions before submitting a bug report.
What version of OpenTelemetry are you using?
v1.0.2 (issue exists in latest release as well)
What version of Node are you using?
14.18
What did you expect to see?
The http.host attribute is missing on HTTP requests that timeout or have an error.  This looks to be due to the attribute only being set when a response is successful (https://github.com/open-telemetry/opentelemetry-js/blob/922e963ecac9e427b0e45bea4fd54bf73218cf97/experimental/packages/opentelemetry-instrumentation-http/src/utils.ts#L363) even though the HTTP host header is available during the request in a HTTP Request Header (Host).
A possible fix for this would be to retrieve the http.host header from the http request rather than attaching it only during a successful response.
Thanks, love the library and your work.

Totally think we could set most of the attributes at the begining of the request since we have most of the information, do you think you could make a PR for that ?

was checking the code and looks like the port part is from response, not sure if we want to set the value by inferring from options like what we did for hostname?
https://github.com/open-telemetry/opentelemetry-js/blob/3cc40d78b482fde87cfd873a2f5cb198de4d2308/experimental/packages/opentelemetry-instrumentation-http/src/http.ts#L565-L568
and when sending the requests, we are setting the attribute net.peer.name, not sure if that can be used when request timeout?

not sure if we want to set the value by inferring from options like what we did for hostname?

I think we can, port should be the same anyway

we are setting the attribute net.peer.name, not sure if that can be used when request timeout?

From the spec the peer name is simply the hostname so i think we can

@cuichenli @zahirhoney do either of you want to be assigned this?

yeah, i can work on it

Any update?


Render children on top of canvas

Expected Behavior
To be able to render react components on top of the canvas
Actual Behavior
If I use
const App = () => {
  return (
    <ReactP5Wrapper sketch={sketch}>

Hello

    </ReactP5Wrapper>
  );
};

The canvas is simply rendered after the

Steps to Reproduce the Problem

Follow tutorial on readme
Add child components

Package Version: ^3.3.0

Can you give me an example of what you're trying to achieve with this? Have you a specific idea in mind?
The elements are added as children of the canvas wrapper and so simple css can be used to do any positioning you require, right?

I just noticed in the docs for the children section it doesn't state clearly that to put children "on top" you need to use css. I'll also update that section of the readme and add an example as soon as I can or feel free to make a PR if you'd like to take that on 👌🏻!


Unable to rip liveleak url

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.02.10. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[ ] I've verified and I assure that I'm running youtube-dl 2017.02.10
2017.02.07

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

youtube-dl -v -k -f mp4 -o '/tmp/liveleak037_1486100393.mp4'  'https://www.liveleak.com/view?i=037_1486100393'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-k', u'-f', u'mp4', u'-o', u'/tmp/liveleak037_1486100393.mp4', u'https://www.liveleak.com/view?i=037_1486100393']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.02.07
[debug] Python version 2.7.6 - Linux-3.13.0-93-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: ffmpeg N-80901-gfebc862, ffprobe N-80901-gfebc862
[debug] Proxy map: {}
[LiveLeak] 037_1486100393: Downloading webpage
ERROR: Unable to extract embed URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 696, in extract_info
    ie_result = ie.extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 369, in extract
    return self._real_extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/liveleak.py", line 107, in _real_extract
    webpage, 'embed URL')
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 672, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract embed URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.liveleak.com/view?i=037_1486100393

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
It's unclear why this Liveleak URL cannot be ripped

1 http://www.liveleak.com/view?i=4f7_1392687779 has lost.
2 liveleak has changed again. the code is new extract:
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)

        video_title = self._og_search_title(webpage).replace('LiveLeak.com -', '').strip()
        video_description = self._og_search_description(webpage)
        video_uploader = self._html_search_regex(
            r'By:.*?(\w+)</a>', webpage, 'uploader', fatal=False)
        age_limit = int_or_none(self._search_regex(
            r'you confirm that you are ([0-9]+) years and over.',
            webpage, 'age limit', default=None))
        video_thumbnail = self._og_search_thumbnail(webpage)

        formats = []
        for (_, url, _, label) in re.findall(r'source src=(["\'])(?P<url>.+?)\1.+?label=(["\'])(?P<label>.+?)\3', webpage):
            format_id = self._search_regex(r'_(\d+?)p', url, 'format_id', default=None)
            a_format = { 'format_note': label, 'url': url }
            if format_id:
                a_format['format_id'] = '%s' % format_id
            formats.append(a_format)
        if len(formats) > 0:
            self._sort_formats(formats)
            return {
                'id': video_id,
                'title': video_title,
                'description': video_description,
                'uploader': video_uploader,
                'formats': formats,
                'age_limit': age_limit,
                'thumbnail': video_thumbnail,
            }
        else:
            alt_source = self._search_regex(
                r'(file: ".*?"),', webpage, 'video URL', default=None)
            if alt_source:
                sources_raw = '[{ %s}]' % alt_source
            else:
                # Maybe an embed?

                embed_url = self._search_regex(
                    r'<iframe[^>]+src="((?:https?:)?//(?:www\.)?(?:prochan|youtube)\.com/embed[^"]+)"',
                    webpage, 'embed URL', default=None)
                if not re.match(r'https?:', embed_url):
                    mobj = re.search(r'(https?:)', url)
                    protocol = mobj.group(1) if mobj else 'http:'
                    embed_url = '%s%s' % (protocol, embed_url)
                return {
                    '_type': 'url_transparent',
                    'url': embed_url,
                    'id': video_id,
                    'title': video_title,
                    'description': video_description,
                    'uploader': video_uploader,
                    'age_limit': age_limit,
                }

Any update on this?

Ran into this as well, as I think:
~ date
Mit Mai 10 07:37:09 CEST 2017
--- 07:37:09, Mit Mai 10 ---
~ sudo youtube-dl -U
youtube-dl is up-to-date (2017.05.09)
--- 07:37:21, Mit Mai 10 ---
~ youtube-dl --verbose https://www.liveleak.com/view?i=018_1494125504
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.liveleak.com/view?i=018_1494125504']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.05.09
[debug] Python version 2.7.9 - Linux-4.9.0-0.bpo.2-amd64-x86_64-with-debian-8.7
[debug] exe versions: avconv 11.8-6, avprobe 11.8-6, rtmpdump 2.4
[debug] Proxy map: {}
[LiveLeak] 018_1494125504: Downloading webpage
ERROR: Unable to extract embed URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 760, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 433, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/liveleak.py", line 107, in _real_extract
webpage, 'embed URL')
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 782, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract embed URL; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Yes, can confirm, same error here.
ERROR: Unable to extract embed URL;
...
..\youtube_dl\extractor\common.py", line 782, in _search_regex
youtube_dl.utils.RegexNotFoundError: Unable to extract embed URL;

Liveleak is listed as a supported site, so this is a legit issue.
Okay, this already has a suitable label.

Thanks for the report and sorry for the late response. Liveleak videos will be supported again in the next version.


Examples

Hi @codekie
I'm interested in running some validation, but I struggle with how to use it.
I'm looking in the tests, but I don't fully understand how the openapi-examples-validator -s $.paths./.get.responses.200.schema  -e example.json openapi-spec.json works?
Could you a  simple "example.json" and the "openapi-spec.json"?
That would help with understanding if the schema should be defined or not?

I think that you don't need to define any "e" or "s" parameters if the examples are in the spec.

Exactly. As stated in the readme:

If an external example has to be verified, the -s and -e option has to be used.
For example:
$ openapi-examples-validator -s $.paths./.get.responses.200.schema -e example.json openapi-spec.json

This means, if you have an example that is not included in the spec, you have to pass the example file and the JSON-Path to the corresponding OpenAPI-definition within the spec.
For example:
node dist/cli.js -s $.paths./.get.responses.200.schema -e test/data/v2/external-examples-valid-example1.json test/data/v2/external-examples-schema.json


HTTP/2 bi-directional connectivity for real-time apps

WebSockets were a great tool for implementing real-time updates without the long-polling of XHR, but now HTTP/2 is a thing, we can achieve the same level of instant bi-directional communication between server and client with the HTTP protocol itself.
HTTP/2 allows for TCP connections to be kept open between server and client, allowing for the server to push new updates to the page without requiring any extra libraries.
To get the benefits of real time applications without breaking the principal of single logic authority, WebEngine should have the ability to render the changed parts of the page over HTTP/2 persistent connections, requiring very minimal JavaScript usage on the client.

This idea in this issue is to be able to have a WebEngine way of pushing updates to the view after the first render has completed. This will be possible after #307, but there is a lot of change required to the Lifecycle object to achieve this, so definitely won't be in v3.

Out of scope.


Dark mode

Full blown theming support seems overkill but simple dark mode would be nice.

This have to wait the GUI as you propose. In the GUI a comboBox could allow choose between dark / light theme, collapse references or not.

I like to your way to keep just one file (because this I did #19). So maybe create different CSS styles on the plugin folder and merge it inside the html on the final process.

Even the java script and other could be a external file added by the python code in the final HTML. This will allow some users to easy modify the HTML template adding some personal logo (example).
But most important you could keep the *.js file format indented and remove the '\n' in the moment before add at the html.

I was thinking to have dark/light mode be a toggle in html itself so that you could change it dynamically.
On splitting js and css, that is a good idea, although not because of indentation. Js that is not indented is minified anyway.

Rendering of the canvas is the biggest CPU/GPU consumer, parsing the file is nothing in comparison so I won't spend too much time on it.
Also in the large board you posted in the other issue, 99% of the html file volume is the pcbdata, which is already compact as it is. Some corners can be cut there like using shorter string keys and reducing floating point precision, but ultimately not much, unless I switch to a compact serialization protocol, something like protobuf. But that is a huge dependency to pull in, which I will only do if wins are really justified.
Anyway, we digress :) Lets keep this bug about dark mode.

When in dark mode, the board edge should go white? Just a question about the better contrast to visualize.


Put stderr output in log

Paket errors are not shown in ionide output
/cc @tforkmann

@tforkmann what was the command we did here?

Fixed in 1.5.0


Change string column default value

Remove the optional string value default by create an unlimited varchar column instead of 255 default length.
Closes #4409
Doc https://github.com/knex/documentation/pull/401

Coverage decreased (-1.1%) to 91.18% when pulling 9d416c52e1988477d63b782eb4082f3e4c0c679d on OlivierCavadenti:varchar-default-length into 2670c2f8b563bb2ddcbf292172af0c57ed02c16f on knex:master.

Still one test that fail, I don't know why (seems count order change...)
 1) Integration Tests
       postgresql | pg
         Aggregate
           support the groupBy function:

      AssertionError: expected [ { count: '2' }, { count: '6' } ] to deeply equal [ { count: '6' }, { count: '2' } ]
      + expected - actual

       [
         {
      -    "count": "2"
      +    "count": "6"
         }
         {
      -    "count": "6"
      +    "count": "2"
         }
       ]


use label variables in metrics

Pre-requisites

[X] I have double-checked my configuration
[X] I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
[X] I have searched existing issues and could not find a match for this bug
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?
https://cloud-native.slack.com/archives/C01QW9QSSSK/p1717447990226269
I tried using "{{=workflow.labels['workflows.argoproj.io/cron-workflow']}}" as a metric label inside a workflow created by a cron and received error MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/cron-workflow']"
Version
latest
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
  name: foo
spec:
  ...
  workflowSpec:
    metrics:
      prometheus:
        - name: exec_count
          help: "Count of workflow execution by result status - workflow level"
          labels:
            - key: name
              value: "{{=workflow.labels['workflows.argoproj.io/cron-workflow']}}"
            - key: status
              value: "{{workflow.status}}"
          counter:
            value: "1"

Logs from the workflow controller
time="2024-06-04T18:05:19.505Z" level=info msg="Processing workflow" Phase= ResourceVersion=50667015 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.512Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=0 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.512Z" level=info msg="Updated phase  -> Running" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.512Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.513Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.513Z" level=info msg="DAG node oltp-backup-to-olap-ingest-xnlls initialized Running" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.513Z" level=info msg="All of node oltp-backup-to-olap-ingest-xnlls.backup dependencies [] completed" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.513Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.513Z" level=info msg="Pod node oltp-backup-to-olap-ingest-xnlls-411105871 initialized Pending" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.555Z" level=info msg="Created pod: oltp-backup-to-olap-ingest-xnlls.backup (oltp-backup-to-olap-ingest-xnlls--411105871)" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.555Z" level=info msg="All of node oltp-backup-to-olap-ingest-xnlls.delete-old-backups dependencies [] completed" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.555Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.555Z" level=info msg="Pod node oltp-backup-to-olap-ingest-xnlls-2719863104 initialized Pending" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.593Z" level=info msg="Created pod: oltp-backup-to-olap-ingest-xnlls.delete-old-backups (oltp-backup-to-olap-ingest-xnlls--2719863104)" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.593Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.593Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:19.593Z" level=info msg="Workflow to be dehydrated" Workflow Size=3567
time="2024-06-04T18:05:19.610Z" level=info msg="Workflow update successful" namespace=cron phase=Running resourceVersion=50667028 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:23.201Z" level=info msg="Processing cron/oltp-backup-to-olap-ingest" cronWorkflow=cron/oltp-backup-to-olap-ingest
time="2024-06-04T18:05:23.202Z" level=info msg="CronWorkflow cron/oltp-backup-to-olap-ingest added" cronWorkflow=cron/oltp-backup-to-olap-ingest
time="2024-06-04T18:05:29.556Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=50667028 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=0 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg="node changed" namespace=cron new.message="Unschedulable: 0/2 nodes are available: 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.." new.phase=Pending new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 old.message= old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg="node changed" namespace=cron new.message="Unschedulable: 0/2 nodes are available: 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.." new.phase=Pending new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 old.message= old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:29.557Z" level=info msg="Workflow to be dehydrated" Workflow Size=4189
time="2024-06-04T18:05:29.580Z" level=info msg="Workflow update successful" namespace=cron phase=Running resourceVersion=50667105 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.576Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=50667105 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.577Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=0 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.577Z" level=info msg="node unchanged" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.577Z" level=info msg="node unchanged" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.578Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:05:39.578Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.289Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=50667105 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.290Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=2 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.290Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.290Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.290Z" level=info msg="node changed" namespace=cron new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 old.message="Unschedulable: 0/2 nodes are available: 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.." old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.290Z" level=info msg="node changed" namespace=cron new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 old.message="Unschedulable: 0/2 nodes are available: 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.." old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.291Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.291Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:06.291Z" level=info msg="Workflow to be dehydrated" Workflow Size=4052
time="2024-06-04T18:06:06.320Z" level=info msg="Workflow update successful" namespace=cron phase=Running resourceVersion=50667577 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.762Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=50667577 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.763Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=2 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.763Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.763Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.763Z" level=info msg="node changed" namespace=cron new.message= new.phase=Succeeded new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.763Z" level=info msg="node changed" namespace=cron new.message= new.phase=Running new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.764Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.764Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:22.764Z" level=info msg="Workflow to be dehydrated" Workflow Size=4356
time="2024-06-04T18:06:22.785Z" level=info msg="Workflow update successful" namespace=cron phase=Running resourceVersion=50667715 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.786Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=50667715 namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.786Z" level=info msg="Task-result reconciliation" namespace=cron numObjs=2 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.786Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.786Z" level=info msg="task-result changed" namespace=cron nodeID=oltp-backup-to-olap-ingest-xnlls-2719863104 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="Pod failed: Error (exit code 254)" displayName=backup namespace=cron pod=oltp-backup-to-olap-ingest-xnlls--411105871 templateName= workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="node changed" namespace=cron new.message="Error (exit code 254)" new.phase=Failed new.progress=0/1 nodeID=oltp-backup-to-olap-ingest-xnlls-411105871 old.message= old.phase=Running old.progress=0/1 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="Outbound nodes of oltp-backup-to-olap-ingest-xnlls set to [oltp-backup-to-olap-ingest-xnlls-411105871 oltp-backup-to-olap-ingest-xnlls-2719863104]" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="node oltp-backup-to-olap-ingest-xnlls phase Running -> Failed" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="node oltp-backup-to-olap-ingest-xnlls finished: 2024-06-04 18:06:32.78750981 +0000 UTC" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="TaskSet Reconciliation" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg=reconcileAgentPod namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="Updated phase Running -> Failed" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="Marking workflow completed" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=error msg="unable to substitute parameters for metric 'exec_count': failed to evaluate expression \"workflow.labels['workflows.argoproj.io/cron-workflow']\"" namespace=cron workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:32.787Z" level=info msg="Workflow to be dehydrated" Workflow Size=4724
time="2024-06-04T18:06:32.793Z" level=info msg="cleaning up pod" action=deletePod key=cron/oltp-backup-to-olap-ingest-xnlls-1340600742-agent/deletePod
time="2024-06-04T18:06:32.802Z" level=info msg="Queueing Failed workflow cron/oltp-backup-to-olap-ingest-xnlls for delete in 24h0m0s due to TTL"
time="2024-06-04T18:06:32.803Z" level=info msg="Workflow update successful" namespace=cron phase=Failed resourceVersion=50667795 workflow=oltp-backup-to-olap-ingest-xnlls
time="2024-06-04T18:06:33.222Z" level=info msg="Deleted Workflow 'oltp-backup-to-olap-ingest-gt97s' due to CronWorkflow 'oltp-backup-to-olap-ingest' history limit" namespace=cron workflow=oltp-backup-to-olap-ingest
time="2024-06-04T18:06:33.232Z" level=info msg="Processing cron/oltp-backup-to-olap-ingest" cronWorkflow=cron/oltp-backup-to-olap-ingest
time="2024-06-04T18:06:33.233Z" level=info msg="CronWorkflow cron/oltp-backup-to-olap-ingest added" cronWorkflow=cron/oltp-backup-to-olap-ingest

Logs from in your workflow's wait container
time="2024-06-04T18:06:03.887Z" level=info msg="Starting Workflow Executor" version=v3.5.6
time="2024-06-04T18:06:03.891Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2024-06-04T18:06:03.891Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=cron podName=oltp-backup-to-olap-ingest-xnlls--411105871 templateName= version="&Version{Version:v3.5.6,BuildDate:2024-04-19T20:54:43Z,GitCommit:555030053825dd61689a086cb3c2da329419325a,GitTag:v3.5.6,GitTreeState:clean,GoVersion:go1.21.9,Compiler:gc,Platform:linux/amd64,}"
time="2024-06-04T18:06:03.912Z" level=info msg="Starting deadline monitor"
time="2024-06-04T18:06:21.921Z" level=info msg="Main container completed" error="<nil>"
time="2024-06-04T18:06:21.921Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2024-06-04T18:06:21.921Z" level=info msg="No output parameters"
time="2024-06-04T18:06:21.921Z" level=info msg="No output artifacts"
time="2024-06-04T18:06:21.922Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: oltp-backup-to-olap-ingest-xnlls/oltp-backup-to-olap-ingest-xnlls--411105871/main.log"
time="2024-06-04T18:06:21.931Z" level=info msg="Creating minio client using AWS SDK credentials"
time="2024-06-04T18:06:21.995Z" level=info msg="Saving file to s3" bucket=REDACTED endpoint=s3.amazonaws.com key=oltp-backup-to-olap-ingest-xnlls/oltp-backup-to-olap-ingest-xnlls--411105871/main.log path=/tmp/argo/outputs/logs/main.log
time="2024-06-04T18:06:22.080Z" level=info msg="Save artifact" artifactName=main-logs duration=158.513309ms error="<nil>" key=oltp-backup-to-olap-ingest-xnlls/oltp-backup-to-olap-ingest-xnlls--411105871/main.log
time="2024-06-04T18:06:22.080Z" level=info msg="not deleting local artifact" localArtPath=/tmp/argo/outputs/logs/main.log
time="2024-06-04T18:06:22.081Z" level=info msg="Successfully saved file: /tmp/argo/outputs/logs/main.log"
time="2024-06-04T18:06:22.100Z" level=info msg="Alloc=9313 TotalAlloc=17333 Sys=23141 NumGC=4 Goroutines=12"

Follow-up from this Slack thread.

I tried using "{{workflow.labels.workflows.argoproj.io/cron-workflow}}"

received error MetricsError: unable to substitute parameters for metric 'exec_count': failed to resolve {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}, unable to substitute parameters for metric 'exec_duration': failed to resolve {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}

            value: "{{workflow.labels.workflows.argoproj.io/creator}}"

The description here seems to have copy+pasted a few different ones, so it's not exactly clear which variants you tried and which you didn't. All 3 of the above are different.
dot syntax works for labels other than cron-workflow
In the thread, @tico24 wrote that this works:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: wibblewobble-
spec:
  metrics:
    prometheus:
      - name: exec_count
        help: "Count of workflow execution by result status - workflow level"
        labels:
          - key: name
            value: "{{workflow.labels.workflows.argoproj.io/creator}}"
          - key: status
            value: "{{workflow.status}}"
        counter:
          value: "1"

but not when using as a workflowSpec of a CronWorkflow. Tim's hypothesis there was: "My assumption is that the metrics parameters are unpacked before the cron label is applied to the running workflow, so it can't find the label - hence the error. I would agree that this is an issue."
expr brackets syntax
I suggested using the expr style brackets, i.e. {{=workflow.labels['workflows.argoproj.io/cron-workflow']}}. Per the error it seems like that didn't work either, although not entirely clear due to copy+paste errors as per above
I'm also not sure if brackets are potentially affected by https://github.com/argoproj/argo-workflows/issues/11549

ya im all over the place, working on cleaning it up... sorry

the description has been updated with a new run ensuring consistency between all the parts

did some testing

{{workflow.labels.workflows.argoproj.io/creator}}: worked
{{workflow.labels.workflows.argoproj.io/cron-workflow}}: worked
{{=workflow.labels.workflows.argoproj.io/creator}}: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression: unknown name creator (1:39) | workflow.labels.workflows.argoproj.io/creator | ......................................^
{{=workflow.labels.workflows.argoproj.io/cron-workflow}}: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression: unknown name cron (1:39) | workflow.labels.workflows.argoproj.io/cron-workflow | ......................................^
{{=workflow.labels['workflows.argoproj.io/creator']}}: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/creator']"
{{=workflow.labels['workflows.argoproj.io/cron-workflow']}}: MetricsError: unable to substitute parameters for metric 'exec_count': failed to evaluate expression "workflow.labels['workflows.argoproj.io/cron-workflow']"

the middle two shouldn't work, so those are correct. the last two I would expect to work, so are buggy.
Regarding the first two -- those worked on a CronWorkflow as well? Since Tim said it didn't work when he tried a Workflow generated by a CronWorkflow

all of my testing was done by setting the metrics in the controller config workflow defaults, creating a cronworkflow, and submitting it through the UI

I have slightly similar issue to this one and https://github.com/argoproj/argo-workflows/issues/3989 when using a workflowTemplate and a custom label in it.

Although the label monitoring is set in the workflowTemplate, it is not resolved. However, if I manually add this label in the UI as in the image below, it works.

It seems that the labels defined in the manifests are not being considered.

{{workflow.labels.workflows.argoproj.io/cron-workflow}}: worked

In my case this didn't work, Argo v3.6.0-rc2

For those looking for immediate solutions, the workaround which works for me is to use workflow.labels.json along with fromJSON() from expr.
value: '{{= fromJSON(workflow.labels.json)["example.com/environment-name"] }}'


[NTR] 'CD24-positive, CD-133-positive, vimentin-positive proximal tubular cell'

Preferred term label
CD24-positive, CD-133-positive, vimentin-positive proximal tubular cell
Synonyms
(broad): CD24-positive tubular cell PMID:23124355
(broad): CD133+CD24+ tubular cell PMID:23124355
(broad): CD133+CD24+CD106- tubular cell https://doi.org/10.1002/stem.1130
Definition (free text, with reference(s), please. PubMed ID format is PMID:XXXXXX)
A CD24-positive, CD-133-positive, vimentin-positive cell found scattered throughout a renal proximal tubule and that may participate in tubular regeneration. Compared to other proximal tubular cell types, this cell contains less cytoplasm, fewer mitochondria and no brush border.
PMID:23124355
PMID:21281815
Parent cell type term
CL:0002306 'epithelial cell of proximal tubule'
expresses some 'signal transducer CD24'
expresses some prominin-1
expresses some vimentin (needs imported: http://purl.obolibrary.org/obo/PR_000017298)
What is the anatomical structure that the cell is a part of?
‘part of’ some ‘proximal tubular epithelium’
Additional notes or concerns
Comment: After acute tubular injury, this cell type becomes more numerous. In healthy rat kidneys, this cell is not detectable, whereas after unilateral ureteral obstruction (renal injury), it may appear de novo, arguing against the notion that this cell type represents a pre-existing progenitor cell population. In at least one study, data indicate that this cell type represents transiently dedifferentiated tubular cells involved in regeneration.
PMID:23124355

Are these rat markers?

Are these rat markers?

The papers mention human and rat subjects.
The PR terms refer to the human gene product or "1:1 orthodox thereof".

Note: adding this term is paused until the new GO release is available.

CL:4030038 'CD24-positive, CD-133-positive, vimentin-positive proximal tubular cell' has been to CL and should appear in the next release.


Update sbt-scalafmt to 2.4.6

Updates org.scalameta:sbt-scalafmt from 2.4.5 to 2.4.6.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "sbt-scalafmt" } ]

labels: sbt-plugin-update, early-semver-patch, semver-spec-patch

Codecov Report

Merging #690 (2c7a781) into main (a3a840c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #690   +/-   ##
=======================================
  Coverage   56.28%   56.28%
=======================================
  Files          63       63
  Lines        1702     1702
  Branches       45       45
=======================================
  Hits          958      958
  Misses        744      744

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3a840c...2c7a781. Read the comment docs.


[Feature Request] Update to branch / fork branch

In Hestia we have to option to install any branch via:
v-update-sys-hestia-git fork branch to install/test a  branch (of a fork).   It would be useful to have this option available for testing...
Is your feature request related to a problem? Please describe.
Option to install test branch  instead preview or release.
https://github.com/FOSSBilling/FOSSBilling/pull/418
Works great for preview version but it would be awesome to also allow other branches so easier test can be done.
Describe the solution you'd like
Provide Fork + branch and let fossbilling install it from source
Describe alternatives you've considered
Tried to write something in bash:
#!/bin/bash

username=$1
branch=$2
if [ -z "$branch" ]; then
 branch="main"
fi

FB_ARCHIVE_LINK='https://github.com/'$username'/FOSSBilling/archive/'$branch'.tar.gz'

wget $FB_ARCHIVE_LINK
tar -xvf  $branch.tar.gz

cd FOSSBilling-$branch

npm install
gulp
cd src
composer install
rm -fr install
cd ..
cp -rlf ./src/ ../../
cd ..
rm -fr ../src/bb-data/cache/*
rm -fr FOSSBilling-$branch
rm -fr $branch.tar.gz
ls

Does work
Additional context
Add any other context or screenshots about the feature request here.

So, I'd like to hear other people provide their input on this, but imo there's no reason to build this feature into FOSSBilling.
It's added complexity for little benefit. To properly automate that, we'd need it to download the source code, install composer packages, run NPM and Gulp, clear the cache, and run the foss-update.php file.
All while having no way to ensure that FOSSBilling will have the correct permissions on the server to actually exec the functions, that the functions will work on the OS it's being hosted on, and that all required packages are installed.
Both are issues that Hestia doesn't need to deal with nearly as much, as it only supports Debian / Ubuntu. It's also installed system-level meaning that you can ensure all of the require packages and software is installed and that the permissions are all sorted out.
If you want to write a bash script to help aid the process, I'm fine with including it as a stand-alone script in the github repo, but trying to add it to the main software is a bit much

No point in keeping this open if we aren't going to implement it


Gmaps revival

Let's get back GMaps!
this is what changed:

backend:

basemaps are not static anymore, they depend on the user. Basically this allow us to set different basemaps depending if the user has google maps enabled or not. Check user.rb changes https://github.com/CartoDB/cartodb/pull/3337/files#diff-4676c008b11a5480d73d4a6de01e45b9R2352
map creation: before app_config.yml configuration was being used, that was changed to be the default map for the user. Check map.rb and table.rb to see the changes. Not needed keys have been removed (so they could be removed from deploy cc @CartoDB/systems )

@Kartones I think there are some things to change in the new map.rb

systems:

basemap configuration needs to be changed: https://github.com/CartoDB/cartodb-platform/issues/506

frontend

enabled google maps again, torque, edition and so on is working (@fdans worked on it)
google maps baselayer support in the dropdown
lots of views changed to include google maps libray when needed

To make it work:
In order to have google maps you need to:

Add the following to the basemaps section in app_config.yml

GMaps:
      roadmap:
        name: 'GMaps Roadmap'
        maxZoom: 40
        minZoom: 0
        baseType: 'roadmap'
        className: 'googlemaps'
      hybrid:
        name: 'GMaps Hybrid'
        maxZoom: 40
        minZoom: 0
        baseType: 'hybrid'
        className: 'googlemaps'
      satellite:
        name: 'GMaps Satellite'
        maxZoom: 40
        minZoom: 0
        baseType: 'satellite'
        className: 'googlemaps'
      gray_roadmap:
        name: 'GMaps Gray Roadmap'
        maxZoom: 40
        minZoom: 0
        baseType: 'gray_roadmap'
        className: 'googlemaps'
        style: '[{ "stylers": [ { "saturation": -100 } ] },{ "featureType": "water", "stylers": [ { "gamma": 1.67 }, { "lightness": 27 } ] },{ "elementType": "geometry", "stylers": [ { "gamma": 1.31 }, { "lightness": 12 } ] },{ "featureType": "administrative", "elementType": "labels", "stylers": [ { "lightness": 51 }, { "gamma": 0.94 } ] },{ },{ "featureType": "road", "elementType": "labels", "stylers": [ { "lightness": 57 } ] },{ "featureType": "poi", "elementType": "labels", "stylers": [ { "lightness": 42 } ] }]'

Enable feature flag google_maps.

Fixes #3289.

@kartones @xavijam @viddo please review

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

retest this please

retest this please

retest this please

Frontend tests were OK :+1: (details)

@javisantana It seems everything is working fine.


Make Travis fail non-formatted builds

It feels like the cargo fmt setup we have is a bit half-assed: on the one hand, we test for it in CI; on the other, stuff regularly gets merged without cargo formatting. I think #5355 is the third time this week that I had to clean up other people's stuff. It feels to me like we should either enforce it or let it go; having contributors deal with other people's formatting issues is not so nice.

We had it once, but then removed it: https://github.com/rust-lang/cargo/pull/5278

it creates more friction for new contributors. It would be awesome if we could share some kind of pre-commit hook for this, but they are not sharable

it complicate back porting, because rustfmt slightly changes formatting with new versions.

So does (2) mean we don't want to do much reformatting at all? It would be nice maybe if bors/homu gave friendly feedback about it in PRs that don't conform.

The plan is to just run rustmt manually once in a while.

here's another idea (for this and other non-functional aspects of code, like performance) is, instead of blocking a PR, check on master only or nightly, and raise an alert (e.g a chat message or open an issue).

rustfmt slightly changes formatting with new versions

another option is to pin the rust version in CI instead of using a moving channel, like https://github.com/assert-rs/assert_cli/blob/d3daabbcef3bb1323af5cf71e40a0d4ee16f7647/.travis.yml#L17-L22. wdyt?

Ashley shared a post on how to enforce style in CI: https://medium.com/@ag_dubs/enforcing-style-in-ci-for-rust-projects-18f6b09ec69d
With Rustfmt reaching 1.0 RC1 (https://www.ncameron.org/blog/rustfmt-1-rc/), do we want to experiment with this at this time?
Another possible option is look at integrating with GitHub Checks (https://blog.github.com/2018-05-07-introducing-checks-api/)? So PRs have the info about where style has deviated, but without having it fail CI.

https://crate-ci.github.io/ (which seems like a great resource) suggests in https://crate-ci.github.io/pr/rustfmt.html#specifying-your-style to lock down the project's style with a .rustfmt.toml. Would that alleviate back-porting concerns?
(By the way, the modern command to run is rustfmt --print-config default .rustfmt.toml)

I think we should just close this.  For reason mentioned above there's no interest to do this (to the best of my knowledge).

Always happy to change this decision, perhaps in the face of Rust 2018 rustfmt 😄


Clean up and improve coding-style of CM1 GPU code

This PR mainly:

Add default(present) clause to acc parallel and remove the private clause.
Add missing gang or vector clause wherever applicable.
Change the coding style of some OpenACC directives to improve readability.

I found a few GPU bugs during the code changes but I would report them in a separate PR.
I merged in the latest gpu-opt branch (https://github.com/george-bryan/CM1/commit/e5479c45e5281efcb126289e593fb040be65b619) and performed CPU (intel compiler), OpenACC and MPI+OpenACC runs. The verification output is summarized below:
CPU vs. OpenACC:
Identical variables: 31
variables with red_diff > 1.e-6: 48
variables with red_diff <= 1.e-6: 4
four variables with largest error: KSVMAX, KSHMAX, KHVMAX, KHHMAX
CPU vs. MPI+OpenACC:
Identical variables: 31
variables with red_diff > 1.e-6: 49
variables with red_diff <= 1.e-6: 3
four variables with largest error: KSVMAX, KSHMAX, TKEMAX, KHVMAX

@sjsprecious,   It would probably be a good idea to include a second verification plot in this PR.  The one that you included previously was before the merge of the getgamk bug fix.

@sjsprecious I was a bit surprised to see a large number of changes that removed the B4B regions. They all look fine though.

Sorry about the large amount of elimination of BFB regions and the inconvenience it caused for reading. I think we eventually need to remove those regions and they are not used in our tests anyway. Thus I prefer to remove them while I am going through the codes.

@sjsprecious, It would probably be a good idea to include a second verification plot in this PR. The one that you included previously was before the merge of the getgamk bug fix.

@johnmauff, thanks for your suggestions. I merged your changes of fixing the getgamk bug and reran the verification tests (with tqnudge = .true.). The new results are listed below:
CPU vs. OpenACC:
Identical variables: 29
variables with red_diff > 1.e-6: 48
variables with red_diff <= 1.e-6: 6
four variables with largest error: KSVMAX, KSHMAX, TMW, TKEMAX
CPU vs. MPI+OpenACC:
Identical variables: 26
variables with red_diff > 1.e-6: 57
variables with red_diff <= 1.e-6: 0
four variables with largest error: STHPMX, THPMAX, TMW, PPMAX
The results are identical with those in https://github.com/george-bryan/CM1/pull/56.
The new plot looks like:


Could Bitkit show the EUR equivalent

From a user

Please see a proposed solution in our internal Slack channel.

https://synonymworkspace.slack.com/archives/C01H106JRQB/p1724232490247799

added to prototype v51

This was implemented in https://github.com/synonymdev/bitkit/pull/2337


Integrate DAB with Stoic Wallet

Link to forum
https://forum.dfinity.org/t/icdevs-org-bounty-integrate-dab-with-stoic-wallet-6-100-icp/9824

[x] Have stoic wallet query getAllgetUserTokens in DAB and append those items to the ones that stoic already finds search for duplicates and remove them.
[x] Keep track of if the NFT is in DAB and use DAB’s transfer function to call transfer if it is a DAB listed token.
[x] Use DAB’s details function to get info and show it when a user clicks on an NFT in stoic wallet and wants to see details

Open questions:
Stoic collections has the following properties but DAB doesn't, what should we default to in these cases?
   comaddress ?
   commission ?
   route ?

An update on the progress so far.
Query DAB Nfts is working fine.
A little issue with the thumbnails in DAB, since the returned NFTS could be of many types we need to check first the content_type and based on the that we should know how to render, currently I am falling back to the collection icon.
A link to discussion on DAB discord
Transfer function:
I am a bit blocked on the transfer function since there's a mismatch between agent-js versions used by DAB-js and stoic that causing an error.
Stoic is using 0.8.9 and we need to upgrade to at least version 0.9.3 which has a breaking changes.
I will be spending sometime with the upgrade but the changes could grow out of proportion.
It would have been nice if there's some tests that can tell if there something was broken.

Ready for review

Thanks will review and test locally, thanks for this!

Open questions: Stoic collections has the following properties but DAB doesn't, what should we default to in these cases?
   comaddress ?
   commission ?
   route ?

We will pull from a second API (entrepot specific) to pull these in - these are entrepot specific/marketplace variables, but soon will be stored in-canister

Tried to run locally and got the following, please advise:
./src/components/SendNFTForm.js
Module not found: Can't resolve '@dfinity/principal' in 'C:\Users\Stephen Andrews\Documents\GitHub\stoic-wallet\src\components'

/components/SendNFTForm.js

Can you delete your node_modules and reinstall again?
I suspect there's a conflict in @dfinity/principal versions.

Probably you will also need to follow this in order to install DAB
https://docs.dab.ooo/nft-list/getting-started/#0-preparing-your-environment
To pull and install from @Psychedelic via the NPM CLI, you'll need:

A Github account
A Github personal access token (you can create a personal acess token here)
The personal access token with the correct scopes, repo and read:packages to be granted access to the GitHub Package Registry.
Authentication via npm login, using your Github email for the username and the personal access token as your password:

Once you have those ready, run:
npm login --registry=https://npm.pkg.github.com --scope=@Psychedelic

Merged, built and tested locally - some issues:

NFTs are displaying the wrong image - seems to be the same image for all NFTs of the same collection. Clicking the image does go to the right image though
For some reason it is showing the wrong NFT count - I'm seeing the same number of NFTs for every wallet, which is wrong

Merged, built and tested locally - some issues:

NFTs are displaying the wrong image - seems to be the same image for all NFTs of the same collection. Clicking the image does go to the right image though
For some reason it is showing the wrong NFT count - I'm seeing the same number of NFTs for every wallet, which is wrong

For the first issue this happens because DAB NFTS can be of types other than image, in order to get the correct image we will have to make a http request for each image to figure out how to display.
@letmejustputthishere has a opened a PR  that is targeting the same issue.
We can merge his PR after merging this one or I can integrate his PR in this one what do you think.
For the second issue how can we debug this? I only tested on my personal wallet but not on multiple ones.
But let me try to test it again.

I have fixed the thumbnail display issue, now it's showing the actual image for each NFT.

Merged, built and tested locally - some issues:

For some reason it is showing the wrong NFT count - I'm seeing the same number of NFTs for every wallet, which is wrong

DAB connects NFTS to the principal id not sub-account ids so for each principal you will see the same NFT count for each subaccount, maybe that's what you meant or did you mean a different identity has the same NFT count?

I think he meant that DAB starts indexing from 0 and stoic from 1


Sherlock service database port

Sherlock service needs configuring to use the correct database port (3306/9001) depending on whether Gelera is in use or not.

Fixed.


feat(resolver): Add CLI option to resolve minimal version dependencies

Fixes #4100
Test cases are still missing. We need to come up with a plan what cases we want to cover.
Thanks a lot to @Eh2406 for very helpful instructions to kick this off.

Thanks for working on this! Let me know when this is ready for review, or if there is anything else I can do to help?
One complication is that @alexcrichton has recently https://github.com/rust-lang/cargo/pull/5180 add code that uses the sort to preserve the lock file. I think we should always have in_previous first; then only the .summary.version() should be subject to this switch. @alexcrichton what do you think?
Also where is @rust-highfive to set:
r? @alexcrichton

I think we should always have in_previous first; then only the .summary.version() should be subject to this switch. @alexcrichton what do you think?

Agreed!

Added a test case, this should now be ready for review.
I tried to follow your suggestion to only apply the switch when in_previous has been checked:
ret.sort_unstable_by(|a, b| {
    let a_in_previous = self.try_to_use.contains(a.summary.package_id());
    let b_in_previous = self.try_to_use.contains(b.summary.package_id());
    let previous_cmp = a_in_previous.cmp(&b_in_previous);
    match previous_cmp {
        Ordering::Equal => {
            let cmp = a.summary.version().cmp(&b.summary.version());
            match self.minimal_versions {
                true => cmp.reverse(),
                false => cmp,
            }
        },
        _ => previous_cmp,
    }
});

But that does not work and then my test case fails. Maybe I need to specify a more elaborate test case that would then drive around this in_previous check? Or maybe we always need to reverse() as it is in the PR changes right now?

Thanks for working on this. That looks like it should work. I wonder why the test fails with that code. I can experiment with it when I get home.

Got it the old code was a.cmp(&b).reverse()! So that means the new code needs a .reverse() at the end asswell. (or any number of equivalent ways of writing it, if it makes things clearer. For example the .then_with( method may be easier to read, and match on a bool is somtimes cleaner as an if.

@bors: r+
This looks great, thanks @klausi!

:pushpin: Commit 5907a8b has been approved by alexcrichton

@bors: r-
oh? How so? (I think I missed that...)

https://github.com/rust-lang/cargo/pull/5200#issuecomment-374771936 suggested how to handle it correctly, but it was failing the test. So it was Not committed. https://github.com/rust-lang/cargo/pull/5200#issuecomment-374794220 figured out how to get it working, but that has not yet been incorporated.
After that is fixed there are some nits, working on a revue now, then we will be good.

Thanks a lot @Eh2406 , again :-)
Very good feedback, I think I covered your points now.

And now with integration test. tests/testsuite/generate_lockfile.rs might not be the best place to put it, any better suggestions?

Moved the integration test right after the resolver test, I guess it makes sense to just have them together there.

This looks good to me, r+! Thank you @klausi.

@bors: delegate=Eh2406 r=Eh2406

:v: @Eh2406 can now approve this pull request

:pushpin: Commit a38184f has been approved by Eh2406

:hourglass: Testing commit a38184ffe95f93481c3a02abd5b3f06c87434c32 with merge bcd0300f0e2b572e81fa17e88dcb927aa95d6b4e...

:sunny: Test successful - status-appveyor, status-travis
Approved by: Eh2406
Pushing bcd0300f0e2b572e81fa17e88dcb927aa95d6b4e to master...


Getting the value(object/index/properties) of the PlotVLine I just dragged?

How do I get the value(object/index/properties) of the PlotVLine I just dragged, or am presently dragging in either realtime for MouseDown, or on MouseUp?
It would be nice to know what it's exact location for the xAxis (yet again). I can loop through all of the PlotLines and get the values, but knowing that actual line and addressing it directly would be better.
I have added the following to my code, to know when the mouse is up on the control, but, it doesn't give me much at all.
  public Form1()
  {
        InitializeComponent();

        this.formsPlot1.MouseUp += new System.Windows.Forms.MouseEventHandler(this.formsPlot1_MouseUp);
}

and
private void formsPlot1_MouseUp(object sender, MouseEventArgs e)
{
      Console.WriteLine("MouseUp");
      Console.WriteLine(sender.ToString());
      Console.WriteLine(e.ToString());

     // This would be ideal, though it will fail at this present time.
      var X = sender.PlotVLine.X;

    // ... update values based on X etc
}

This next bit, I realise this may be the wrong, or not the original intended way of doing this, but I couldn't find any other way... it seems to work for individual elements for a short search, but not at all good for real-time. Basically, it you need to find the index of a known 'label'.
formsPlot1.plt.PlotVLine(label: "green", lineWidth: 3, x: position, draggable: true, dragLimitLower: 0, dragLimitUpper: 100000);
String plotname = "";

int plottablescount = formsPlot1.plt.GetSettings(false).plottables.Count;
for (int i = 0; i < plottablescount; i++)
{

Console.WriteLine(formsPlot1.plt.GetSettings(false).plottables[i].ToString());      //  PlottableVLine (green) at X=516587
Console.WriteLine(formsPlot1.plt.GetSettings(false).plottables[i].GetLegendItems());//  ScottPlot.Config.LegendItem[]
Console.WriteLine(formsPlot1.plt.GetSettings(false).plottables[i].GetLimits());     //  x1=516587.000, x2=516587.000, y1=NaN, y2=NaN
Console.WriteLine(formsPlot1.plt.GetSettings(false).plottables[i].GetType());       //  ScottPlot.PlottableVLine
Console.WriteLine(formsPlot1.plt.GetSettings(false).plottables[i].GetHashCode());   //  2383799
var stringlabel = formsPlot1.plt.GetSettings(false).plottables[i].ToString();       //  PlottableVLine (green) at X=516587

Console.WriteLine(Regex.Match(stringlabel, @"\(([^)]*)\)").Groups[1].Value); // This will extract the name of the moveable plotline

plotname = Regex.Match(stringlabel, @"\(([^)]*)\)").Groups[1].Value;

if (plotname == "green")
{
    // do stuff as the 'green' label was found
}

}

Thanks again.

Hi @vrdriver,
Just save your PlottableVLine on Forms level:
public Form1
{
PlottableVLine vLine;

public someButtonClick(...)
{
 ...
  vline = formsPlot1.plt.PlotVLine(label: "green", lineWidth: 3, x: position, draggable: true, dragLimitLower: 0, dragLimitUpper:
  100000);
...
}
}

And use it whenever and wherever you need it.
ScottPlot.WinForms control have MouseDragPlottable event, you can subscribe to it and check value of your stored vLine.
formsPlot.MouseDragPlottable += (s, e) => SomeCaption.Text = vLine.X.ToString();

You don't need iterate over plottables. GetSettings() for debug and experiments porpouse only. If you have a problem that can only be solved using GetSettings, this is a reason to expand the existing API.

If this is not possible in the current version then, not to worry, as I may be able to use a different system altogether. I was just wondering if it was possible. After writing this question, I noticed that it's probably marked for version 5.

Thanks @StendProg. I'll take a look at this.

@StendProg . Thanks for helping with guiding me the right direction. It has helped a lot.
As I have multiple vplots, I've redirected the entire event, which keeps that real-time thing happening and I'm then able to get all of them separately and detect changes.
PlottableVLine vLine_cue;
PlottableVLine vLine_overlap;
PlottableVLine vLine_intro;
// .. etc

public Form1()
{
    InitializeComponent();
    this.formsPlot1.MouseDragPlottable += (s, e) => { formsPlot1_MouseDragPlottable(s, e); };
}

private void formsPlot1_MouseDragPlottable(object s, EventArgs e)
{
    // This is for detecting the plot lines that have been changed in ScottPlot

    // check if each link has changed
    if (plot_Meta_Cue_sample != vLine_cue.position)
    {
        textBox_vLine_cue.Text = vLine_cue.position.ToString(); // with added formating later
        plot_Meta_Cue_sample = vLine_cue.position;
        button_Save.BackColor = Color.LightPink;
    }

    // repeat the above statement for the rest of them...
    if (plot_Meta_Overlap_sample != vLine_cue.position)
    {
        textBox_vLine_overlap.Text = vLine_overlap.position.ToString(); // with added formating later
        plot_Meta_Overlap_sample = vLine_overlap.position;
        button_Save.BackColor = Color.LightPink;
    }

    // etc

}

private void Form1_Load(object sender, EventArgs e)
{

    double position = 100;

    vLine_cue = formsPlot1.plt.PlotVLine(label: "vLine_cue", lineWidth: 3, x: position, draggable: true, dragLimitLower: 0, dragLimitUpper:   100000);
    vLine_overlap = formsPlot1.plt.PlotVLine(label: "vLine_overlap", lineWidth: 3, x: position+10, draggable: true, dragLimitLower: 0, dragLimitUpper: 100000);
    vLine_intro = formsPlot1.plt.PlotVLine(label: "vLine_intro", lineWidth: 3, x: position+20, draggable: true, dragLimitLower: 0, dragLimitUpper: 100000);
}

I've included this code for others to discover if required.

Before I close this question, I have just one more.
For context, I'm needing to move the playhead of an audio track when the song is playing.
I use vlinePlayhead.position = song.samplesposition; to reposition the vlineplayhead.
Now the problem I have is that it appears to refresh the entire plot.
I didn't see it in any examples, but is there a way to only refresh that vlineplayhead, and leave the rest of the plot untouched?
I can use plt.render to do the entire thing,  or I can do
plt.remove(vlineplayhead);
Vlineplayhead.position = song.samplesposition;
plt.add(vlineplayhead);

But this seems a little cumbersome and still ends up with the same result.

This is the base concept of ScottPlot. Full plot rerender from scratch on each render call without any caching. There is nothing to do with that. This concept work fine, because ScottPlot have fast render time in most cases.
For signal with 40 million samples i want to point you to PlottableSignalConst. It super fast for any size.
There are few negative sides of PlottableSignalConst compare to PlottableSignal:

You can't update its data after ploting (Const). In fact, it have Api for updates, but it works very slow compare to PlottableSignal updates.
First render take significant additional time for data preprocessing.
It allocate  2-4 times more memory then source array. (You should switch to x64 for 50-100 million points plot).

If the proposed does not suit you, then you can manual draw the line yourself on top of ScottPlotControl. Of course this is a bad solution, but I cannot suggest anything else.

Thanks, that's what I suspected. As I'm able to get the range of the plot of the in and out points (can't remember the exact variable names) I can do some maths and plot an image of 3 pixels over the graph as the playhead then. Not a difficult task to do.
I'll have a test of the PlottableSignalConst and see what the results are.
Thanks again. Really appreciate all the help, and I look forward to seeing the new time only axis for the plots. 😊

Hey @vrdriver, sorry it took so long for me to get to this but I'm really happy that @StendProg was able to provide an excellent and thorough answer.
To summarize the conclusion about how to continuously read the position of dragged axis lines, the best way is to save the object PlotVLine() returns and access its position property whenever you want to know where the line is. Whether reading this property is triggered by a timer or some type of mouse move event is up to you.
Regarding the rendering system, ScottPlot uses System.Drawing to draw a plot as a Bitmap. User controls are just thin wrappers that watch the mouse and request next Bitmap whenever something changes. The appearance of animation comes from ScottPlot being pretty fast at drawing this way, but all interactivity is achieved by creating new Bitmaps. On one hand this makes it really easy to use and it's supported on many systems, but on the other hand adding advanced interactivity requires writing your own control.
I second @StendProg's recommendation to use PlotSignalCost() for plotting things like audio data. It is so fast, even with millions of samples, that refreshing the whole screen takes very little time.
I think this answers all the questions raised in this thread so I'll close this issue, but feel free to open it again if more questions pop up! --Scott


Background width non updating on window resize

Desc
Hi, I'm using your snippet for my private website but I'm facing this issue. If I resize the browser window to a smaller size it doesn't update the div size and I have to refresh the page to make width and height aligned.
Moreover by resizing to a larger size it's working w/o problems.

Not an issue, I set wrong div dimensions


Make sequence token optional

Problem
https://aws.amazon.com/about-aws/whats-new/2023/01/amazon-cloudwatch-logs-log-stream-transaction-quota-sequencetoken-requirement/
Sequence tokens are no longer required. Would be nice to have a switch to make them optional and get rid of those pesky InvalidSequenceTokenException.

Looks like InvalidSequenceTokenException should not appear anymore according to https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html .
But anyway there is prepared PR for getting rid of it in code https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs/pull/251


Divide molecule (input: SMILE) into user defined groups

Hi there,
I wanted to ask if it is possible to divide any given molecule into segments/groups which a user can define in a list/file with rdkit.
I'm a chemical engineer and want to apply the Marrero/Gani Group Contribution Method (http://citeseerx.ist.psu.edu/viewdoc/download?doi=<IP_ADDRESS>0.6493&rep=rep1&type=pdf).
This method dissects a molecule into first, second and third order groups (the lists of these groups can be found in Table 6, 7 and 8 of the linked paper above) and counts the occurrences of the found groups in the given molecule.
I've searched the manual and came across section 3.11 Molecular Fragments.
This comes quite close to what I would like to do: Store the 1st, 2nd and 3rd order groups in a txt or cvs file with SMARTS declaration and then use the datafile to count the occurring groups in the defined molecule.
In rdkit Fragments are defined to be made up of a set of connected atoms that may have associated functional groups. For the Group Contribution method I'm only interested in dividing the molecule into specific fragments. Thus, I should create my own FragmentDescriptors.csv file (https://github.com/rdkit/rdkit/blob/master/Data/FragmentDescriptors.csv) and then I could use the fcat.GetEntryDescription() to run through the found groups. But I don't see any way to count the occurence of the groups.
I would be very thankful for some feedback on this.

Found the first step to the final solution:
http://ctr.wikia.com/wiki/Unique_SMARTS_matches_against_a_SMILES_string
from rdkit import Chem
mol = Chem.MolFromSmiles('C1CC12C3(C24CC4)CC3')
patt = Chem.MolFromSmarts('*1**1')

print "number of matches:", len(mol.GetSubstructMatches(patt,uniquify=False))
print "number of unique matches:", len(mol.GetSubstructMatches(patt))

Would like to reopen this issue since this could also be interesting for other people.
So the above mentioned code snippet makes it possible to find unique matches of defined SMARTS (Groups) in SMILES (Molecule).
The problem I have now is that I want to dissect molecules in its biggest possible groups. This means that unique matched groups which are part of a bigger unique matched group should be excluded.
Does rdkit already provide a solution for this problem? Otherwise I will think of a way to solve this and post the solution here.

Hi, I recently published a paper on this: https://doi.org/10.1186/s13321-019-0382-3

@Plancalkuele
Fun fact, the method was developed at my uni (DTU) and I am actually working with this group segmentation.
w have an inhouse software that does it for us (ProPred).
Would be interesting to benchmark you algorithm against ours :)
So any updates?

@Plancalkuele
Fun fact, the method was developed at my uni (DTU) and I am actually working with this group segmentation.
w have an inhouse software that does it for us (ProPred).
Would be interesting to benchmark you algorithm against ours :)
So any updates?

I am seeing "invalid smile string error" in ProPred 4.4 for small drug molecule do I have to update the software? Thanks.

@Plancalkuele
I've been interested this problem for a while, so any updates? You said "I will post it here when I have written a publication..."
Thanks.

Hi, I recently published a paper on this: https://doi.org/10.1186/s13321-019-0382-3

@muratyalcinmy: if @Plancalkuele  publishes the algorithm it would be interesting to compare with the one I published to see whether we can learn something from each other.

Dear @muratyalcinmy @simonmb @arnaou @danilomendesdias sorry for my absence. I have been very busy with other projects and it's only possible to work sporadically on the algorithm.
I tested the algorithm with a data set of around 1160 molecules and there are some issues with the algorithm originating from the SMARTS identifiers and the next point on my ToDo list for this algorithm is to apply natural language processing (NLP) to generate the SMARTS (with the current SMARTS as initial guesses) to dissect the molecules into the correct groups. The following plot shows that the algorithm is not perfect and still needs work. The black spots show where the occurence of a group does not conform with the correct dissection data set.
The x-axis is the 1st and 2nd group id of the Marrero-Gani method and the y-axis is the id of the specific chemical.

So if you are interested to have a more validated dissection algorithm, I would rather recommend to go with Simon's code and see if you can construct the SMARTS identifiers for the type of group contribution method you are interested in and see if it works in your cases. But Simon's algorithm is tailored to the UNIFAC method and just substituting SMARTS will not be enough I guess. I find it interesting to have an adaptive group dissection algorithm which dissects molecules with respect to the data set available to the most necessary groups declared as SMARTS but generated via NLP. If a new chemical is introduced to retrieve a prediction and a fragment of the molecule can not be found in the NLP generated SMARTS set, then the adaptive algorithm has to be run again with the SMILES string of the new molecule.
Happy to discuss this further, since this is just an idea and I have so far only found two papers discussing NLP and applying it to SMILES so far:
https://arxiv.org/pdf/2002.06053.pdf
https://arxiv.org/pdf/1602.06289.pdf

My algorithm is not tailored for UNIFAC. There are two algorithms in my paper. The simple one would require some tweaking to work with other SMARTS than for UNIFAC.  The complete one, as I call it in the paper,  should find all possible fragmentations allowed for any kind of set of SMARTS.

The idea about using NLP sounds quite interesting. I don't know much about NLP and had never thought of using it although I thought of the idea of automagically generating a predictive model for some kind of molecular property including the needed SMARTS.

Hi, I recently published a paper on this: https://doi.org/10.1186/s13321-019-0382-3

@muratyalcinmy: if @Plancalkuele publishes the algorithm it would be interesting to compare with the one I published to see whether we can learn something from each other.

very interesting. Thank you

Hi, anyone still working on this? I'm very interested in this work too. However, I don't know how to transfer the substructures (groups) into SMART string, since the official document of RDKit introduced the 'mol.GetSubstructMatch(patt)' method which would be very useful.  Can anyone help me transfer the transfer the substructures (groups) into SMART string?  Thanks~

@tyyfkpo You can learn how to define SMARTS here. I can also recommend this webpage to test your SMARTS. For simple things you will be able to use GetSubstructMatch. For more complex things you'll need a better algorithm like the one I mention above.

Hi @simonmb . I read your paper on this issue. I found a list of 119 groups and their SMILES in your paper. Is is possible to extend these groups? I need to find all 1st, 2nd, and 3rd order groups of marrero and gani groups to estimate some properties for my work.

Yes definetly, the code was developed to be used with whatever SMARTS groups you need. I recommend the official SMARTS documentation and having a look at smarts.plus when you are testing your SMARTS.

Yes definetly, the code was developed to be used with whatever SMARTS groups you need. I recommend the official SMARTS documentation and having a look at smarts.plus when you are testing your SMARTS.

I just saw I had posted the same urls in the previous comment... :)

Hi @simonmb . I read your paper on this issue. I found a list of 119 groups and their SMILES in your paper. Is is possible to extend these groups? I need to find all 1st, 2nd, and 3rd order groups of marrero and gani groups to estimate some properties for my work.

If you look at the paper , you will see, that the levels of groups are relatively well defined,sonyou can search for the groups sequentially.

I have a set of 10,000 canonical SMILES and I want to estimate some properties using group contribution methods for the. I think your code can do this for me. But I have a problem while using your code. Lower case c represents aromatic carbon in the SMILES that I have, but your code returns an error which says "Flowing simles is not valid: c". Same problem with double bounds which are represented with "=" in my set of SMILES but your code can not interpret them and gives error. How can I deal with this problem?

The error you describe is correct, as a single aromatic atom is not a molecule you can fragment. You probably have the 10000 molecules from somewhere and there might be errors in the SMILES. Whenever rdkit is unable to create a molecule from a SMILES you provide, you will get this error.

@sardaar11 please open an issue in the fragmenter repository and provide the details. i.e. example code to reproduce the error.

Great discussion, everyone!
Please move further discussions to the "discussion" tab.


nixos/prometheus/pushgateway: add module and test

Motivation for this change
We have a package for the prometheus-pushgateway but not a corresponding NixOS module. Let's fix that.
Things done

Added the NixOS module nixos/modules/services/monitoring/prometheus/pushgateway.nix.
Extended the prometheus2 NixOS test with a test for the pushgateway.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg test prometheus2


MultiValue control

When selecting multiple values...get a message at top indicating that the values field contains values that are not in the list of supported values.
The field type is Picklist String.  I can select a single value and it works fine but when another value is selected i get the message.

Please ensure that you have allowed users to input their own values in the field


[Markdown] Ensure rendering Alert component

Closes #379
Description
This PR will enable the Note syntax to be rendered as an Alert component even when line breaks are used within it.
Current behavior (updates)
When I wrote Markdown like the following, it did not render correctly as an Alert component.
:::note

This is the first text.

This is the second text.

:::

New behavior
The above Markdown renders as intended;

Is this a breaking change (Yes/No):
Yes
Additional Information

@koralle
Please check Quality.


Prevent Bound Armour being repaired via Crafting

PrepareItemCraftEvent doesn't give us a usable result when asking for the Recipes result, therefore we cannot support ArmourBound armour being repaired initially. This implementation could make the result be the ArmourBound item + the durability acquired in repair however is open to allowing enchanted items to be repaired if done, hence the decision to just make the result AIR

Doesn't craft-repairing cause the item to be reset (de-enchanted, de-named, etc)?

Yes, it's like a fresh new item out of thin air whereas Anvils, Smithing tables and Enchanting tables all just modify the base item

Wouldn't this make bound armour overly restrictive? Craft-repairing the item would remove the binding, yes, but also all the effects that made the binding worth it, right?

Also, take a quick look at Smithery#AlloyUtils as an example of a simpler means of testing crafting matrices.


GMM based entropy model

Hi, thanks for nice works!
I have some questions about GMM based entropy model in the framework of CompressAI.

Do you have any plans to implement GMM based entropy model?

If it's not a matter, could you give me some hints to implement it myself?

I have implemented GMM based entropy model for training, but not for test (real encoding/decoding).
That is, I successfully modified entropy_parameter module in Cheng2020 model and _likelihood() of GaussianConditional, but I have no idea how to modify update() of GaussianConditional.
How should I change update function for real compression?
Thanks.

Hi, we don't have plan to implement it soon but we'd gladly accept a pull request :-).
So far the quick experiments I've done haven't shown better results for this class of model, so it hasn't been a priority.
The update function computes a finite list of quantized CDFs from the scales table for faster encoding/decoding. I'm not sure how this would apply in the GMM case since the pmf/cdf of each latent element would be dependent not only on the scale but also the weights of the GMM.
As a first experiment you can skip the update function and modify compress and decompress to manually compute the pmf/cdf for each element. Then there might be a way to compute a table for faster processing.

Thank you for the answer.
Is there any way to use current rans implenmentation without cdf, cdf_lengths and offsets?
I can manually compute the pmf/cdf for each element, but these variables are used currently in compress and decompress functions.
Thanks!

Not at the moment, you'll have to write new bindings to the C api (https://github.com/InterDigitalInc/CompressAI/blob/master/third_party/ryg_rans/rans64.h). You can also use Lucas Theis rangecoder instead.

I see. Thanks.


Recompile plugin with steampipe-plugin-sdk v4.1.13

Integration test logs

Logs
Add passing integration test logs here

Example query results

Results
Add example SQL query results here (please include the input queries as well)

Closing this PR in favor of v5 SDK release which will be done next week.


Scheduling recurring function calls

Hello,
I'm using this package for a side project, so first of all thank you for providing it!
I'd like to hack on my local copy and add the capability to call an arbitrary function on a recurring/scheduled basis (using e.g. time.Tickers) to do things like announce to channels and also to poll GPIO devices at regular intervals and present the results on IRC.  But it's not immediately clear to me where would be the best place to add this.  Would it be better...

Define a function similar to maybeStartPingLoop() and call it from RunContext()? or...
Edit the loop in startReadLoop() and call scheduled function calls there?

Or is there a better approach?  I'm still kinda feeling my way around Go, coming from C#.
Thanks!

Well I have a working patch so I'll create a PR in case anyone is interested.

Hi, thanks for posting! I've been purposefully keeping go-irc fairly minimal so I'm not sure how likely this feature would be to make it into core, but I've got an example below which does pretty much the same thing without modifying the framework. If you've got any questions, I'd be happy to answer them.
package main

import (
	"sync"
	"log"

	"gopkg.in/irc.v3"
)

func main() {
	// sync.Once allows you to create a container which will track when it's
	// been called - it's generally used for initialization, so it's helpful
	// here.
	var tickStarter sync.Once

	// Using a cancellable context allows us to cancel the connection outside a
	// connection error. This is also what allows us to use ctx.Done() later to
	// roughly track when the connection is done.
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// TODO: insert conn here, rather than nil
	c := NewClient(nil, irc.ClientConfig{
		Handler: HandlerFunc(func(c *irc.Client, m *irc.Message) {
			log.Printf("Got Message: %+v\n", m)

			// "001" is also known as RPL_WELCOME - every compliant server I'm
			// aware of sends this when the user has successfully connected. This
			// should only be sent once, but we use the sync.Once to ensure we only
			// start the tickers once.
			if m.Command == "001" {
				// Here we use the sync.Once we created earlier to fire off our
				// tickers in the background. It's helpful to do this in the
				// Handler because you have access to the Client.
				tickStarter.Do(func() {
					go func() {
						// Fairly standard ticker code
						ticker := time.NewTicker(30 * time.Second)
						defer ticker.Stop()

						for {
							select {
							case <-ticker.C:
								_ = c.Writef("PRIVMSG #some-channel :I ticked!")
							case <-ctx.Done():
								log.Println("Closing ticker")
								return
							}
						}
					}()
				})
			}
		}),
	})

	err := c.RunContext(ctx)
	if err != nil {
		log.Print(err.Error())
		return
	}
}

Thanks. Your example will hopefully be of use to someone; it's well written and you should put it in the documentation!
As for me I think I'm going to rewrite in a better language that has sane packaging. I'm not interested in wrestling with Go any more, and rewriting from scratch in C# or Lisp or something will probably take less time than figuring out how to make go not choke on a package with a version greater than 1.

Fair enough! Best of luck with your project! Either, way thanks for reaching out - better examples are one thing I really could provide.
On a side note, if you ever have any questions about the IRC protocol itself (there are definitely a few "gotchas", https://modern.ircdocs.horse is a great resource, despite the strange URL. Also, a number of people who know the protocol forwards and backwards hang out in #botters and #ircdocs on Libera.


Remove deref() from openlineage-sql impl.

Problem
New release of rustc started to fail on unnecessary defer.
Solution
Remove all defer() calls listed in failed build logs.
Checklist

[x] You've signed-off your work
[x] Your pull request title follows our guidelines
[ ] Your changes are accompanied by tests (if relevant)
[x] Your change contains a small diff and is self-contained
[ ] You've updated any relevant documentation (if relevant)
[ ] Your comment includes a one-liner for the changelog about the specific purpose of the change (if necessary)
[ ] You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
[ ] You've added a header to source files (if relevant)

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2023 contributors to the OpenLineage project

I'll fix pre-commit failure in separate PR.