Example folder out-of-date

The example directory has just 1 file in it - pipeline.py. My understanding is that we want a nice commented up template version of recipe.py/meta.yml instead.
@cisaacstern is the highly commented copy of those files from the sandbox something we could get in here?

Yes, though I'm not sure the best way to avoid maintenance redundancy here.
Open option would be to not provide an example at all in staged-recipes, and then just make it really clear that example files live in the sandbox?

I like that idea for reducing duplication @cisaacstern. I do think we would want to make sure that someone who navigates directly to staged recipes examples could directly find the sandbox.
Would it be odd to add a readme to staged-recipes/recipes that directs a user to the sandbox?

Would it be odd to add a readme to staged-recipes/recipes that directs a user to the sandbox?

This sounds reasonable, plus maybe we can add a PR template for staged-recipes with a header note that says something like
":wave: Thanks for opening a PR! Do you have a meta.yaml and a recipe.py in this PR? If you don't know what those are, please check out the sandbox"

closed with #105


[tlse] internal TLS support for Nova

Creates TLS certs via cert-manager for NovaAPI, NovaMetadata and NovaNoVNCProxy when spec.tls.endpoint.internal.enabled: true
Depends-On: https://github.com/openstack-k8s-operators/lib-common/pull/428
Depends-On: https://github.com/openstack-k8s-operators/openstack-operator/pull/620
Depends-On: https://github.com/openstack-k8s-operators/nova-operator/pull/646
Jira: OSPRH-3294

needs to be rebased when https://github.com/openstack-k8s-operators/openstack-operator/pull/620 landed. only last commit is relevant for this PR

/retest

recheck

OpenStackDataPlaneNodeSet failed with:
  - lastTransitionTime: "2024-01-30T11:32:17Z"
    message: NodeSetIPReservationReady error occurred
    reason: Error
    severity: Error
    status: "False"
    type: Ready

recheck

recheck


chore(main): release matterbridge-home-assistant 0.5.0

:robot: I have created a release beep boop
0.5.0 (2024-06-13)
Features

docker: #44 add config-file support for docker deployments (#63) (f3c38ee)

Bug Fixes

update dependencies and add missing docker build-args (#65) (cfb69f5)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

matterbridge-home-assistant-v0.5.0
:sunflower:


Sending Wireguard-Config via Postfix Mailserver Fails

This is my config.yml mail configuration:
...
host: mailserver.domain.tld
port: 25
encryption: starttls
cert_validation: false
username: smtpuser
password: xxxx
auth_type: plain
from: Wireguard Portal<EMAIL_ADDRESS>...
Failed to send peer configuration:  failed to send peer email for 3gVy/SNBS7Z+Va9Ogn4oG0VWS4hsJJdMkZu1aFWiwT4=:
failed to send mail:
failed to connect to SMTP server: Mail Error on dialing with encryption type STARTTLS:
dial tcp [2001:xxxx:xxxx:2d00::70]:25: --> whatever "dial" means, this is an IPV6 Address!
connect: cannot assign requested address
Am I right, the WG-Portal mail sender part cannot use a IPV6 connected mailserver?
Sending test mails via mailx work without issue.
testsaslauthd smtpuser:xxxx returns ok.
This error message is from the Firefox ESR F12 console.
This console comes handy for many WG-Portal error messages.
Especially those related to CORS-header problems.
The error messages in the GUI fade away too quickly to read them.
Any advice would be much appreciated.

are you running wg-portal inside docker? is docker IPv6 enabled?


build: try/catch the @angular/bazel rules_nodejs 1.0.0 patch

This is needed for the material unit test to pass on angular repo in https://github.com/angular/angular/pull/34589 otherwise the patch fails (since the changes are already made in that context) and throws and the CI job fails with:
      if (config.fatal) throw e;
                        ^

Error: exec:
    at Object.error (/tmp/material2/node_modules/shelljs/src/common.js:110:27)
    at execSync (/tmp/material2/node_modules/shelljs/src/exec.js:102:12)
    at String._exec (/tmp/material2/node_modules/shelljs/src/exec.js:205:12)
    at String.<anonymous> (/tmp/material2/node_modules/shelljs/src/common.js:335:23)
    at applyPatch (/tmp/material2/tools/bazel/postinstall-patches.js:186:26)
    at Object.<anonymous> (/tmp/material2/tools/bazel/postinstall-patches.js:35:1)
    at Module._compile (internal/modules/cjs/loader.js:936:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
    at Module.load (internal/modules/cjs/loader.js:790:32)
    at Function.Module._load (internal/modules/cjs/loader.js:703:12)
error Command failed with exit code 1.
)
``

@mmalerba Thanks for merging https://github.com/angular/components/pull/18064. This PR is a quick follow-up to that as the change did not work as expected to green up https://github.com/angular/angular/pull/34589.


languagetool: 5.1 -> 5.2

Motivation for this change
https://github.com/languagetool-org/languagetool/blob/v5.2/languagetool-standalone/CHANGES.md
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS

[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Not required but it is better Bash.
https://github.com/koalaman/shellcheck/wiki/SC2086
https://github.com/koalaman/shellcheck/wiki/SC2035

Not required but it is better Bash.
https://github.com/koalaman/shellcheck/wiki/SC2086
https://github.com/koalaman/shellcheck/wiki/SC2035

https://github.com/NixOS/nixpkgs/pull/105641

https://github.com/NixOS/nixpkgs/pull/105641

#105641

That is for sh files which are often writte in sh. If we would add " to packages we would need to change all packages specifying any phase, pre or post.

#105641

That is for sh files which are often writte in sh. If we would add " to packages we would need to change all packages specifying any phase, pre or post.

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108912 run on x86_64-linux 1

3 packages built:

languagetool
vimPlugins.lens-vim
vimPlugins.vim-grammarous

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108912 run on x86_64-linux 1

3 packages built:

languagetool
vimPlugins.lens-vim
vimPlugins.vim-grammarous


Feign : Reuse RibbonLoadBalancer

A bug was pointed on Feign Ribbon integration https://github.com/Netflix/feign/issues/182
Ribbon does not expect multiple RibbonLoadBalancer (IClient/AbstractLoadBalancerAwareClient) instances for the same client.
spring-cloud-netflix seems to have the same bug (WARN  c.n.servo.jmx.JmxMonitorRegistry - Unable to register Monitor::MonitorConfig{name= ...).
Feign Ribbon integration was first corrected by adding cache on FeignRibbonClient.lbclient() method to check if a client exists before creating a new one.

Even with the implemented fix issue is observable if two threads simultaneously try to access a LBClient for a missing client name.

@ivelkov if there is a problem can you open a new issue with what you're seeing and how to reproduce?


west.yml: Update hal_nordic to align to clock_control

hal_nordic required update in ieee802154 radio driver which
is controlling the clock.
PR in hal_nordic:
https://github.com/zephyrproject-rtos/hal_nordic/pull/17
Signed-off-by: Krzysztof Chruscinski<EMAIL_ADDRESS>

Can you update this now @nordic-krch ?

@ioannisg updated.


Use the Pyreal sample application for housing example

Pyreal now has a sample application for Ames Housing. We should use this for our housing example in Sibyl-API

Closed by #26


Changelog?

Hi, I cannot find a changelog for new versions, either in the repo or in the tags in the repository. Is it available somewhere? I'm considering updating from v3 to v4 but I'm afraid of any BC breaks.

The only difference between 3.0x and 4.x is the removal of the AcmeDemoBundle skeleton and of the composer hook allowing to install it (which is why it is a major version)

OK thank you. I don't know if this issue needs to stay open if you plan to add such changelog somewhere. Else feel free to close it :)

I cannot close it. I'm not a maintainer on this bundle


Added background in Pomodoro page

Background same as Iskcon page has been added to the Pomodoro page

What to do to pass all checks ? Should I have to create pull request again @akshitagupta15june ?

@Viveksati5143  how to pass all these above changes? Will it create any problem in merging this PR? Any suggestions?
Thanks

Thanks
You're welcome @akshitagupta15june


running= npm i node-red-contrib-http-event-collector

I am getting this error when running the following
npm i node-red-contrib-http-event-collector
npm ERR! code ENOENT
npm ERR! syscall spawn git
npm ERR! path git
npm ERR! errno ENOENT
npm ERR! enoent Error while executing:
npm ERR! enoent undefined ls-remote -h -t https://github.com/splunk/splunk-javascript-logging.git
npm ERR! enoent
npm ERR! enoent
npm ERR! enoent spawn git ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in: Logfile
I get the same error when trying from the Node red palette. Any help appreciated.

24 verbose node v12.18.3
25 verbose npm  v6.14.8

Node has been updated.


Verbose mode for comments

trigger: pull_request doesn't re-trigger on force push. Is there a way how to achieve re-build/re-testing upon force push?
Have an option to add a new comment after each built.
edited by @lachmanfrantisek

@TomasTomecek
After we merge your PR with logs, this needs to be redesign.
What about this:

By default, no comments and everything is accessible via status URL.
If the verbose mode is configured, add a comment after each run:

zuul-like format (list of all chroots)  -- harder since we receive response per chroot
one-comment-per target (a lot of spam)
only error messages (like the current behaviour)

honestly, since we have our own status page now, I'd disable all commenting, maybe even the congrats comment - we can discuss when I get back

https://github.com/packit-service/packit-service/pull/447 is now merged (and will be deployed in the next cycle, meaning on Monday). That PR should resolve this issue: packit-service now has a status page for every build with info and logs - the logs should be present even when the SRPM creation fails.
I'm closing this issue - feel free to reopen if the proposed solution doesn't work for you.


Keep up the good work!

Thanks for the database update.
Anyway you can update the database to the current date?
Also is their anyway to make this use GPU instead of CPU?

Thank you for the nice words!
I will update the database again, if I find some time.
For questions about Plutus, you should ask in the original repository. I haven’t coded Plutus.

@DigitalCorrosion : Can you please let me know if you were able to find any btc with balance using this? If yes could you pleaee let me know how? Can I DM you?


Fix the PullApprove approve_regex to cover more cases

'^(Approved|lgtm|LGTM|:shipit:|:star:|:+1:|:ship:)'
Signed-off-by: Chris Aniszczyk<EMAIL_ADDRESS>

cc: @vbatts and sorry for the tooling churn, we will be better for it :)

LGTM

OHMAN
LGTM

On Wed, Jun 01, 2016 at 01:02:42PM -0700, Chris Aniszczyk wrote:

'^(Approved|lgtm|LGTM|:shipit:|:star:|:+1:|:ship:)'

This is quite a few more cases ;).  Does it come with a corresponding
update to the docs 1?  Personally I think lgtm/LGTM is sufficient.

please, let the maintainers decide.

:ship:

ah damn, did the PR wrong which fails the build, going to move discussion to a new PR:
https://github.com/opencontainers/image-spec/pull/106


extrisic between camera and lidar

hi! thanks for your datasets, it's a good work. And i want to ask if you have the extrinsic between camera and lidar, and camera intrinsic?

I have the same problem. @ShiJian1995  Have you figured out the extrinsic between camera and lidar, and the camera intrinsic?


本地运行镜像，不成功

指令：
sudo docker run pai-image-manage-registry.cn-wulanchabu.cr.aliyuncs.com/pai/pytorch-training:2.0-ubuntu20.04-py3.10-cuda11.8-megatron-patch-llm
只输出了下面内容：
== NVIDIA TensorRT ==
NVIDIA Release 22.09 (build 44877791)
NVIDIA TensorRT Version 8.5.0
Copyright (c) 2016-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Container image Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
https://developer.nvidia.com/tensorrt
Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh
To install the open-source samples corresponding to this TensorRT release version
run /opt/tensorrt/install_opensource.sh.  To build the open source parsers,
plugins, and samples for current top-of-tree on master or a different branch,
run /opt/tensorrt/install_opensource.sh -b
See https://github.com/NVIDIA/TensorRT for more information.
WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
Use the NVIDIA Container Toolkit to start this container with GPU support; see
https://docs.nvidia.com/datacenter/cloud-native/ .


Wrap python library in flask or fastapi, exposing an openAPI endpoint

Part of larger issue: https://github.com/microbiomedata/nmdc-metadata/issues/330

Met with @turbomam and @wdduncan today. This will be assigned to @cmungall for now although someone else from his team may work on it.  I've added it for September but please let me know if the timing should be pushed back (or forward).
@kfagnan @emileyfadrosh


Refactor wideband

Related issue is #66.

It's faster, since we only have to do a scalar * vector product instead of
a scalar * matrix product.
On Mon, 4 May 2020 at 13:27, MichaelDoebereiner<EMAIL_ADDRESS>wrote:

@MichaelDoebereiner commented on this pull request.
In eadf/core.py
https://github.com/EMS-TU-Ilmenau/EADF/pull/67#discussion_r419369271:

@@ -563,95 +308,26 @@ def evaluatePatternNarrowBand(

     def funCoEle(arrCoEle):
         # equation (7) in the EADF Paper

       return xp.exp(1j * xp.outer(muCoEle, arrCoEle))

       return xp.exp(xp.outer(1j * muCoEle, arrCoEle))

Does this change anything?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/EMS-TU-Ilmenau/EADF/pull/67#pullrequestreview-404902109,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAINSRUHSJ7NUTFAKNAL4T3RP2RC3ANCNFSM4MTUIDVQ
.

Let's see what @StellarSankalp has to say. :)

Refactored code works pretty smoothly. As Michael, PerformanceEADF seems to perform slower than normal EADF as shown by the results of performance.py demo.
Good to go from me as well! May the force be with you ^^


[Derived Field] Dynamic FieldType inference based on random sampling of documents

Description
This class performs type inference by analyzing the _source documents. This will be useful in inferring the type of nested derived field of object type. See https://github.com/opensearch-project/OpenSearch/issues/13143 for more details on requirement.
It uses a random sample of documents to infer the field type, similar to dynamic mapping type guessing logic. Unlike guessing based on the first document, where field could be missing, this method generates a random sample to make a more accurate inference. This approach is especially useful for handling missing fields, which could be common in nested fields within derived fields of object types.
The sample size should be chosen carefully to ensure a high probability of selecting at least one document where the field is present. However, it's essential to strike a balance because a large sample size can lead to performance issues since each sample document's _source field is loaded and examined until the field is found.
Determining the sample size (S) is akin to deciding how many balls to draw from a bin, ensuring a high probability (>=P) of drawing at least one green ball (documents with the field) from a mixture of R red balls (documents without the field) and G green balls:
 P >= 1 - C(R, S) / C(R + G, S)

Here, C() represents the binomial coefficient. For a high confidence level, we aim for P >= 0.95. For example, with 10^7 documents where the field is present in 2% of them, the sample size S should be around 149 to achieve a probability of 0.95.
Here is the small python script which i used to calculate above
Related Issues
Resolves https://github.com/opensearch-project/OpenSearch/issues/13143

Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Also, why skip-changelog? This will be consumed by the DerivedFieldMapper in some way I am assuming, wouldn't we want to keep track of this change?

@msfroh @harshavamsi what do you think about order in which we should scan the segments here? If we start with the smaller segments and if the field is found, then it would be pretty fast, whereas, if we start with a bigger segment, the odds of finding a field is high but comes at a cost of loading a bigger segment. So for rare fields, later performs better whereas for common fields, the former performs better.

@msfroh @harshavamsi what do you think about order in which we should scan the segments here? If we start with the smaller segments and if the field is found, then it would be pretty fast, whereas, if we start with a bigger segment, the odds of finding a field is high but comes at a cost of loading a bigger segment. So for rare fields, later performs better whereas for common fields, the former performs better.

I think I would optimize more for the common fields.
I appreciate that an advantage of this feature is that it's another way of handling a mix of different document types, similar to flat_object fields -- just pushing the hard work to search time, rather than flattening at indexing time. But at the same time, I feel like it makes more sense to assume that you want to search on "relatively" common fields (i.e. fields present in at least 5-10% of documents).

@msfroh looking at holistic picture, I agree that optimizing on common fields is a wiser choice. If you think this isn't super critical, I can take it up as a subsequent PR.


Full Nextclade runs

Description of proposed changes
This PR adds --nextclade_full_run flag for ingest-gisaid and ingest-genbank scripts as well as a pair of GitHub actions. This flag is needed to trigger full run of Nextclade on all available sequences.
In current implementation, clade results (and other metrics) are cached in nextclade.tsv file. Upon rerun, the sequences in sequences.fasta and  nextclade.tsv files are compared and only sequences that are missing in nextclade.tsv undergo analysis with Nextclade. The results are then appended to nextclade.tsv. Then nextclade.tsv gets merged with metadata.tsv and that's how clades and some of the other Nextclade metrics appear in the medatata.
With the new flag --nextclade_full_run provided, the nextclade.tsv (cached clades from the previous run) does not get downloaded. So the comparison always gives a list of all sequences and Nextclade processes them all again, and metadata table gets refreshed clades and metrics for all rows.
This refresh is necessary every time when new clades are defined or when upgrading to the new version of Nextclade with breaking changes.
Previously we used to perform these runs manually on external resources, but now we are thinking about automating this.
The trigger should probably still be manual, because not every Nextclade release needs the full run.
Depending on the manual data curation flow, these runs can be performed on master or on a separate branch. If on separate branch, the resulting metadata.tsv and nextclade.tsv will end up in the branch/ subdirectory and should be copied manually to the location when the next ingest from master can find them.
Another challenge arises when we want to perform this full run before the required Nextclade version is released. Nextclade currently does not publishes pre-releases of CLI, but that can be worked out. Although automated C++ builds from master branch might be too costly in terms of CI credits. I currently experiment with the pre-released binaries in #215, branch nextclade-full-run-2021-09-30
Related issue(s)

#215

Testing
Let's experiment a little in [#215] and see how well it goes :)
We need to make sure the workflow for full reruns is convenient and the results are correct (that is that the full run actually happens and clades get recomputed).

I realized that we don't need to mix the usual ingest flow with Nextclade full run. It can be made separate. The new version is here: https://github.com/nextstrain/ncov-ingest/pull/218/files


1.17 Expandable Table Row

Existing functionality in <vaadin-grid>
Click on a row affordance, display area below the row with additional information. Will be flexible to display any component (such as a chart or a graph) inside the expanded area.
Provide an example.
UI specs

Tagging to delivery 5, just so that we do not forget this.

Assigned to @drewharvey. To be implemented by using similar template as vaadin-grid, just passing it via px-data-grid to vaadin-grid.
And then API for application to ask item to expanded and collapsed.

Included to delivery 5 A


Fixed another windows line ending issue

This completely fixes #4 since re-importing and renaming was still broken.

Ah, good catch.


Fix upgrade script changes

Description
Move temporary procedure babelfish_revoke_guest_from_mapped_logins from
babelfishpg_tsql--4.4.0--4.5.0 to babelfishpg_tsql--4.3.0--4.4.0 upgrade script.
Task: BABEL-5389
Signed-off-by: Rishabh Tanwar<EMAIL_ADDRESS>Test Scenarios Covered

Use case based -

Boundary conditions -

Arbitrary inputs -

Negative test cases -

Minor version upgrade tests -

Major version upgrade tests -

Performance tests -

Tooling impact -

Client tests -

Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>8
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 74.712%

Totals

Change from base Build<PHONE_NUMBER>4:
0.003%

Covered Lines:
45752

Relevant Lines:
61238

💛  - Coveralls


System out of Memory Exception during Runtime with ML.net <IP_ADDRESS>1904 (Suspected Memory Leakage)

System Information:

Model Builder <IP_ADDRESS>1904
Visual Studio Version 16.10.4

Describe the bug

I have trained Model for Image Classification using 2000 Image Sets with Right and Wrong Classification after I deployed the model, with each iteration of the Image Prediction Process the Memory Usage increases and after a 'n' no of Iteration the system runs into "System out of Memory Exception"

Clear description of the problem:
I am building an application that will constantly take images from a camera and classify the images as wrong / right using the pretrained prediction model (Built using ML Model Builder 16.10.4), during both development and deployment it is noted that with each iteration of the Infinity loop that captures Image from the camera the Memory Graph Shoots up by 100 MB (Approximate Value) and finally the system crashes after certain time due to Out of Memory Exception.
In order to have a better understanding I created a separate application with a Normal File Browser where the infinity loop was avoided, and the behavior was same, every time I uploaded a Image, the RAM / Memory Usage shoots up by 100 MB (Approximate Value)
However when I am working with ML Builder Version <IP_ADDRESS>7905 we don't experience any of such issues, and the system is running stable and no such memory leakage detected. I have worked on with ML Builder Version <IP_ADDRESS>7905 with approximate 1 Lakh Images in an infinite loop for over 36 hours runtime and the system is completely stable.
I have tried with IDisposable(), GC.Collect() but no Improvement or Drop in Memory Usage Noted.
To Reproduce
Steps to reproduce the behavior:

Train an Image Prediction Model using Model Builder <IP_ADDRESS>1904
Create an Application with Simple File Browser to run a prediction algorithm that will show the Output as Right or Wrong with Prediction Percentages
Every time you run the Process you will see a spike in memory usage

Expected behavior
The Increase in Memory Usage in each Iteration should be stopped.
Screenshots
The below Image was generated using Performance Profiler in Visual Studio with 2600 Images. We can see that GC is fired but the Memory Consumption is constantly increasing.

Sample Code
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace MLModel
{
public partial class Form1 : Form
{
string rightImg = AppDomain.CurrentDomain.BaseDirectory + "right";
string wrongImg = AppDomain.CurrentDomain.BaseDirectory + "wrong";
string rawImage = @"D:\rowImage";
public Form1()
{
InitializeComponent();
ImgSorting();
}
protected void ImgSorting()
{
try
{
if (!Directory.Exists(rightImg))
{
Directory.CreateDirectory(rightImg);
}
if (!Directory.Exists(wrongImg))
{
Directory.CreateDirectory(wrongImg);
}
DirectoryInfo ds = new DirectoryInfo(rawImage);
int c = ds.GetFiles().Count();
while (c > 0)
{
DirectoryInfo di = new DirectoryInfo(rawImage);
string firstFileName =
di.GetFiles()
.Select(fi => fi.Name)
.FirstOrDefault();
string filePath = rawImage + "\" + firstFileName;
                var input = new MLModel1.ModelInput();
                input.ImageSource = filePath;
                MLModel1.ModelOutput resulty = MLModel1.Predict(input);
                if (resulty.Prediction.ToString() == "Wrong")
                {
                    string newcropFilePath = Path.Combine(wrongImg, firstFileName);
                    if (File.Exists(filePath))
                    {
                        File.Move(filePath, newcropFilePath);
                    }
                }
                else
                {
                    string newcropFilePath = Path.Combine(rightImg, firstFileName);
                    if (File.Exists(filePath))
                    {
                        File.Move(filePath, newcropFilePath);
                    }
                }

            }
        }
        catch (Exception x)
        {
            throw x;
        }
    }
}

}

Same problem occur with me.    Please help...

I face this issue for a loop. Inside this loop I use GC also. But still the issue is there. Please let me know how to solve this. Thank you in advance..

I am experiencing similar issues with Out of Memory Exception with Build <IP_ADDRESS>1904. After 15 - 20 mins of running the system consumes entire 16 GB of my RAM and crashes

@LittleLittleCloud Can you please take a look at this?

Hi @sbose768 @rahulgorai
Thanks for ring this bell. I'm able to reproduce it on my side. Although I don't know the actual cause, this might be related to the creation of MLContext/PredictionEngine every time Predict gets called.
One workaround is to use lazy-loaded PredictEngine, which should resolve the memory leak as well as increase predicting speed.
#1638
This bug is more likely in ML.Net framework so I will transfer this issue to their repo, let us know if there's any questions.


An Aside about MVVM

I have an app that is going to use Facebook as my authentication, Xamarin. I have tried several different techniques for logging on. This seemed the most straight forward. As a first step, I got the example code running, which worked like a champ. Next, I tried to get the same code working in my starter app, I am not real familiar with the MVVM use frankly, understand it basically, but not the details. Here is the problem. I know I am logging on, I know I'm pulling the email, Full Name but after those are pulled the fields are not displayed on the LoginPage? Anyone have an idea what I could look for? Man it sure looks the same to me with just name changes for my solution name.

This issue doesn't seem to be related to this plugin.


Exclude log4j from transitive dependencies

Excludes log4j from transitive dependencies
mvn dependency:tree -Dincludes=log4j:log4j:jar | grep log4j comes back clean after this, have not tested functionality though.

Thanks Marcus, I'll merge the PR and we'll do a round of testing.


Reshape fails for # dimensions with 2 digits

Hi,
small bug fix in our fork for commit # 16521d04fcfabfd1a8c0c40cbe5a0f3195173bb1.
In case the commit # is not sufficient, here the line of code that changed:
line 466 in Dimensions_GPI.py

           index = int(event[10:12])

           index = int(event[10:13].replace(']',''))

Is there a way to do a pull request for certain commits only rather than all of them?
best
Gabri and Mike

Funny I was just looking at your updates.  I think if you push to the same branch as your pull request it should be automatically included.

Hi Mike,
Great work! There isn't a simple way to specify which commits. You need to create another branch to do it:
http://stackoverflow.com/questions/25955822/git-cherry-pick-a-single-commit-for-pull-request
Thanks!

Thanks, I gave that a try..
in github I made a now fork of bni_core
I used SourceTree to make a local clone.
Then in a command window I did:
git fetch origin
git reset --hard origin/HEAD
git cherry-pick 16521d0
but then I got:
fatal: bad revision '16521d0'
Using the full length commit number gave me this:
fatal: bad object 16521d04fcfabfd1a8c0c40cbe5a0f3195173bb1
anything obvious I'm doing wrong?
On Thu, Mar 10, 2016 at 4:04 PM, Ashley Anderson<EMAIL_ADDRESS>wrote:

Hi Mike,
Great work! There isn't a simple way to specify which commits. You need to
create another branch to do it:
http://stackoverflow.com/questions/25955822/git-cherry-pick-a-single-commit-for-pull-request
Thanks!
—
Reply to this email directly or view it on GitHub
https://github.com/gpilab/core-nodes/issues/6#issuecomment-195048559.

--
Michael Schär, PhD
Division of MR Research, Department of Radiology, Johns Hopkins University
Park Bldg 330, 600 N. Wolfe St., Baltimore, MD 21287
Phone: +1<PHONE_NUMBER>
<EMAIL_ADDRESS>

It sounds like maybe you're (understandably) getting branches and forks mixed up. Git keeps track of branches within each repository, while forks are really a GitHub thing with no official standing in git (in git a fork is no different from any other clone).
I don't normally use SourceTree, but I think this is how you'd do it:

Enter your existing repository and click the Branch+ button (red circle). That will give you the popup you see.
Give your new branch a name (maybe a name related to the feature).
Pick the last commit in common with our develop branch (before you started making changes as the starting point for the branch.
Create the branch and check it out.

Push that branch to the GitHub remote for your fork (michaelschar/core-nodes).
Create a pull request for that branch against our develop branch.


toga-demo throws and exception

Follonwing the Quick Start instructions on https://toga.readthedocs.io/en/latest/ results in
** (toga-demo:13985): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
(toga-demo:13985): Gtk-WARNING **: Theme parsing error: gtk.css:13:32: The style property GtkRange:stepper-size is deprecated and shouldn't be used anymore. It will be removed in a future version
(toga-demo:13985): Gtk-WARNING **: Theme parsing error: gtk.css:14:28: The style property GtkRange:slider-width is deprecated and shouldn't be used anymore. It will be removed in a future version
Traceback (most recent call last):
File "/home/juhaj/venv_toga/lib/python3.5/site-packages/toga_gtk/app.py", line 116, in _startup
self.startup()
File "/home/juhaj/venv_toga/lib/python3.5/site-packages/toga_demo/app.py", line 55, in startup
self.main_window.toolbar = [cmd1, toga.SEPARATOR, cmd2]
AttributeError: module 'toga' has no attribute 'SEPARATOR'

see: https://github.com/pybee/toga-demo/issues/15

This is fixed in version 0.2.15


Refusing to work on anything except Chrome

As the title says, works perfectly in Chrome, but not firefox, safari etc. Have tried everything, document ready fires as expected, then the time elements just stay blank. Any ideas?

Sounds like you're not using a ISO8601 timestamp in the original DOM element.

Please read the docs: http://timeago.yarp.com/

Turns out it's working fine when I remove the milliseconds from the end of the stamp. But yet when I use milliseconds in an example JSBin in Firefox on its own it works fine, weird


How to send 'cacheDirectory' to CQuery, non Editor application

Hi,
I want to use CQuery to find e.g. C/C++ symbols to use them for UML-modelling.  I thought it should be doable with CQuery/Language Server. I don't want to use it in the context of an Editor. I only want to use some services of CQuery.
I've tinkered a little around. Then I got stuck by the configuration of the server with e.g.  cacheDirectory.  Is there a specification what and how this information is to be sent to the server.
Maybe I've to send the information according to language_server_api.h
I haven't found anything in the Language Server Protocol specification.
Thanks for every help or advice,
Helmut

cquery pulls grabs cacheDirectory from the LSP initialize method. It's a non-standard configuration, so it ends up getting stashed down in initializeOptions like so:
{
"processId": null,
"rootUri": "file:///home/josh/Repos/cquery",
"initializationOptions": {
  "cacheDirectory": "file:///home/josh/.cache/cquery"
},
"capabilities": []
}

Thanks a lot for the quick answer!!!! Looks good, I'll give it a try.
Helmut

It works, happy new year!


Avoid joblib warning

Replace the from sklearn.externals.joblib import ... into from joblib import ... to avoid the following warning (that will eventually turn into an error with sklearn==0.23):
sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.

Thanks, this is great.
Could you look at the test and try to fix them. I dont think its related to this PR but would be awesome to bring them back to life.

Codecov Report

Merging #74 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #74      +/-   ##
==========================================
- Coverage   98.47%   98.47%   -0.01%
==========================================
  Files          18       18
  Lines        1312     1311       -1
==========================================
- Hits         1292     1291       -1
  Misses         20       20

Impacted Files
Coverage Δ

pyriemann/clustering.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 585f5f4...790162e. Read the comment docs.

Here it is.
I switched to travis virtualenv instead of conda as there are no hard binary dependencies that whould require it.


Change the default namespace

To reflect the change of the Governance model I'd like to adjust the default namespace and trim the "Ploeh" namespace prefix. That should perfectly fit the SemVer as we are releasing the major version. We made a similar decision in AutoFixture project, so I don't see too much issues with that.
@moodmosaic @adamchester @jwChung let me know if you have any concerns. Also confirm that you are fine, so I could proceed with that change and the v2 release.

Go for it 👍

Go for it
Also, sorry again I haven’t responded to all issues where I’m mentioned, still very busy :sadface:

Go for it.


[uwp] Frame.Navigate( typeof(non_xaml_class_derived_from_Page)) crashes w/ Native Toolchain

Describe the bug
When given a Non-XAML (pure code) class derived from Page, Frame.Navigate crashes when compiled using Native Toolchain.
Steps to reproduce the bug
USING SAMPLE PROJECT

Clone the following repo: https://github.com/baskren/NativeToolchain_NonXamlPage_crash.git
Run in Debug WITHOUT Native Toolchain
Click NOT A XAML PAGE button to navigate from XAML PAGE to NOT A XAML PAGE, successfully
Run in Debug WITH Native Toolchain or run in Release
Click NOT A XAML PAGE button to crash app.

FROM SCRATCH

Create a new UWP App
Create a new class file, named NoXaml.cs and derive the class NoXaml from Windows.UI.Xaml.Controls.Page
Add [Windows.UI.Xaml.Data.Bindable] attribute to the NoXaml class
In App.xaml.cs, replace rootFrame.Navigate(typeof(MainPage), e.Arguments); with rootFrame.Navigate(typeof(NoXaml), e.Arguments);
In the project's options, enable the Native Toolchain
Build, run, watch crash

Expected behavior
Frame.Navigate works with Non-XAML Pages.
Screenshots

NuGet package version
No response
Windows app type

[X] UWP
[ ] Win32

Device form factor
Desktop
Windows version
May 2021 Update (19043)
Additional context
Note that recommendation from #1949 was applied to test case

so @asklar closed a duplicate of this bug immediately (in a different project) but no one has responded in over a week.  Does this mean that I did not post either bug in the right place?  Does this mean that UWP is no longer being supported?

@RealTommyKlein do you know what not xaml page is missing here?

This is because NotXamlPage is missing an access modifier, so it's considered internal instead of public and inaccessible to Xaml in Release/.NET Native toolchain builds.  Changing the declaration to:
   [Windows.UI.Xaml.Data.Bindable]
   public class NotXamlPage : Page
   {
       ...
   }

Will fix it.  To make native toolchain vs. non-native toolchain and Debug vs. Release builds have the same behavior (so they would've both crashed instead of the Debug non-native toolchain scenario working), you can add this MSBuild property to your project file although the Designer/Hot Reload experience will be less robust:
  <PropertyGroup>
    <EnableTypeInfoReflection>false</EnableTypeInfoReflection>
  </PropertyGroup>


FIX: Don't hide the new messages indicator beside the channel header

Reverts discourse/discourse-chat#835
Isolated:

Drawer:

Mobile:

I realized  this also affected highlighted messages, so ended up moving all the scrolling logic inside the scrollToMessage function instead of having a separate one for the separator.


How to scrape in promotheus

Hello I'm trying to scrape this metrics-server-exporter, using prometheus outside kubernetes cluster
I already expose metrics-server-exporter as NodePort service
I have try prometheus yaml file from this link https://grafana.com/grafana/dashboards/12363
But prometheus not scraping the metrics, what should I do?
Here are my prometheus config,
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

scrape_configs:
  - job_name: 'prometheus-pushgateway'

    static_configs:
    - targets: ['myip:port']

  - job_name: 'prometheus-node-exporter'

    static_configs:
    - targets: ['myip:port']

  - job_name: 'prometheus-metrics-server-exporter'

    static_configs:
    - targets: ['myip:port']

    kubernetes_sd_configs:
    - role: endpoints
    relabel_configs:
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
      action: keep
      regex: true
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
      action: replace
      target_label: __scheme__
      regex: (https?)
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
      action: replace
      target_label: __metrics_path__
      regex: (.+)
    - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
      action: replace
      target_label: __address__
      regex: ([^:]+)(?::\d+)?;(\d+)
      replacement: $1:$2
    - action: labelmap
      regex: __meta_kubernetes_service_label_(.+)
    - source_labels: [__meta_kubernetes_namespace]
      action: replace
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_service_name]
      action: replace
      target_label: kubernetes_name
    - source_labels: [__meta_kubernetes_pod_name]
      action: replace
      target_label: kubernetes_pod_name
    - source_labels: [__meta_kubernetes_pod_name]
      action: replace
      target_label: pod

I only get this, only few metrics
HELP process_virtual_memory_bytes Virtual memory size in bytes.
TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 26853376.0
HELP process_resident_memory_bytes Resident memory size in bytes.
TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 21909504.0
HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
TYPE process_start_time_seconds gauge
process_start_time_seconds<PHONE_NUMBER>.65
HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.32
HELP process_open_fds Number of open file descriptors.
TYPE process_open_fds gauge
process_open_fds 7.0
HELP process_max_fds Maximum number of open file descriptors.
TYPE process_max_fds gauge
process_max_fds 1048576.0
HELP python_info Python platform information
TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="6",patchlevel="10",version="3.6.10"} 1.0
HELP kube_metrics_server_response_time Metrics Server API Response Time
TYPE kube_metrics_server_response_time gauge
kube_metrics_server_response_time{api_url="https://kubernetes.default.svc/metrics.k8s.io"} 0.076778
HELP kube_metrics_server_nodes_mem Metrics Server Nodes Memory
TYPE kube_metrics_server_nodes_mem gauge
HELP kube_metrics_server_nodes_cpu Metrics Server Nodes CPU
TYPE kube_metrics_server_nodes_cpu gauge
HELP kube_metrics_server_pods_mem Metrics Server Pods Memory
TYPE kube_metrics_server_pods_mem gauge
HELP kube_metrics_server_pods_cpu Metrics Server Pods CPU
TYPE kube_metrics_server_pods_cpu gauge

I only get this, only few metrics
HELP process_virtual_memory_bytes Virtual memory size in bytes.
TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 26853376.0
HELP process_resident_memory_bytes Resident memory size in bytes.
TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 21909504.0
HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
TYPE process_start_time_seconds gauge
process_start_time_seconds<PHONE_NUMBER>.65
HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.32
HELP process_open_fds Number of open file descriptors.
TYPE process_open_fds gauge
process_open_fds 7.0
HELP process_max_fds Maximum number of open file descriptors.
TYPE process_max_fds gauge
process_max_fds 1048576.0
HELP python_info Python platform information
TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="6",patchlevel="10",version="3.6.10"} 1.0
HELP kube_metrics_server_response_time Metrics Server API Response Time
TYPE kube_metrics_server_response_time gauge
kube_metrics_server_response_time{api_url="https://kubernetes.default.svc/metrics.k8s.io"} 0.076778
HELP kube_metrics_server_nodes_mem Metrics Server Nodes Memory
TYPE kube_metrics_server_nodes_mem gauge
HELP kube_metrics_server_nodes_cpu Metrics Server Nodes CPU
TYPE kube_metrics_server_nodes_cpu gauge
HELP kube_metrics_server_pods_mem Metrics Server Pods Memory
TYPE kube_metrics_server_pods_mem gauge
HELP kube_metrics_server_pods_cpu Metrics Server Pods CPU
TYPE kube_metrics_server_pods_cpu gauge


runtime: unix domain socket crash on darwin High Sierra 10.13.6, go 1.21.0

What version of Go are you using (go version)?

$ go version
go version go1.21.0 darwin/amd64

Does this issue reproduce with the latest release?
This is the latest
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE='auto'
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/jaten/Library/Caches/go-build'
GOENV='/Users/jaten/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/jaten/go/pkg/mod'
GOOS='darwin'
GOPATH='/Users/jaten/go'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go1.21.0'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go1.21.0/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD=''
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6s/zdc0hvvx7kqcglg5yqm3kl4r0000gn/T/go-build406580306=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?
I started using unix domain sockets on go1.21.0 on High Sierra. I observed once a sporadic crash.   This was not easy to reproduce, it happens only sporadically; probably some race.
I assume if High Sierra is no longer supported, that may be why... perhaps there was some runtime evolution of the code, and I'll just need to go back to an older Go release. But it would be nice to know if that is the case, or perhaps this really does indicate a bug in the darwin unix domain socket handling Go runtime code.
What did you expect to see?
Not a crash.
What did you see instead?
A crash a moment after starting a unix domain listener.

SIGSEGV: segmentation violation
PC=0x100051e37 m=0 sigcode=1

goroutine 0 [idle]:
runtime.(*sigctxt).sigcode(...)
/usr/local/go1.21.0/src/runtime/signal_darwin_amd64.go:43
runtime.(*sigctxt).sigFromUser(...)
/usr/local/go1.21.0/src/runtime/os_unix_nonlinux.go:14
runtime.sighandler(0x1c, 0x100a783a0?, 0x7ffeefbfd360?, 0x100a777a0)
/usr/local/go1.21.0/src/runtime/signal_unix.go:676 +0x177 fp=0x7ffeefbfd338 sp=0x7ffeefbfd2d0 pc=0x100051e37
runtime.sigtrampgo(0x1c, 0x8300, 0x820000008303)
/usr/local/go1.21.0/src/runtime/signal_unix.go:490 +0x13c fp=0x7ffeefbfd3b0 sp=0x7ffeefbfd338 pc=0x10005181c
runtime.sigtramp()
/usr/local/go1.21.0/src/runtime/sys_darwin_amd64.s:189 +0x46 fp=0x7ffeefbfd400 sp=0x7ffeefbfd3b0 pc=0x100071486
the place where this crash occurred:
file runtime/os_unix_nonlinux.go,  Go version 1.21.0 on darwin/amd64
package runtime

// sigFromUser reports whether the signal was sent because of a call
// to kill.
//
//go:nosplit
func (c *sigctxt) sigFromUser() bool {
    return c.sigcode() == _SI_USER  // line 14 (segfault) is here
}

caller was
file: runtime/signal_unix.go, Go version 1.21.0 on darwin/amd64

    flags := int32(_SigThrow)
    if sig < uint32(len(sigtable)) {
        flags = sigtable[sig].flags
    }
    if !c.sigFromUser() && flags&_SigPanic != 0 && (gp.throwsplit || gp != mp.curg) {  // segfault here, line 676
        // We can't safely sigpanic because it may grow the
        // stack. Abort in the signal handler instead.
        //
        // Also don't inject a sigpanic if we are not on a
        // user G stack. Either we're in the runtime, or we're
        // running C code. Either way we cannot recover.
        flags = _SigThrow
    }

here is the unix domain code; it creates a simple cooperative lock between processes, to avoid
both working on the same file
package main

import (
	"fmt"
	"net"
	"os"
	"path/filepath"
	"strings"
	"time"
)

// UDLock uses a unix-domain socket to lock
// a file to prevent two cooperating processing from
// using the same file.
type UDLock struct {
	Path     string
	lsn      net.Listener
	Finished chan struct{}
}

// NewUDLock obtains a lock based on path. If the
// lock was obtained, a nil err will be returned.
// path will have ".lock" appended if it does not
// already have it. Call lock.Close() to release
// the lock.
func NewUDLock(path string) (lock *UDLock, err error) {

	if strings.HasSuffix(path, ".lock") {
		// okay as is.
	} else {
		path = path + ".lock"
	}

	path2, err0 := filepath.Abs(path)
	panicOn(err0)
	path = path2

	staleLock := false
	if FileExists(path) {
		// see if process at this unix domain socket is
		// still alive and responding to lock queries, or
		// if the .lock file is just stale and we can
		// grab it.
		conn, err1 := net.Dial("unix", path)
		//vv("dial err1 = '%v' on path '%v'", err1, path)
		if err1 == nil {
			var buf [4096]byte
			err2 := conn.SetReadDeadline(time.Now().Add(3 * time.Second))
			panicOn(err2)
			n, err3 := conn.Read(buf[:])
			if err3 != nil {
				if netErr, ok := err3.(net.Error); ok && netErr.Timeout() {
					staleLock = true
				}
			}
			if staleLock {
				os.Remove(path)
			} else {
				err = fmt.Errorf("path '%v' has a live lock already: '%v'", path, string(buf[:n]))
				return
			}
		} else {
			//vv("removing stale socket file b/c could not contact the process holding it")
			os.Remove(path)
		}
	}
	lsn, err2 := net.Listen("unix", path)
	if err2 != nil {
		return nil, err2
	}

	lock = &UDLock{
		Path:     path,
		lsn:      lsn,
		Finished: make(chan struct{}),
	}
	lock.start()
	return lock, nil
}

// Close releases the lock
func (lock *UDLock) Close() {
	conn, err1 := net.DialTimeout("unix", lock.Path, 5*time.Second)

	if err1 == nil {
		conn.SetReadDeadline(time.Now().Add(time.Second))
		var buf [4096]byte
		conn.Read(buf[:])

		conn.Write([]byte("shutdown"))
		<-lock.Finished
	}
	os.Remove(lock.Path)
}

func (lock *UDLock) start() {
	go func() {
		defer close(lock.Finished)
		for {
			//vv("top of UDLock.start() loop")

			// Accept new connections, tell them the lock is held,
			// and check if we are shutting down
			conn, err := lock.lsn.Accept()
			if err != nil {
				//vv("lsn.Accept error (probably closed due to shutdown): '%v'", err)
				return
			}

			go func(conn net.Conn) {
				//vv("Client connected [%s]", conn.RemoteAddr().Network())

				err = conn.SetWriteDeadline(time.Now().Add(5 * time.Second))
				panicOn(err)
				fmt.Fprintf(conn, "'%v' locked! by pid:%v", lock.Path, os.Getpid())

				// check if is request to shutdown
				err = conn.SetReadDeadline(time.Now().Add(5 * time.Second))
				panicOn(err)
				var buf [4096]byte
				n, err := conn.Read(buf[:])
				if err == nil && n >= len("shutdown") {
					if string(buf[:len("shutdown")]) == "shutdown" {
						conn.Close()
						//vv("exit lock listening loop by closing lsn.")
						lock.lsn.Close()
						return
					}
				}
				conn.Close()
			}(conn)
		}
	}()
}

If the stack trace is correct this is a SIGWINCH signal, which is sent when the terminal window size changes.  Did you happen to resize the terminal window?  Then while trying to handle that signal, the process gets a SIGSEGV, which kills it.  But I don't know why it would get a SIGSEGV there.

Hm... Possibly related to https://github.com/golang/go/issues/60449? Though, if it's signal specific it might unrelated; that's more general memory corruption we're seeing. How easy is this to reproduce?

If the stack trace is correct this is a SIGWINCH signal, which is sent when the terminal window size changes. Did you happen to resize the terminal window? Then while trying to handle that signal, the process gets a SIGSEGV, which kills it. But I don't know why it would get a SIGSEGV there.

@ianlancetaylor
I didn't manually try to resize the window, but I do remember the terminal "blinking", so perhaps there was a terminal resize invoked by some code... the Go process loads R 4.0.5 as a library, which, being a huge old crusty C codebase, might very well be doing something with the terminal... or gnu readline, or libedit, might be doing stuff. The process tells R to open a graphics window, so XQuartz on darwin gets launched and a graphics window attempts to be opened... and OSX tries to bring that graphics window to the top of the desktop, which perhaps sends a signal to the terminal (which is Iterm2 build 3.1.5) a  Any number components might have done stuff to the terminal, I guess.

How easy is this to reproduce?

@mknyszek
I was only able to see this once. So, very difficult to reproduce.

Ah hah. Thanks Ian for the idea about SIGWINCH. I was able to reproduce a crash again by resizing the terminal window. So perhaps this is unrelated to the unix domain socket stuff.
Also I realized that because the R library (loaded by the main Go process and called by cgo) loads a shared object (.dylib) that also contains Go code, there are two copies of the Go runtime involved (one from the main process, one embedded in the .dylib), and I cannot tell up front which one is doing the crashing.
I'm attaching the full goroutine stack dump from the crash caused by resizing the ITerm window on darwin.
full.stack2.txt

Looking at the source for libedit, which is what I'm linking the R library against,
https://opensource.apple.com/source/libedit/libedit-31/src/sig.c.auto.html
I notice that it does not use the SA_ONSTACK flag when taking over signals while it reads a command line.
So I think I can see a race where, between the time the libedit reading a command line returns, and before my C code can restore the SA_ONSTACK flag to the signal handlers, here
https://github.com/glycerine/embedr/blob/master/cpp/embedr.cpp#L349
and here
https://github.com/glycerine/embedr/blob/master/cpp/embedr.cpp#L378
a signal could get through to the top/main Go runtime that does not have SA_ONSTACK set.

@ianlancetaylor Is there a way to have the Go runtime not crash if SA_ONSTACK is not set? making sure it is always set is super tricky when using these big old C systems like R and libedit...
(referring to

If the non-Go code installs any signal handlers, it must use the SA_ONSTACK flag with sigaction. Failing to do so is likely to cause the program to crash if the signal is received. Go programs routinely run with a limited stack, and therefore set up an alternate signal stack.

from https://pkg.go.dev/os/signal
)

possibly related: https://github.com/golang/go/issues/22805

I'm trying the following to keep signals away from the Host (main) Go runtime. I'd appreciate any thoughts about whether this would be expected to work... or is there still a subtle race here?
right after crossing the CGO border into running C code, I save the set of signal handlers (which would include any host Go runtime signal handlers for growing stacks), and then null out all the signal handlers:
https://github.com/glycerine/embedr/blob/master/cpp/embedr.cpp#L176
https://github.com/glycerine/embedr/blob/master/cpp/embedr.cpp#L158
So if the libedit/libreadline code that sets SIGWINCH does stuff, the host Go runtime should never see it.
Then I restore the host Go runtime signal handlers before returning from the CGO call.
https://github.com/glycerine/embedr/blob/master/cpp/embedr.cpp#L210
I think this should prevent the kernel from ever giving the host Go runtime a signal callback, right?
I'm still puzzling about how to protect the guest Go runtime (inside the .so/DLL/.dylib) from getting signals. Thinking on it. Any suggestions welcome.  I think it has to get signals in order to grow its stacks, no? (or maybe not? that would be greatly simplifying)...
I can save the guest Go runtime signal handlers after it has initialized, but there are alot of calls back and forth between R and Go guest code in the DLL, where I don't have a way to intercept in C and make sure the guest runtime signal handlers are installled... without modifying the C code for R, which would be painful.

So reading about -buildmode=c-shared in https://pkg.go.dev/os/signal, which would apply to the guest Go runtime in the DLL:

Non-Go programs that call Go code:
...
For the synchronous signals and SIGPIPE, the Go runtime will install a signal handler. It will save any existing signal handler. If a synchronous signal arrives while executing non-Go code, the Go runtime will invoke the existing signal handler instead of the Go signal handler.

This seems to say that I would need get the synchronous signals and SIGPIPE back in place for the guest Go runtime to be 100% happy.
If this is an accurate conclusion, it seems like some kind of callback mechanism to let my code restore to the guest Go runtime code its proper/expected signal handlers without having to modify the calling C code would be highly useful (to put it mildly, really we want to avoid random sporadic crashes to our working programs in most all cases).

Anywho, its pretty clear to me this crash was not a Go bug but rather a coding mistake on my part. Feel free to close this issue.  But if you all do have advice on how to handle the signals for the guest c-shared Go runtime, I would be grateful for guidance. Thank you.

Thanks for the thorough investigation.
When any Go code is used in a process, all signals must have the SA_ONSTACK flag set.  The Go runtime will attempt to make this happen when it is initialized.  However, if signal handlers are installed after the Go runtime is initialized, then there is nothing to force the SA_ONSTACK flag to be set.
If SA_ONSTACK is not set, and a signal is delivered to a thread while it is running Go code, it will generally crash.    Deliver a signal to a Go thread will often require more stack space than the Go thread has available.  This will lead to a stack overrun and a crash.

Is there a way to have the Go runtime not crash if SA_ONSTACK is not set?

There is not.  That said, since you mention it, we could consider having some knob for this somewhere.  Though I don't know where.  Turning the knob on would tend to waste a lot more memory on goroutine stacks, but it might be an acceptable tradeoff for a C program calling Go code that doesn't want to run very many goroutines.  That should be a different issue, though.
Assuming you can't change the C code that you are calling, one possible fix for your program might be to have a hook after the signal handlers are installed that just sets the SA_ONSTACK flag for them.  You can this by calling sigaction to fetch the current signal status, then, if SA_ONSTACK is not set, call sigaction again with the same values only with SA_ONSTACK set.
In any case it doesn't sound like there is anything to change in the Go for this issue, so closing.  Thanks again.


[BUG] PSACT View not displaying Description field correctly

As per https://github.com/rancher/dashboard/issues/8335

Closing this as a duplicate of https://github.com/rancher/dashboard/issues/8335.


Deleting a Virtual Machine

Hi
when i specify the parameters for a virtual machine, a storage blob gets created for the disk. When the VM is Deleted, the storage blob doesn't get deleted
Is Delete the correct method to call for VMs? Or should we use something like Deallocate first?
Paul

@stack72 Good question. We don't know. You may try filing the same question against the related Swagger. If you do not get a response, re-raise it here and we'll see what we can learn.

@stack72 In Azure Resource Manager mode (Microsoft.Compute resource provider), there is no option to delete the blob (at least as of today). This was a little bit disappointing for me as well when I figured it out.
I advise inspecting the VM first, figuring out its OS disk, deleting the VM itself and finally deleting its OS disk.

Hi @ahmetalpbalkan
thanks for the response here. So I don't think it's quite that simple. All we have for a VHD is a uri - are you suggesting we try and split the URI to find the resource group and the storage account and the container name? That could work (but it's very hacky). We need all of those pieces to delete a blob.
I guess, for me, the functionality I would expect would be to be able to pass a flag to the os_disk, something like delete_on_termination and then Azure would clean that up for me.
P.


Ability to send task types to k8s or worker task runner

Description
Follow on to #14918
Some cluster administrators may want to send particular tasks to the worker task runner instead of k8sTaskRunner, as the task startup time is usually faster on the worker task runner system.
An example use case is high throughput streaming ingestion where a long pod startup time can result in user visible lag. To get around this, an administrator could configure the cluster to have enough idle capacity to reasonably handle the high throughput stream and configure all other tasks to be handled by the k8s task runner while the streaming tasks are handled by the worker task runner.
The other idea I explored was implementing something like WorkerSelectStrategy for the KubernetesAndWorkerTaskRunner to provide more flexibility. But I was not certain whether or not the added complexity of this implementation was worth it, so I went with the simpler approach
Release note

This PR has:

[ ] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

Thought about it a bit. The approach that extends WorkerSelectStrategy indeed sounds better since it's much more extensible and flexible than the current design. What's the complexity involved?

I submitted another PR https://github.com/apache/druid/pull/15196 to use a more extensible task runner selector strategy for this work. Please review that one instead.

Thanks @YongGang !


Outputs the unsafe redirect warning message even if the default redirectUri is used

Doesn't seem to happen anymore.


UploadSet issue with "Upload" button

OpenUI5 version: 1.112.0 and later
Browser/version (+device/version): Google Chrome Version 108.0.5359.124 (Official Build) (arm64)
Any other tested browsers/devices(OK/FAIL):
URL (minimal example if possible):
User/password (if required and possible - do not post any confidential information here):
Steps to reproduce the problem:

Open the uploadSet sample in the official documentation.
Delete all of the files.
There are two "Upload" buttons (see screenshot). This looks redundant. In previous versions there was only one "Upload" button.
Set the "uploadButtonInvisible" prop to true. The top button disappears, the bottom one does not. If you click on the bottom button, nothing happens.

What is the expected result?
Should be only one "upload" button.
What happens instead?
There are two "Upload" buttons. When uploadButtonInvisible is true, the top button disappears, but the bottom one does not. If you click on the bottom button, nothing happens.
Any other information? (attach screenshot if possible)

Hello @Vir45,
Thank you for sharing this finding. I've created an internal incident with ID:<PHONE_NUMBER>. The status of the issue will be updated here in GitHub.
Regards,
Ivaylo


A way to pass a custom comparator to provider families

Is your feature request related to a problem? Please describe.
We want to create a provider, let us call it getFoosByIds. It is a family provider, ideally just taking a List<String>
This provider may be called multiple times with separate reference Lists with the same content, in which case separate providers will be instantiated doing the exact same logic, because Lists. are compared by reference, not by content
We seem to have two primary options:

Join the list to a string, to use as family, and split the string back into a List inside the provider
or
Create a specialized object with a field containing the List, and override the compare function to compare on values inside the list.

Neither seem very sleek.
Describe the solution you'd like
It would be nice, if we could pass a compare function, perhaps a lambda like (a, b) => boolean
Describe alternatives you've considered
I am not sure how to implement the compare function in a sleek way. Could be a named parameter reflected to the name of the family parameter for example.
Additional context
N/A

I don't think that's really something I want to implement.
I'd suggest using a custom List implementation. There are a few packages with it.

Yeah - i do not see a way to make a pretty implementation of that anyway


fix: flag check of node logs command and incorrect download source

Description
This pull request changes the following:

Fixed unparsed node alias node assigned error
Fixed the incorrect log download source directory
Added a unit check to check if download is working

Related Issues

Closes #847

:tada: This PR is included in version 0.32.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Does not seams to work under Godot 4 beta 10

When trying to add a mesh to multimesh nothing show up, but i managed to get this error "ERROR: Condition "instance_count > 0" is true."

Seems to be working fine for me.
I think this is caused when you delete and recreate the multimesh. Don't delete the multimesh, instead use the multimesh that is created automatically when you add the node.
Try recreating the node :)

Thx, i think the multimesh is bugged with my AMD Radeon.

I've closed this issue. If you have questions or found a bug, you can reopen this or create a new issue.


[TCE-1040] Default value for parameter 'registry-address' breaking default behavior

It looks like the new registry-address parameter just introduced in #129 has broken the default behavior of this action. Since the value is set to "gh" by default - it now tries to pull docker images from ghcodacy namespace rather than codacy
Logs from an actions run using codacy/codacy-analysis-cli-action@v4
Status: Downloaded newer image for codacy/codacy-analysis-cli:7.9.10
07/11 12:16:40 ERROR c.c.a.c.a.CodacyPluginsAnalyser:31 - Failed analysis for pmd
java.lang.Throwable:
Docker exited with code 125
stdout:
stderr: Unable to find image 'ghcodacy/codacy-pmd:3.13.13' locally
docker: Error response from daemon: pull access denied for ghcodacy/codacy-pmd, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

Unfortunately this isn't avoidable since if we pass in an empty value to the action, it reverts back to the default

Just pushed a PR with a fix. Thanks for the heads up

@hjrocha just confirmed it's fixed for us. Appreciate the quick turnaround


Bumblebee fails to load gpt models in main.

In bumblebee main (73c6e5f5d90aa625eda7247e60c2f63198a5c154), the following code is sufficient to fail:
Nx.default_backend({EXLA.Backend, client: :host})
auth_token = System.get_env("HF_AUTH_TOKEN")
{:ok, model} = Bumblebee.load_model({:hf, "meta-llama/Llama-2-7b-chat-hf", auth_token: auth_token})

with the following error:
** (RuntimeError) conversion failed, invalid format for "rope_scaling", got: nil

same code works with "~> 0.4.2".
nx is pinned to 0.6

Fixed in 3cc0ff9d9a57839a249af81f24d0b58f6b56261d.


Add containerd support to "--local" mode

Running ko with --local is super convenient 💯  Unfortunately it seems to works only with Docker daemons (correct me if I'm wrong).
I recently switched to k3s in my local setup, which does not rely on a Docker daemon. Instead, k3s exposes a containerd GRPC server at /run/k3s/containerd/containerd.sock.
What do you think about supporting publishing images directly to containerd, and enabling the feature via and environment variable, e.g. KO_CONTAINERD_HOST?

Ref #149

I'm certainly open to it.

This issue has suddenly become very relevant :)

Is this as simple as containerd.New and then client.Import? That shouldn't be too hard to add a new publisher and wire it up with flags.

/reopen
Still relevant AFAIK

A simple (albeit inefficient) workaround is to export a tarball using --tarball=foo.tar --push=false and import the resulting image into k3s/containerd using k3s ctr images import.
Simple example usage: https://github.com/monogon-dev/NetMeta/blob/94fcccddcb03faef99ce46ea4738b5d14804ffb8/scripts/build_containers.sh#L17

A more efficient strategy might be to just run a very simple registry implementation in-process and have ko act as the registry? Serializing to tarballs is annoying because you lose deduplication...

A more efficient strategy might be to just run a very simple registry implementation in-process and have ko act as the registry? Serializing to tarballs is annoying because you lose deduplication...

It should be possible to just write the layers straight to the containerd local store, right? Just like a Docker build does.

It should be possible to just write the layers straight to the containerd local store, right?

Definitely, but IMO it would be really nice if we could avoid it. I don't really want to link the containerd client library.
OTOH, there are cases where containerd might be on another machine, and my idea doesn't work if the target runtime is on a different network.


Update roadie dependency

I was getting this issue https://github.com/Mange/roadie/issues/106, so I needed an update.

Hmm... this didn't fix the issue for me. Scratch this PR then, I guess it's not quite that simple.


"Lastest" tag is not being updated

Hi,
Big fan of this project 🥇
I've noticed that new versions are being released but not being marked as latest. Is this because the newer versions are not considered stable yet?
The reason I ask is that I've got some automation that looks at the latest tag and is triggered to update my builds when a new latest is released.
Thanks,
Ronald

@roribio Is that on the docker image?

@adamw My apologies for not being clearer before, I'm referring to GitHub releases.
If you go to the releases page, you can see that the tag release-0.13.8 was promoted to "Latest release". After that, new versions have been released but not promoted to "Latest release". Coincidentally, that is also the last tag to follow the release-X.Y.Z naming pattern.

Ah :) TBH, no idea how GH determines the "latest" tag. I've switched to an automated release process which uses a different naming convention for tags, hence the change.
The latest release is definitely 0.14.6

Ah, I suppose that's because earlier I used to do "GitHub releases" based on the tag - now it's just a tag.

Thanks for pointing me in the right direction. With that I've managed to cobble something together and submit a PR: https://github.com/adamw/elasticmq/pull/157.

With some adjustments, seems to work. Thanks!

Awesome! Thank you :)


WinError 10060 - Timeout

Hello,
there are no real steps to reproduce the error. I'm just getting it about 80% of the time on every single command. As I would like to execute a sequence of correlated commands in a batch, this is getting very tiring because I cannot and I have to insert the commands one by one, many times until they have success.
I understand that this may not be a direct GAM problem, meaning that for some reason it may depend on the answer from the server being very slow.
What I'm asking is if there is a way to raise the timeout from the GAM side so that commands can complete.
Hope to have been clear enough.
Thanks

The timeout is already very high, I don't think raising it higher will help here.
What you might want to look at is running GAM itself on a GCP VM or Google Cloud Shell so that it's connectivity is no longer an issue.


Fix repo description

The description of the repo on Github now says:
Ruby gem for retrieving banks account and balance.
I'd change it to:
Ruby gem to extract balance and transactions from banks.
but I don't have privileges to do so. @ismaGNU can you do it? :D

@javiercr Sorry, but I think I have forgotten this issue :(. Fixed now!


Blazor and Office 365 auth

Issue Title
Blazor rc1 and Office 365 auth doesn't work
General
I just created the project Blazor Wasm with single org authentication and show the CORS error

@danroth27 can you help? Should we transfer this to aspnetcore?


WIP Rtl fixes

Hey guys. In relation to issue 1227
I believe this is not a full PR, but that's sure a start. If you can comment on points listed below I think I'll be able to finish it. Or maybe you'll crack it fast and efficient with this tip.

Fixes initial load and enabling\disabling of arrows for sliders with slidesToScroll: 1
I didn't check any other modes (vertical, slideToScroll > 1, center mode ...)
Architecture \ code style : I kept current coding style, but if condition now look ugly. Maybe use smth like let disabled and make several if-else statements that changed it to true and after that a check for value of this variable. I.e. here
Architecture \ code style : maybe check for previous arrow should be moved to a separate method, same as for the nextArrow?
I'm not sure the places I've changed are the right places to change. Maybe the check should be somewhere else, changing currentSlide value for rtl cases or something like that.
How do I proceed from here? :)

just noticed some diffs with brackets styling, sorry for that. can re-submit if needed.
also the fix surely doesn't work with variableWidth = true

Can this PR be merged now? Facing the same issue mentioned in 1227

@mcfoton How to get rtl working with variableWidth? I mean, the extra spacing on the right side of first item.

@mcfoton How to get rtl working with variableWidth? I mean, the extra spacing on the right side of first item.
@sanishkr Never got it working :(


JdbcBlobstore use null as QueryDSL configuration.

JdbcBlobstore#resolveQuerydslConfiguration method returns with null.

Fixed in 2.0.1


Stop pAIs from being able to wipe each other/do other odd things

About the PR
So apparently having no hands wasn't enough to stop them from being able to interact.
Changelog
:cl:

fix: pAIs can no longer wipe each other or do other odd things. You can still wipe yourself with the ghost command.

before any further stuff, should I also make them auto-rename to "XYZ's pAI"?

"GitHub Actions has encountered an internal error when running your job."

before any further stuff, should I also make them auto-rename to "XYZ's pAI"?

i say yes

well I ended up doing it anyway as a GH test-runner bug metagaming moment


Updated TopNSortedList data structure to support duplicate values

Motivation and Context

The change modifies TopNSortedList data structure to handle duplicate values.
It fixes the scenario of getting nearest matches from VolatileMemoryStore when matches have the same score.

Description

Current version of TopNSortedList data structure doesn't support duplicate values. Instead of mapping score to a value, we will map score to the list of values, which have the same score. To obtain flatten list as a result, ToTuples method was added.
Contribution Checklist

[x] The code builds clean without any errors or warnings
[x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
[x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
[x] All unit tests pass, and I have added new tests where possible
[x] I didn't break anyone :smile:

I was a bit concerned, thinking this logic applied to all vector dbs, but if it's only for the in memory, LGTM, plus it has unit tests, love that :-)   On a side note, for demos and experimentation, looking forward to expand our support for vectors over sqlite, elasticsearch, etc.


CI with Travis

I found this:
https://docs.travis-ci.com/user/languages/haskell/
should be nice. It already uses Cabal

https://chshersh.github.io/posts/2019-02-25-haskell-travis
maybe more readable


Add ftdetect for CMake filetypes.

Add ftdetect directory for detecting CMake filetypes.

It does set the correct ft for CMakeLists.txt but not for *.cmake files such as cmake_install.cmake and Makefile.cmake which I found myself looking into on a few occasions.
I'll update the PR to remove setting the ft again for CMakeLists.txt files.


some questions about rnn

hello.
I read your paper and am looking into your codes.
And got some questions about rnn.
in rnn_one_hot.py : _prepare_networks()

at your code :
if not self.use_movies_features:
l_recurrent = self.recurrent_layer(self.l_in, self.l_mask, true_input_size = self.n_items + self.n_optional_features(), only_return_final=True)
there's NOT in if condition.
why did you set true_input_size like above when use_movies_features is false?

self.recurrent_layer() calls __ call __() of recurrent_layers.py
It seems that it only returns 1 layer - prev_layer, though you write for loop in that part.
I think it returns multiple layers when RecurrentLayers.layers is set like 100-50-50 like your example
Does it work as you intended?

what does l_last_slice do?
Should it exist?

TY for your help.

Hi,
Regarding your first question:
Firstly I have to confess that all the parts about movies_features, optional_features, etc. is kind of legacy code that I used in my experiments to exploit extra information from the movielens dataset, but I didn't want it in this public repo because it worked only on movielens. I removed some of it from the code, but some pieces stayed in (partly due to laziness, and partly because I hope to be able to generalize it to any dataset at some point).
That being said, here is what's happening in the lines you pointed to: using the argument "true_input_size" when constructing the recurrent layer will cause the recurrent layer to accept a sparse encoding  of the inputs. Unfortunately, in my implementation all the inputs must have the same number of non-zero element (e.g. one for the movie id, one for the rating and one for the user age = 3 non-zero element). This doesn't work when we use the "movies_features" of movielens, because those features are actually movies genres, and a movie can have any number of genres, which means that the number of non-zero elements in the input is not the same for every movie. Therefore when you use those movie features the recurrent layer will use a dense encoding of the input (you can see this difference reflected in the _get_features method of the RNNBase class).
This could have been avoided with a more flexible type of sparse encoding, but it required more involved modification of the Lasagne recurrent layers.
Second question:
self.recurrent_layer() does work as intended. It indeed returns only one layer (the last one), but this layer has a reference to its input layer, which has a reference to its input layer, and so on. So by returning only the last layer I actually return the whole network.
Third one:
Indeed, l_last_slice is exactly the same as l_recurrent and I could have simply used l_recurrent.
This is a relic of a time when they referred to different things, but I should clean that up now.
I hope this helps !

Thanks for reply!
Let me ask a few more questions.

All parameters of networks are shared for all users in each set, is it right?

And the training, testing, predicting process are conducted by batch_size * users?

The number of movies each user rated will be different according to users.
It needs the network process input with various length.
If one's rating record is shorter, does the network recurrents less?

Does network predict only 1 time(k movies) per each user?
If not, the movie with the largest softmax value becomes next input?

I want to confirm some details.
TY.

Hi,

that's right
I'm not sure what you mean by that. The training uses mini-batches whose size is set by batch_size. One epoch consist in one loop over all the users. Does that help ?
In Lasagne, RNNs use a mask, i.e. an additional input consisting of ones and zeros, with the zeros indicating that the corresponding step should be ignored. So the max length is the length of the mask, but any smaller length is possible.
It ranks all the items after the last step, and recommends the top k. I tried feeding the largest item as next input instead to observe which are the next k predicted items, but it gives worse results.

Thanks for your reply.
I'm here to ask another question.
In rnn-base.py, your code made input not in form of one-hot encoding. It was like [[23], [35], [57]...]. And the target is like [[60]]. In this case, how does the Theano.categorical_crossentropy calculate error? Does it do automatically same as if the input, target are encoded in one-hot?
I can figure out how to calculate error with CCE when the input, target are encoded as one-hot. But can't imagine when they are not encoded. And there's no parameter to determine type of encoding.
And how about there are multiple movies in target? What happens in that case?
Thanks.

Hi,
It's sad that a month has gone but didn't get any reply from you.
I have another question.
I tested with hinge loss function and got 11.07% sps for netflix dataset.
Also other metrics were much worse than those I got when used cce.
Did I get right result?
Or hasn't you completed with multi target loss function?
It would help me a lot if you tell me the result you got with hinge loss.
Thanks.

Hi,
Sorry for the late reply.
The categorical cross-entropy function of theano can indeed deal with both one-hot encodding and as a list of index. http://deeplearning.net/software/theano/library/tensor/nnet/nnet.html#theano.tensor.nnet.nnet.categorical_crossentropy
If you want to use multiple targets you have to use the hinge loss, but it is harder to train.
Concerning the results that you get, did you tune properly the learning rate and used early stopping to avoid overfitting ?
Try Adagrad with a learning rate of 0.1 or Adam with its default parameters.
I'll see if I can upload a model that I trained.
Cheers,
Robin

Hi,
It would help me a lot if you upload models you trained.
The more, the better.
Particularly, I want to check cce and hinge loss models for netflix dataset.

Does it take more time to train?
Thanks

Yes it tends to take more time with the hinge loss.

Hi,
Is it able to upload the models you performed experiment with?
The more kinds of models - RNN CCE, hinge, BPR-MF, UKNN, etc, -  the better.

Hi,
I tried to add the data and the models to the repo using git lfs, but I couldn't make it work so far.
So in the meantime you can get some models there: http://iridia.ulb.ac.be/~rdevooght/netflix.tar.gz

Hi, the data you uploaded was greatly helpful.
Moreover, I found that you trained your model with max_length 30 on netflix dataset.
Did you also applied same max_length parameter on movielens 1m dataset?
And I would like to know that if the model which made the results on your paper, was obtained when the validation performance was best or the model was chosen among multiple candidate models.
Thanks for your help!

Hi,
Indeed I used max_length = 30 on movielens as well.
It is a rather arbitrary choice, but I observed that using longer sequences barely affects the results, while slowing down the training.
The parameters of the models where tuned on a validation set using a random parameter search. We kept the parameters that optimised the sps@10.
The test is made with models trained using those "optimal" parameters + early stopping based on the validation set.

Oh, thanks for answering right away.
Sorry for questioning again, but did you removed some rare items and users who saw few movies during preprocessing?
And I think that selecting users randomly during preprocessing step could affect the model's performance.
I wrote this comment as I keep failing to get a model with performance same as the one in your paper unfortunately.
Thanks!

You can find the precise datasets and the training/validation/test splits that we used here: iridia.ulb.ac.be/~rdevooght/rnn_cf_data.zip

Hi, I've just read your recent paper and got a question.
Have you tried Recall@20, MRR@20 on RSC15 data with your model?
It would be great to see comparison between your model and the one in "Improved RNN for Session-based Recommendations" and I thought you might did it.
Please let me know the results made by your model in the same setting(metric) as "Improved RNN for Session-based Recommendations" if you did it.
Or I can do it.

Well, I'm training rnn-cce model for RSC15 data.
It's been 29 epochs, 5 days so far but the highest sps marked is 0.32 and highest recall is about 0.26.
Could you give me any advice?


Switched from pretty-quick to lint-stage

Current dependencies on/for this PR:

main

PR #345   👈

This comment was auto-generated by Graphite.


Epic: investigate VM boot time and ideally decrease it

Motivation
We have pools of pre-created computes to amortize k8s api latency and vm boot latency. This is how the current boot time distribution looks like:

in case of pool miss end user will experience bigger start latency. Since we are moving everything to VMs soon it would be good to improve VM start speed.
DoD
Understand where time is actually spend during the boot. Fireckracker start times are below 1s, so we should understand what are the main reasons for our start times to be slower and whether there are any easy improvements that we can do right away.
So for this task DoD is a text explaining the difference in boot times and list of suggestion on how to improve them without losing ability to scale and live migrate VM's.
Implementation ideas

start firecracker with their default kernel/image
start firecracker with our image -- will it be slower? how much?
look into qemu-microvm

Tasks
- [ ] ...

Other related tasks, Epics, and links

@kelvich

Fireckracker start times are below 1s

I'd like to clarify what exactly we mean by start time. Time to boot the kernel and start the init process (pid 1)? Our VMs do this in about 0.4s (proof link), which I don't think is the major overhead that VMs have. Need to understand why startup processes inside a VM are slower than in a Pod.

Earlier, I drew a workflow diagram to show what happens when we create a VM resource object. There other steps before getting into Kubernetes API server, which Stas has mentioned in his Slack message.

Based on @cicdteam's comment, this is probably due to the communication between the components involved in the process. Though still we have to investigate why k8s-neonvm - no pool is much slower than k8s-pod no pool.
As something for the future, to deal with this set of issues and generally improve the observability in our system, it may worth to invest in distributed tracing platforms, such as Jaeger.

@shayanh
The workflow diagram for the pod provisioner is almost the same. Just Deployment instead of neonvm and no step named neonvm-runner (where we run QEMU).
We can try a test like this - run NGINX as k8s deployment and poll it to see when it responds to an http request. Do the same with a neonvm and compare results.  I think the latencies will be almost the same.

I took a particularly long start_compute1, taking 36.5s and investigated it:
56:54 - operation's created_at 1
56:55 - Creating a new Pod 2
56:56 - Started container init-rootdisk 2
56:58 - Created container sysct 2
56:59 - Started container sysctl 2
57:04 - Created container compute-init 2
57:16 - Started container compute-init 2
57:17 - Created container neonvm-runner 2
57:18 - Started container neonvm-runner 2
57:18 - First dmesg entry 3
57:19 - Error opening config file "postgres_exporter.yml" 3 (is this a separate issue?)
57:19 - database system is ready to accept connections 3
57:20 - compute start finished 3
57:20 - connection authorized 3
... (investigate deeper)
57:25 -  Plug one more Memory module into VM 2
57:30 - updated_at 1
57:34 - sending powerdown command to QEMU
Some on-the-spot conclusions:

This validates the claim that QEMU start is negligible
Userspace inside the VM also starts fast
A main bulk of time is spend starting auxiliary containers

Further actions:

Verify if this picture reproduces in multiple cases
Investigate deeper what happens between 57:20 (compute start finished) and 57:30 (VM is ready to accept connections). Although even the reduced time is quite high (26s vs full 36s)
Look deeper into those auxiliary containers

Startup times by different provisioner.
 provisioner | bucket |   range   | freq |              bar
-------------+--------+-----------+------+--------------------------------
 k8s-neonvm  |      3 | [7,9)     |   86 | ■■■■
 k8s-neonvm  |      4 | [9,12)    |  469 | ■■■■■■■■■■■■■■■■■■■■■
 k8s-neonvm  |      5 | [12,15)   |  170 | ■■■■■■■■
 k8s-neonvm  |      6 | [15,18)   |   91 | ■■■■
 k8s-neonvm  |      7 | [18,21)   |   96 | ■■■■
 k8s-neonvm  |      8 | [21,24)   |   82 | ■■■■
 k8s-neonvm  |      9 | [24,27)   |   88 | ■■■■
 k8s-neonvm  |     10 | [27,30)   |   61 | ■■■
 k8s-neonvm  |     11 | [30,33)   |   54 | ■■
 k8s-neonvm  |     12 | [33,36)   |   47 | ■■
 k8s-neonvm  |     13 | [36,39)   |   51 | ■■
 k8s-neonvm  |     14 | [39,42)   |   30 | ■
 k8s-neonvm  |     15 | [42,45)   |   30 | ■
 k8s-neonvm  |     16 | [45,83)   |   79 | ■■■■
 k8s-pod     |      2 | [3,6)     |  120 | ■■■■■
 k8s-pod     |      3 | [6,9)     |  677 | ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
 k8s-pod     |      4 | [9,12)    |  181 | ■■■■■■■■
 k8s-pod     |      5 | [12,15)   |   65 | ■■■
 k8s-pod     |      6 | [15,18)   |   36 | ■■
 k8s-pod     |      7 | [18,21)   |   12 | ■
 k8s-pod     |      8 | [21,24)   |   13 | ■
 k8s-pod     |      9 | [24,27)   |    7 |
 k8s-pod     |     13 | [36,37)   |    1 |
 k8s-pod     |     16 | [46,1110) |    8 |
             |      1 | [2,3)     |   56 | ■■
             |      2 | [3,6)     |  188 | ■■■■■■■■
             |      3 | [6,9)     |   10 |

Query

with histogram as (
   select width_bucket(startup_time, 0, 45, 15) as bucket,
          int4range(min(startup_time), max(startup_time), '[]') as range,
          count(*) as freq,
          provisioner
     from (
        select (extract(epoch from updated_at) - extract(epoch from created_at))::integer as startup_time,
                metadata->'compute_metadata'->>'provisioner' as provisioner
          from operations
         where action = 'start_compute'
           and metadata->'start_compute'->>'from_pool' = 'false'
           and operations.created_at > current_date - interval '24 hour'
      ) as subquery
 group by bucket, provisioner
 order by provisioner, bucket
)
 select provisioner, bucket, range, freq,
        repeat('■',
               (   freq::float
                 / max(freq) over()
                 * 30
               )::int
        ) as bar
   from histogram;

Another one:  https://console.neon.tech/admin/regions/console/operations/e1a18e48-7786-42df-a6b7-477e5df359a2

0:26 - created_at
0:47 - Creating a new Pod
0:53 - Created container compute-init
0:54 - Started container compute-init
1:11 - Created container neonvm-runner
1:12 - Started container neonvm-runner
1:16 - First dmesg entry
1:16 - compute start finished
1:27 - updated_at

Another one: https://console.neon.tech/admin/regions/console/operations/5195384a-c676-4bcb-b52d-d76b50244364
compute_id: compute-super-mountain-87037727

0:40 - created_at
0:52 - Creating a new Pod
1:03 - Started container neonvm-runner
1:11 - First dmesg entry
1:11 - compute start finished
1:10 - updated_at

This looks suspicious, as I'd expect compute start finished would be always before updated_at. Clock skew?

I would split the VM startup time into the following phases:

Scheduler: from created_at to Creating a new Pod
Pod init: from Creating a new Pod to Started container neonvm-runner
Runner: from Started container neonvm-runner to first dmesg entry
Boot: first dmesg entry to compute start finished
Finalize: from compute start finished to updated_at

Hypothesis to verify:

We don't need to optimize phase 5, as the compute is operational at the end of phase 4
Phases 3 and 4 take a relatively small amount of time (TODO: define p99 latency)
The largest offenders are phases 1 and 2

This analysis doesn't explain why neonvm take longer than pods. Doesn't it takes different steps to schedule and init pod?
We should ideally have the observability tooling to perform such analysis real-time.

https://console.neon.tech/admin/regions/console/operations/e7047e4b-c766-47b9-a1d9-379ae1fa0c42
15s:
1+2 -> 10
3+4 -> 1
5 -> 4

status: @Omrigan will try to reproduce, and go from there

https://console.neon.tech/admin/regions/console/operations/4861083c-dd0a-4ece-a58e-1bbd3671f513
Here I compare  log entries between one start_compute and the one I investigated before. prev means how long it took for compute-misty-hall-16416010 between a particular

(Phase 1 starts) created_at -> 0:45
(Phase 2 starts) Creating a new Pod -> 1:00 (took 15s, prev 1s)
Started container init-rootdisk -> 1:06 (took 6s, prev 1s)
Created container sysctl -> 1:22 (took 16s, prev 2s)
Started container compute-init -> 1:24 (took 2s, prev 18s)
(Phase 3 starts)  Started container neonvm-runner -> 1:25 (took 1s, prev 2s)
(Phase 4 starts) first dmesg entry -> 1:27 (took 2s, prev 0s)
(Phase 5 starts) compute start finished -> 1:30 (took 3s, prev 2s)
updated_at -> 1:39

Judging by this, there is no pattern in which a particular container starts slower. Therefore, the possible the delay originates from slowness of k8s's  state update.

Let's take now the k8s-pod compute and see what happens there
https://console.neon.tech/admin/regions/console/operations/dd7df822-94f0-4582-aa5e-36ad6df808ec

(Phase 1 starts) created_at -> 0:03
Not sure where to get k8s-pods logs from k8s internals (like  Neon/Logs/ Autoscaling)
(Phase 4 starts) logging and tracing started` (first message from inside the pod) -> 0:06 2
Then we have 42s between these log lines:

2024-01-15 13:00:07.133 UTC [1] LOG process up: PgBouncer 1.21.0, libevent 2.1.12-stable (epoll), adns: c-ares 1.17.1, tls: OpenSSL 1.1.1w  11 Sep 2023

2024-01-15T13:00:49.658783Z  INFO start_compute:prepare_pgdata: checking pg_hba.conf

(Phase 5 starts) compute start finished -> 0:49
updated_at -> 0:50

So phase 5 is 1s, far less than we had for VMs.
Therefore, two possible explanations of why VM startup time is more, than pods:

k8s-pod provisioner has less containers, therefore k8s has less work to do (less reconciliation loops?)
The way of reporting of start_compute's end time might differ, meaning we actually have a working compute earlier than recorded in operation's updated_at.

I've managed to set up a local compute: compute-polished-poetry-10720110

(P1) 37:09 created_at
(P2) 37:14 Creating a new Pod
37:18 Created container compute-init
(P3) 37:19 (Started container neonvm-runner)
(P4) 37:19 first dmesg entry
(P5) 37:20 compute start finished
37:21 updated_at

The startup time became more noisy.
How it was at issue creation:

How it is now:

Also, these are apparently p50 images. on p99 there is even more noise:

Status: To  try a solution, will merge two init containers into one (#747) - @Omrigan will write a script to create a lot of new VMs to stress-test. Maybe look at cloudbench / start lots of computes at a time on staging via API

A nit on https://github.com/neondatabase/autoscaling/issues/711#issuecomment-1889219969:

Scheduler: from created_at to Creating a new Pod

In this phase actually the scheduler is not involved at all. Once the pod is created after this phase, scheduler receives it in its queue and binds it to a node.
The scheduling actually happens in the second phase:

Pod init: from Creating a new Pod to Started container neonvm-runner

A nit on #711 (comment):

Scheduler: from created_at to Creating a new Pod

In this phase actually the scheduler is not involved at all. Once the pod is created after this phase, scheduler receives it in its queue and binds it to a node.

What happens in this phase then?

The following happens in this phase:

Control plane sends a request to k8s API server to create a VM object.
K8s API server validates the objects itself
K8s API server calls NeonVM controller webhook to do a different sort of validation.
K8s API server stores the object in etcd.
NeonVM controller notices the new VM object via its watch process.
NeonVM controller reconciles the VM object: first changes the status to Pending and then created the runner pod (where you see the Creating a new Pod in the logs).

Maybe you can name it Create or Initial phase.

Future situation best to be tracked with this graph


Black screen with cursor in top left corner

Upon attempting to start sway on my laptop running intel hd graphics (HD 4000, i5-3317U) all I get is a black screen with a non-responsive cursor in the top left corner. Has anybody encountered this issue before?
Here's the output of sway -dV when running from a plain tty: link

Paging @Cloudef

I should probably add that the issue also occurs with the default config included with sway.

When this happen, is X running at the same time or not ?

@Reventl0v This happens in several cases, straight from a tty, X as main wm, and as a window inside a X wm.

Upon reinstalling my laptop I can no longer reproduce this issue. Must've been a quirk with my setup.