poisson_log_glm sampling statement fails to fit

Summary:
Fitting data with
y ~ poisson_log_glm(X, alpha, beta)
fails with almost 100% treedepth warnings but using
target += poisson_log_glm_lpmf(y | X, alpha, beta)
works well
Description:
Poisson regression on a toy data set using
y ~ poisson_log_glm(X, alpha, beta)
in the model section returns a warning for 3997 (out of 4000) transitions exceeding the maximum treedepth, Rhat is > 100 and neff = 2. Using either
y ~ poisson_log(alpha + X * beta)
or
target += poisson_log_glm_lpmf(y | X, alpha, beta)
results in a high quality fit: Rhat ~ 1.000, neff in the range 2000 -4000. The rstan code for generating the toy data and fitting with the three versions is below.
Reproducible Steps:
set.seed(1)
DF2 <- data.frame(X1 = rnorm(200, 0, 1), X2 = runif(200, -2, 2), X3 = rnorm(200, 1, 2))
DF2 <- DF2 %>% mutate(Y = 0.3 * X1 - 0.15 * X2 + 0.2 * X3 + 1.4)
DF2 <- DF2 %>% mutate(Cnt = rpois(200, lambda =  exp(Y)))

dataList2 <- list(N = nrow(DF2), K = 3, y = DF2$Cnt, X = as.matrix(DF2[, 1:3]))
PoissonMod <- "
data {
    int N; //number obs
    int K; //number of columns in the model matrix
    int y[N]; //response
    matrix[N,K] X;
  }
parameters {
  real alpha;
  vector[K] beta; //the regression parameters
}
model {
  alpha ~ normal(0,1.5);

  beta ~ normal(0,1);

  //y ~ poisson_log(alpha + X * beta); //version 1, works well
  //y ~ poisson_log_glm(X, alpha, beta); //version 2, treedepth failures near 100%
  target += poisson_log_glm_lpmf(y | X, alpha, beta); //version 3, works well
}
"
PoisOut <- stan(model_code = PoissonMod, data = dataList2, verbose = FALSE, control = list(max_treedepth = 10))

RStan Version:
rstan_2.19.2         StanHeaders_2.19.0
R Version:
R version 3.5.2 (2018-12-20)
Operating System:
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 18.3

I think the error is here: https://github.com/stan-dev/math/blob/develop/stan/math/prim/mat/prob/poisson_log_glm_lpmf.hpp#L109
include_summand<propto, T_partials_return>::value should be include_summand<propto, T_x, T_alpha, T_beta>::value
The two different specifications compile to cpp differently. The target+ syntax calls:
lp_accum__.add(poisson_log_glm_lpmf<propto__>(y, X, alpha, beta));
and the ~ syntax calls
lp_accum__.add(poisson_log_glm_lpmf<false>(y, X, alpha, beta));
I don't know if the difference in the calls is a bug or not. Someone else can take this. Just this is what I found looking earlier.

Pinging @t4c1 and @rok-cesnovar who have recently been working on compound functions

For now, I can confirm that I can reproduce this on the current develop of cmdstan which means that its still an issue in the latest Stan Math. I went to double-check because there was a lot of churn on GLMs since Stan Math 2.19 which you are using via Stan Headers 2.19.
I will explore this further but the code to reproduce with cmdstanr is this (put the stan code in a separate file:
library(dplyr)
library(cmdstanr)

setwd("~/Desktop/Stan/tests/")
set_cmdstan_path("/home/rok/Desktop/Stan/cmdstan/")
file <- file.path("poisson_glm_bug.stan")
mod <- cmdstan_model(file)

set.seed(1)
DF2 <- data.frame(X1 = rnorm(200, 0, 1), X2 = runif(200, -2, 2), X3 = rnorm(200, 1, 2))
DF2 <- DF2 %>% mutate(Y = 0.3 * X1 - 0.15 * X2 + 0.2 * X3 + 1.4)
DF2 <- DF2 %>% mutate(Cnt = rpois(200, lambda =  exp(Y)))
dataList2 <- list(N = nrow(DF2), K = 3, y = DF2$Cnt, X = as.matrix(DF2[, 1:3]))

fit <- mod$sample(
   data = dataList2,
   num_chains = 4,
   num_cores = 1,
   max_depth = 10
)

Wit the ~ version I get the same issue:
Total execution time: 15.5 seconds.
3997 of 1000 (400%) transitions hit the maximum treedepth limit of 10 or 2^10-1 leapfrog steps.
Trajectories that are prematurely terminated due to this limit will result in slow exploration.
Increasing the max_depth limit can avoid this at the expense of more computation.
If increasing max_depth does not remove warnings, try to reparameterize the model.

include_summand<propto, T_partials_return>::value should be include_summand<propto, T_x, T_alpha, T_beta>::value

Can confirm that this fixes it.

Actually I think that include_summand doesn't need to be there at all cause this is already before it: https://github.com/stan-dev/math/blob/develop/stan/math/prim/mat/prob/poisson_log_glm_lpmf.hpp#L76
I think the propto thing is another bug as well.
Like, the _lpmf should have 'propto = false' cause in those cases we want the full lpmf, not just something up to a proportionality constant suitable for MCMC (which is what propto = true means). This is important for like the loo calculations and such in generated quantities.
For the beta parameter (which I assume is correct) I get:
lp_accum__.add(normal_lpdf<false>(beta, 0, 1));

for
target += normal_lpdf(beta | 0, 1);

and
lp_accum__.add(normal_log<propto__>(beta, 0, 1));

for
beta ~ normal(0, 1);

I have the fix up on a branch together with the same fix for the GPU implementation, just need to make a write a test that would fail now and pass with the fix. Should be ready tomorrow.

Like, the _lpmf should have 'propto = false' cause in those cases we want the full lpmf, not just something up to a proportionality constant suitable for MCMC (which is what propto = true means).

That's right---using foo_lpfm(...) in Stan should have propto=false generated for the call.

@bbbales2 With cmdstan+stanc3 I get the following:
target += poisson_log_glm_lpmf(y | X, alpha, beta);
produces
lp_accum__.add(poisson_log_glm_lpmf<false>(y, X, alpha, beta));
and
y ~ poisson_log_glm(X, alpha, beta);
produces
lp_accum__.add(poisson_log_glm_lpmf<propto__>(y, X, alpha, beta));
On cmdstan 2.21 (with the c++ stan compiler) I get the following:
target += poisson_log_glm_lpmf(y | X, alpha, beta);
produces
lp_accum__.add(poisson_log_glm_log(y, X, alpha, beta));
and
y ~ poisson_log_glm(X, alpha, beta); produces
lp_accum__.add(poisson_log_glm_lpmf<propto__>(y, X, alpha, beta));
So I think this is all fine? But do please confirm. I also opened the PR with the fix for the Math code.


Support for Vite 4.x

Reporting a bug?
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/vite
npm ERR!   dev<EMAIL_ADDRESS>from the root project
npm ERR!   peer vite@">=2" from<EMAIL_ADDRESS>npm ERR!   node_modules/vite-plugin-eslint
npm ERR!     dev<EMAIL_ADDRESS>from the root project
npm ERR!   3 more (vite-plugin-full-reload, @vitejs/plugin-basic-ssl, @vitejs/plugin-vue)
Expected behavior
Works with Vite 4.x
Reproduction
vite 4.0.0 in package.json
Issue Package
vite-plugin-vue-i18n
System Info
...

Screenshot
No response
Additional context
No response
Validations

[X] Read the Contributing Guidelines.
[X] Read the README
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion.

Thank you for your reporting!
As say docs, vite-plugin-vue-i18n isn't support no longer for Vite 4.
https://github.com/intlify/bundle-tools/tree/main/packages/vite-plugin-vue-i18n#intlifyvite-plugin-vue-i18n
You need to use unplugin-vue-i18n
https://github.com/intlify/bundle-tools/tree/main/packages/unplugin-vue-i18n
Thanks!


Credits to Greg

Issue 15000 credits to @GregCTWang.

 Follow up
On Tuesday, 05-Mar-24 09:10:32 UTC by Gov4Git dev
Issued 15000 credits to member @gregctwang.


fixes https://github.com/PlaytikaOSS/feign-reactive/issues/651

Fixed similarly to https://github.com/spring-cloud/spring-cloud-openfeign/blob/main/spring-cloud-openfeign-core/src/main/java/org/springframework/cloud/openfeign/FeignClientsRegistrar.java#L252

I'm waiting for this PR to be merged and released soon to use Spring Boot 3.2!

I'm waiting for this PR to be merged and released soon to use Spring Boot 3.2!

Same here!

I'm waiting for this PR to be merged and released soon to use Spring Boot 3.2!

Same here!

I'm waiting too !


exampledata folder seems to not be cloned

I installed qtt as a site-package part of another package.
In my setup.py I have:
qtt = 'qtt<EMAIL_ADDRESS>...
setup(
    ...
    install_requires = [ ..., qtt, ...]
    ...
    )

However, it seems that after the installation completes successfully (after a pip install .)the qtt/exampledata/ folder is not part of my qtt site-packages.

Can someone please have a look at this?
Thank you!

The exampledata should be accesible with
import qtt.data
dataset=qtt.data.load_example_dataset('time_rabi_scan.json')

The load_example_dataset is used in several example notebooks, e.g. https://qtt.readthedocs.io/en/latest/notebooks/analysis/example_fermi_fitting.html
@sldesnoo-Delft @QFer

When installing qtt via 'pip install .' only the qtt 'functionality' is installed, not the docs, ipython notebooks etc. So in my opinion exampledata is also one of the components that should not be installed. When you want to run examples or unit tests you have to install in editable mode.
Or am I missing something?

@QFer You've got a good point. I tend to agree and we should remove the docs and examples from the install.
@peendebak Do you agree?

See #721

Closed because this issue is solved in https://github.com/QuTech-Delft/qtt/pull/750


Add article "Building a Scalable Design System for The Dev Protocol Ecosystem"

welp since it's approved for the most part, i think we can merge this now?


set custom cpu model - to enable kvm live migration

System Information
Linux distribution
Ubuntu 20.04.6 LTS
Terraform version
   Terraform v1.4.6
Provider and libvirt versions
dmacvicar/libvirt v0.7.1

Checklist

[ x] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?

[ ] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
[ ] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
[ ] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

Description of Issue/Question
Setup
(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive information)
terraform {
  required_version = ">= 1.0.1"
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
      version = "0.7.1"
    }
  }
}

provider "libvirt" {
  uri =<EMAIL_ADDRESS>}

resource "libvirt_domain" "ubuntu-vm" {
  name = "${var.vm_id}__${var.hostname}"
  memory = var.ram
  vcpu   = var.cpu
  autostart = var.autostart

  cpu {
    mode = "host-passthrough"
  }

  disk {
    volume_id = libvirt_volume.ubuntu-vol.id
  }

  boot_device {
    dev = [ "hd", "network"]
  }

  dynamic "network_interface" {
    for_each = var.network
    content {
      bridge = network_interface.value
    }
  }

  cloudinit = libvirt_cloudinit_disk.cloud-init.id

  console {
    target_type = "serial"
    type        = "pty"
    target_port = "0"
  }
  console {
    target_type = "virtio"
    type        = "pty"
    target_port = "1"
  }
}

resource "libvirt_volume" "ubuntu-base" {
  name = "ubuntu-${var.dist}-${var.hostname}-base.qcow2"
  pool = "images"
  source = "https://ftp.domain.tld/src/${var.dist}-server-cloudimg-amd64.img"
  format = "qcow2"
}

resource "libvirt_volume" "ubuntu-vol" {
  name = "${var.hostname}.qcow2"
  pool = "images"
  base_volume_id = libvirt_volume.ubuntu-base.id
  size = var.vol_size
}

data "template_file" "user_data" {
  template = file("${path.module}/../user-data.yml")
  vars = {
    hostname = var.hostname
    fqdn = "${var.hostname}.${var.domain}"
    chroot = var.vm_id
    swap = var.swap_size
  }
}

data "template_file" "network_config" {
  template = file("${path.module}/../network_config.yml")
  vars = {
    ip4 = var.ip4
    gw4 = var.gw4
  }
}

resource "libvirt_cloudinit_disk" "cloud-init" {
  name = "cloud-init-${var.hostname}.iso"
  pool = "cloud-init"
  user_data = data.template_file.user_data.rendered
  network_config = data.template_file.network_config.rendered
}

Additional information:
For enable libvirt live migration I need to set a specific CPU model in XML definition file.
Live migration dont work with "Host passthrough", you need to set a Named model. In my case it looks like:
  <cpu mode='custom' match='exact' check='partial'>
    <model fallback='allow'>Haswell-noTSX-IBRS</model>
  </cpu>

Also a specific os machine type is needed for live migration:
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
       :
  </os>

I could not find any documentation for setting a CPU model and nothing for set an OS machine type. Is there any way to do this?
Documentation: https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html

more docs about this:
https://libvirt.org/formatdomain.html#cpu-model-and-topology
https://libvirt.org/formatdomain.html#bios-bootloader

without setting the <cpu> feature in main.tf, the deployed VM looks like:
  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>qemu64</model>
  </cpu>

But with this CPU setting a live migration is not possible:
# virsh migrate --live u22-migtest qemu+ssh://dest-system/system
error: operation failed: guest CPU doesn't match specification: extra features: vmx

Hi,
I think it is just not implemented and it's a possible feature.


Tests pass using swift package manager on Linux and Mac, using Swift 4.0.2

This is great!
Thank you @danramteke.

Great! Glad you like it!
Perhaps at the top of Data+Gzip.swift the imports should be
#if os(Linux)
  import zlibLinux
#else
  import zlib
#endif

instead of
#if os(OSX)
import zlib
#elseif os(Linux)
import zlibLinux
#endif

in order to support os's that are not OSX or Linux.


Rename AvroReadGroupGenomicRDD to AvroRecordGroupGenomicRDD.

+1! Thanks Michael!

You're too quick!  I pushed another commit to this branch with WIP towards addressing the second review comment ... IDEA is broken on the new autogenerate stuff so I still need time to complete it.  Let's see, do I need to do another pull request?

I think I may have given you push permissions to my fork when we were working on the VariantContextConverter rewrite... No sweat though; just be mindful of such in the future ;)


Use it with an existing axios instance

Hello,
I try to use this tool in an existing app. It already uses an axios instance in JWTService Class  to handle auth.
Is there a way to use the existing axios instance in the openApiClient instance rather than create a new one.
Thx

That's a cool idea. PRs accepted :)

I check this idea and what can I say:

AxiosInstance does not have  .create method for creating sub-instances
https://github.com/axios/axios/issues/1170
https://github.com/axios/axios/pull/2040

We can't use provided AxiosInstance without sub-instancing or cloning, it looks like antipattern with side effects to unexpected modifying provided object.

Seems like defaultAxiosConfigs will not usable together with instance and constructor type will be like

 { ... } & ({
    axiosConfigDefaults?: AxiosRequestConfig;
    baseAxiosInstance?: never;
  } | {
    baseAxiosInstance?: AxiosInstance;
    axiosConfigDefaults?: never;
  })

@anttiviljami, @gagagast
Maybe you can help me to found a nice solution for 1 and 2 list items.


Unit Tests for: repo_test.go, task_test.go

@bradrydzewski here it is the new PR you requested.
;
doliveira@doliveira-opensuse-dev10:~/go/src/github.com/drone/drone> clear; date; make test
Mon Apr 20 15:19:14 MDT 2015
go vet ./...
go test -cover -short ./...
?       github.com/drone/drone  [no test files]
?       github.com/drone/drone/builder  [no test files]
?       github.com/drone/drone/builder/docker   [no test files]
?       github.com/drone/drone/builder/pool     [no test files]
?       github.com/drone/drone/common   [no test files]
?       github.com/drone/drone/common/ccmenu    [no test files]
?       github.com/drone/drone/common/httputil  [no test files]
?       github.com/drone/drone/common/oauth2    [no test files]
?       github.com/drone/drone/common/sshutil   [no test files]
?       github.com/drone/drone/datastore        [no test files]
ok      github.com/drone/drone/datastore/bolt   10.966s coverage: 72.6% of statements
?       github.com/drone/drone/datastore/rpc    [no test files]
?       github.com/drone/drone/eventbus [no test files]
ok      github.com/drone/drone/parser   0.002s  coverage: 67.5% of statements
ok      github.com/drone/drone/parser/inject    0.002s  coverage: 100.0% of statements
ok      github.com/drone/drone/parser/matrix    0.002s  coverage: 68.6% of statements
?       github.com/drone/drone/remote   [no test files]
?       github.com/drone/drone/remote/client    [no test files]
?       github.com/drone/drone/remote/github    [no test files]
?       github.com/drone/drone/server   [no test files]
?       github.com/drone/drone/server/session   [no test files]
?       github.com/drone/drone/settings [no test files]

great job. just some minor cleanup (like removing some of the commented out code) and we'll get it merged!

@bradrydzewski , on the (ErrConflict), when I tried to use it, I got an error saying it wasn't defined and that's why I commented it out.

maybe try ErrKeyExists?

Yep, ErrKeyExists was the correct one, once we are going thru the insert(). Thanks.

great work!  thanks for all the effort ... I know you are new to Go and I really appreciate you sticking with this PR until ready

Thank you for your helping and coaching!


Update IIS hosting instructions

Thanks for helping fix this!
On Fri, 9 Nov 2018 at 11:30, jflheureux<EMAIL_ADDRESS>wrote:

@jflheureux commented on this pull request.
In README.md
https://github.com/plumber-sc/plumber-sc/pull/21#discussion_r232207517:

+1. Copy the plumber-sc-[version] folder from the release ZIP file into C:\inetpub\wwwroot.
Now I understand!
In the Readme file of https://github.com/plumber-sc/plumber-sc, the
"Releases" link navigates to your private fork of the project
https://github.com/ewerkman/plumber-sc/releases and there are no releases
in your fork.
By manually going in the releases tab of the plumber-sc repo, I can see
the releases ZIP files.
You can discard my pull request and fix the link to navigate to the right
release page.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/plumber-sc/plumber-sc/pull/21#discussion_r232207517,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMVXrVGd7FK1Fa-mP7z0wWQdxfZ45XDks5utVlWgaJpZM4YL27R
.


Added amm4108.yaml

first flight assignment

:one: :airplane:
Be sure to include a key for your firstflight blogpost :)


Webpack serve rendering directory list instead of app

I have a react-native application which i am trying to run in web with react-native-web. I asked this question to react-native-web issue tracker and they told me to ask webpack questions to the webpack community so here i am.

// This is needed for webpack to compile JavaScript.
// Many OSS React Native packages are not compiled to ES5 before being
// published. If you depend on uncompiled packages they may cause webpack build
// errors. To fix this webpack can be configured to compile to the necessary
// `node_module`.
const babelLoaderConfiguration = {
  test: /\.js$/,
  // Add every directory that needs to be compiled by Babel during the build.
  include: [
    path.resolve(appDirectory, 'src/index.js'),
    path.resolve(appDirectory, 'src'),
    path.resolve(appDirectory, 'node_modules/react-native'),
    path.resolve(appDirectory, 'node_modules/react-native-uncompiled'),
    path.resolve(appDirectory, 'node_modules/react-native-gesture-handler'),
    path.resolve(appDirectory, 'node_modules/react-native-dynamic-bundle'),
    path.resolve(appDirectory, 'node_modules/react-native-event-listener'),
    path.resolve(appDirectory, 'node_modules/react-native-fs'),
    path.resolve(appDirectory, 'node_modules/react-native-image-preview'),
    path.resolve(appDirectory, 'node_modules/react-native-settings-list'),
    path.resolve(appDirectory, 'node_modules/react-native-swiper'),
    path.resolve(appDirectory, 'node_modules/react-native-vector-icons'),
    path.resolve(appDirectory, 'node_modules/react-native-image-progress'),
    path.resolve(appDirectory, 'node_modules/react-native-progress'),
    path.resolve(appDirectory, 'node_modules/rn-fetch-blob')
  ],
  use: {
    loader: 'babel-loader',
    options: {
      cacheDirectory: true,
      // The 'metro-react-native-babel-preset' preset is recommended to match React Native's packager
      presets: ['module:metro-react-native-babel-preset'],
      // Re-write paths to import only the modules needed by the app
      plugins: ['react-native-web']
    }
  }
};

// This is needed for webpack to import static images in JavaScript files.
const imageLoaderConfiguration = {
  test: /\.(gif|jpe?g|png|svg)$/,
  use: {
    loader: 'url-loader',
    options: {
      name: '[name].[ext]'
    }
  }
};

module.exports = {
  entry: [
    // load any web API polyfills
    // path.resolve(appDirectory, 'polyfills-web.js'),
    // your web-specific entry file
    path.resolve(appDirectory, 'src/index.js')
  ],

  // configures where the build ends up
  output: {
    filename: 'bundle.web.js',
    path: path.resolve(appDirectory, '/dist/')
  },

  devServer: {
    publicPath: '/dist'
  },

  // ...the rest of your config

  module: {
    rules: [
      babelLoaderConfiguration,
      imageLoaderConfiguration,
      {
        test: /\.m?js$/,
        resolve: {
          fullySpecified: false
        }
      }
    ]
  },

  resolve: {
    // This will only alias the exact import "react-native"
    alias: {
      'react-native$': 'react-native-web'
    },
    // If you're working on a multi-platform React Native app, web-specific
    // module implementations should be written in files using the extension
    // `.web.js`.
    extensions: [ '.web.js', '.js' ]
  }
}

Thank you for creating this issue. However, issues need to follow one of our templates so that we can clearly understand your particular circumstances.
Please help us help you by recreating the issue using one of our templates.

Look at devServer.contentBase


Multi-Instance seems to trigger Idle State once

When you have multiple gallerias in a page the Idle State only gets triggered for the last instance.
For example, a page with 2 instances, only enters idle state for the last galleria that is init on load.  Another example is if you have 2 gallerias next to each other on a page, moving the mouse over both instances so that you exit idle state before either instance enters the idle state, only the instance with the last mouse out enters idle state.
At glance, the code seems to only maintain a single timer, not one per instance...if I read it right.

Thanks, you might be right we’ll have a look.


<IP_ADDRESS> - Add configutation for RQ Job timeout.

What's been done:

Fix bug on #88 by adding new config to set RQ job timeout
Add one more job status: Die in Queue

To do (not in this PR):

Complete way to raise and trace back exception in DB connection class
Add one more job status: Die on working
Test cases for JobWorker

Coverage decreased (-0.2%) to 41.452% when pulling 5c125ff1bfb48f507076f2b6eb84e4f0f07dd377 on <IP_ADDRESS>.FixBug.RQJobTimeout into 56cce38c6b4fa5757654be4f86bef10bad044283 on dev.

Reviewed


changes for json and live options

Thanks for the pull request! But there are so many changes to the core part. Could you please explain what you want? What exactly do you mean by live options?

Hi @ceeaspb
Thanks a lot for your contributions. I merged your changes. After merging I did some improvements to the application to use different commands to produce different outputs.
You can take a look at the commit: https://github.com/chrishantha/jfr-flame-graph/commit/3b91d921edd2016d4291a29cfc9e90264ae59110

Hi @chrishantha thanks and apologies I missed your original review comments

That's okay! Thanks again!


Issue with gevent

Describe the bug
Locust had been running fine a few months ago. Went to start it the other day and started getting errors in gevent/_tblib.py, line 202 - TypeError: an integer is required (got type bytes). Then the process exits.
I don't know what could have changed in the meantime. I had been using 0.13.02 tried 0.14.4 and got the same error.  I'm on Mac OS X 10.15.3 with libev: stable 4.31.
Expected behavior
It should have run as normal
Actual behavior
Throws exception and quits.
Steps to reproduce
Not sure. Don't think it is a code error.
Environment

OS: Mac OS X 10.15.3
Python version: 3.7 & 3.8
Locust version:  0.13.02 and 0.14.4
Locust command line that you ran: locust
Locust file contents (anonymized if necessary):

Hi! Could this be related? (maybe you are running against a dns name that doesnt exist?)
https://github.com/gevent/gevent/issues/1491

Although the claim that is fixed in 1.5a2 though...

Maybe it would be worth to check with 1.5a3...

Maybe it would be worth to check with 1.5a3...

Yes! That is in locust master branch but not yet a release.

I thought it was always the load balancer? :) At least it was at a company I worked at a long long time ago.
Actually, this was a big hint. It wasn't DNS in the sense of trying to find the server under test. It is/was an issue trying to bind to the web host and port for the UI.  If I specify them specifically it works ie.
locust --web-host <IP_ADDRESS> --port 8889
I don't know if something has gone wacky with my machine or its networking or ... ???
Thanks for the help.

Ran into the same issue, the workaround worked:
locust --web-host <IP_ADDRESS> --port 8009

I think it is still an issue, and shouldn't have to explicitly specify localhost.
Here's more info about my system:
Darwin bhutwala-mac 18.7.0 Darwin Kernel Version 18.7.0: Thu Jan 23 06:52:12 PST 2020; root:xnu-4903.278.25~1/RELEASE_X86_64 x86_64
I ran into this problem with both in a docker container and on my host macOS machine as well.  Both with locust==0.8.1 and locust==0.13.0 with libev 4.33

Interesting! Do you have some firewall software installed? Some special networking setup?
It works fine on my Macbook Pro with you exact version.

@JulioBarros @yashbhutwala can you show me your firewall settings? (System settings / Security and Integrity / Firewall / Firewall options)
Here are mine, but in Swedish :) If I try to run Locust with the default settings I often/always get a firewall opening request popup, but not the issues you're describing.

@cyberw
I don't see any issues with the firewall settings

I have attached the logs here, if they help 😃
[2020-03-23 08:01:31,100] bhutwala-mac/INFO/locust.main: Starting web monitor at *:8089
[2020-03-23 08:01:31,101] bhutwala-mac/INFO/locust.main: Starting Locust 0.13.0
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: Traceback (most recent call last):
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/locust/web.py", line 201, in start
    pywsgi.WSGIServer((options.web_host, options.port),
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/baseserver.py", line 367, in serve_forever
    self.start()
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/baseserver.py", line 305, in start
    self.init_socket()
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,179] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/pywsgi.py", line 1491, in init_socket
    self.update_environ()
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/pywsgi.py", line 1503, in update_environ
    name = socket.getfqdn(address[0])
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_socketcommon.py", line 269, in getfqdn
    hostname, aliases, _ = gethostbyaddr(name)
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_socketcommon.py", line 241, in gethostbyaddr
    return get_hub().resolver.gethostbyaddr(ip_address)
[2020-03-23 08:01:31,180] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/resolver/thread.py", line 68, in gethostbyaddr
    return self.pool.apply(_socket.gethostbyaddr, args, kwargs)
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/pool.py", line 159, in apply
    return self.spawn(func, *args, **kwds).get()
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr: File "src/gevent/event.py", line 268, in gevent._event.AsyncResult.get
[2020-03-23 08:01:31,184] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,185] bhutwala-mac/ERROR/stderr: File "src/gevent/event.py", line 296, in gevent._event.AsyncResult.get
[2020-03-23 08:01:31,185] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,185] bhutwala-mac/ERROR/stderr: File "src/gevent/event.py", line 286, in gevent._event.AsyncResult.get
[2020-03-23 08:01:31,185] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,185] bhutwala-mac/ERROR/stderr: File "src/gevent/event.py", line 266, in gevent._event.AsyncResult._raise_exception
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr: File "src/gevent/event.py", line 211, in gevent._event.AsyncResult.exc_info.__get__
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_tblib.py", line 371, in g
    return f(a)
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,186] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_tblib.py", line 432, in load_traceback
    return loads(s)
[2020-03-23 08:01:31,189] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,189] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_tblib.py", line 337, in unpickle_traceback
    return ret.as_traceback()
[2020-03-23 08:01:31,189] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,189] bhutwala-mac/ERROR/stderr: File "/Users/bhutwala/.local/share/virtualenvs/loadgenerator-V00fQKam/lib/python3.8/site-packages/gevent/_tblib.py", line 202, in as_traceback
    code = CodeType(
[2020-03-23 08:01:31,189] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,190] bhutwala-mac/ERROR/stderr: TypeError: an integer is required (got type bytes)
[2020-03-23 08:01:31,190] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,190] bhutwala-mac/ERROR/stderr: 2020-03-23T12:01:31Z
[2020-03-23 08:01:31,190] bhutwala-mac/ERROR/stderr:
[2020-03-23 08:01:31,190] bhutwala-mac/ERROR/stderr: <Greenlet at 0x107fe5040: start([<class 'locustfile.WebsiteUser'>], Namespace(csvfilebase=None, exit_code_on_error=1, )> failed with TypeError
[2020-03-23 08:01:31,190] bhutwala-mac/INFO/locust.main: Shutting down (exit code 0), bye.
[2020-03-23 08:01:31,190] bhutwala-mac/INFO/locust.main: Cleaning up runner...
[2020-03-23 08:01:31,190] bhutwala-mac/INFO/locust.main: Running teardowns...
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)                                       0.00    0.00

Percentage of the requests completed within given times
 Name                                                           # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------

Hmmm... I am on python 3.7.7. That is the only difference I see. I do wonder why your firewall options are greyed out though? Are you 110% sure there is no other firewall or maybe corporate policy in place?

Hmmm... I am on python 3.7.7. That is the only difference I see. I do wonder why your firewall options are greyed out though? Are you 110% sure there is no other firewall or maybe corporate policy in place?

Hmm... not a 110% sure... perhaps there is some corporate policy I'm unaware of, I cannot fully discount it.

Other people have reported similar issues with Python 3.8, so it may have something to do with that...
https://github.com/pyinstaller/pyinstaller/issues/4265

Can you check which version of gevent that you have installed?

using gevent==1.4.0
certifi==2018.11.29       # via requests
chardet==3.0.4            # via requests
click==7.0                # via flask
flask==1.0.2              # via locustio
gevent==1.4.0             # via locustio
greenlet==0.4.15          # via gevent
idna==2.8                 # via requests
itsdangerous==1.1.0       # via flask
jinja2==2.10              # via flask
locustio==0.13.0
markupsafe==1.1.0         # via jinja2
msgpack-python==0.5.6     # via locustio
pyzmq==17.0.0             # via locustio
requests==2.21.0          # via locustio
six==1.12.0               # via locustio
urllib3==1.24.1           # via requests
werkzeug==0.14.1          # via flask

Oh. That is very old. Latest locust release depends on 1.5a3.


Wrong behavior of random command when passing wrong arguments

Issue has been fixed in https://github.com/SecondThundeR/secondthunder-js-bot/commit/a56647a4226ae1a930f8639405c1fdaad58b3531


Windsor - still supported?

I'm looking for a replacement IoC/DI library, following the demise of both spring.net and unity. The last release of Windsor was over two years ago, and although I can see commits in Github over this time, there haven't been any further releases. Is there a plan for a 3.3.1 or 3.4 roll-up? Are any of the commits critical?

@rangercej I've actually got a 3.3.1/3.4 rollup release on my TODO list when I push out Castle Core 4.0, however there hasn't been many (if any) defect fixes mostly just documentation fixes.
Windsor is still used by many and is very mature with over a decade of use, however there isn't an active team anymore (most of us either don't use a container anymore or have moved to another platform) as you'll see with the piling up issues (many of which are support questions). Windsor isn't abandoned, those that have given so much time in the past just don't have the time at the moment, however we welcome people to get involved which is a great way to learn the library.

I should also respond to your question "still supported?", what does that really mean, I am not aware of any .NET container with commercial support other than the Microsoft one which as you'd mentioned is now dead. Stackoverflow is really "support" for most open source, and there are many users contributing there: http://stackoverflow.com/questions/tagged/castle-windsor
Windsor is and always will be open source, so always within your control.

Bad terminology on my part - maintained would have been a better one to use. Ta for the info :-)


Feature: Adding Sync bound on Future

I need a trait method to return a Future of this kind:
Pin<Box<dyn Future<Output = ...> + Send + Sync + 'static>>

but all I'm able to obtain with this library is
Pin<Box<dyn Future<Output = ...> + Send + 'static>>

Would it be possible to make the library able to add this Sync bound optionally?
We could use this syntax:
#[async_trait(Sync)]

If you think this would be relevant, I'm willing to work on implementing this

Related to #142

This is a duplicate of #77. It's not supported by this crate.


Update Client Messages, Part 1.

This PR updates the client messages by:

Replacing Leaf with a nested Entry
Renaming messages to have consistent Client- prefixes
Removes wait_for_commit from batch submit, and renames it in Get Batch Status
Changes type of timeout.

Added a commit to delete a TnT integration test

There are still a few T&T lines in run_lint, if you'd care to remove them.


Falco Graduation Path: Progress Tracker

In alignment with the suggestions from our TOC sponsors, we're actively focusing on specific areas to improve The Falco Project, aiming to be fully prepared for the forthcoming Graduation Public Comment phase.
Comprehensive details of the proposed path are outlined in the following document :point_down:
https://docs.google.com/document/d/12l65c6qC91akgFjzw7BM3KdNJAa5LyDG17sdpdfwf3g/edit?usp=sharing
This issue serves as a tracker for all related activities (see the task lists below).
Get Feedback
- [x] Reach out to Taylor Dolezal
- [x] Understand how adopters use Falco
- [ ] https://github.com/falcosecurity/falco-website/issues/1036
- [ ] https://github.com/falcosecurity/falco-website/pull/1089
- [ ] https://github.com/falcosecurity/falco-website/pull/1070

Tech Writing / Documentation
We will use #falco-docs channel on the CNCF Slack to discuss those efforts.
- [x] File a service desk ticket with CNCF requesting a technical writer
- [ ] https://github.com/falcosecurity/falco-website/issues/1003
- [ ] https://github.com/falcosecurity/evolution/issues/271
- [ ] https://github.com/falcosecurity/falco-website/issues/1004
- [ ] https://github.com/falcosecurity/falco-website/issues/1006
- [ ] https://github.com/falcosecurity/falco-website/issues/1009
- [ ] https://github.com/falcosecurity/falco-website/issues/985
- [ ] https://github.com/falcosecurity/falco-website/issues/1031
- [ ] https://github.com/falcosecurity/falco-website/issues/1063
- [ ] https://github.com/falcosecurity/falco-website/issues/1007
- [ ] https://github.com/falcosecurity/falco-website/issues/1008
- [ ] https://github.com/falcosecurity/falco-website/issues/1005
- [ ] https://github.com/falcosecurity/falco/pull/2594
- [ ] https://github.com/falcosecurity/falco-website/pull/1069
- [ ] https://github.com/falcosecurity/falco-website/pull/1097
- [ ] https://github.com/falcosecurity/falco-website/pull/1085
- [ ] https://github.com/falcosecurity/falco-website/pull/1078
- [ ] https://github.com/falcosecurity/falco-website/pull/1077
- [ ] https://github.com/falcosecurity/falco-website/pull/1072
- [ ] https://github.com/falcosecurity/falco-website/pull/1090
- [ ] https://github.com/falcosecurity/falco-website/pull/1096
- [ ] https://github.com/falcosecurity/falco-website/pull/1099
- [ ] https://github.com/falcosecurity/falco-website/pull/1094
- [ ] https://github.com/falcosecurity/falco-website/pull/1106
- [ ] https://github.com/falcosecurity/falco-website/pull/1108

Use Case Development
Develop two primary use cases
- [x] Detection use case published
- [x] Compliance use case published
- [ ] https://github.com/falcosecurity/falco-website/issues/1035
- [ ] https://github.com/falcosecurity/falco-website/pull/1027
- [ ] https://github.com/falcosecurity/falco-website/pull/1049
- [ ] https://github.com/falcosecurity/falco-website/pull/1054

Kernel Version Testing
- [x] File a service desk ticket with CNCF requesting resources to perform kernel version testing
- [x] CNCF has provided resources to run tests in our CI
- [ ] https://github.com/cncf/cluster/issues/240
- [ ] https://github.com/falcosecurity/libs/issues/1191
- [ ] https://github.com/falcosecurity/libs/pull/1090
- [ ] https://github.com/falcosecurity/libs/pull/524
- [ ] https://github.com/falcosecurity/libs/pull/1131

Rule Modularity/Assumptions
- [ ] https://github.com/falcosecurity/rules/issues/101
- [ ] https://github.com/falcosecurity/libs/pull/1095
- [ ] https://github.com/falcosecurity/rules/pull/76
- [ ] https://github.com/falcosecurity/rules/pull/115
- [ ] https://github.com/falcosecurity/rules/pull/102
- [ ] https://github.com/falcosecurity/rules/pull/103
- [ ] https://github.com/falcosecurity/rules/pull/106
- [ ] https://github.com/falcosecurity/rules/pull/108
- [ ] https://github.com/falcosecurity/rules/pull/109
- [ ] https://github.com/falcosecurity/rules/pull/110
- [ ] https://github.com/falcosecurity/rules/pull/112
- [ ] https://github.com/falcosecurity/rules/pull/113
- [ ] https://github.com/falcosecurity/rules/pull/114
- [ ] https://github.com/falcosecurity/rules/pull/122
- [ ] https://github.com/falcosecurity/rules/pull/125

Status Update
I'm delighted to announce that we've completed all the action items recommended by our TOC sponsors (@TheFoxAtWork and @justincormack) and are on track with the proposed July and August timeline! Of course, this early completion doesn't imply a slowdown in our commitment to improving the project. Instead, it's renovated with the invaluable knowledge we've acquired from our experiences over the past few months.
As we reach this significant milestone, I've detailed a comprehensive, step-by-step status update below. I hope this information proves useful to you.
I've got to give a big shout-out to every person in the Falco community and the extended CNCF family who has been involved in this effort. Your contributions and support have made these initiatives fly. A special thank you goes to @nate-double-u for the help he's provided us.
cc @falcosecurity/core-maintainers
Get Feedback
Under the guidance of Taylor Dolezal, we've engaged multiple end-users to gather their feedback on Falco usage. We've established a robust process for receiving end-user feedback, including interviews with specific sets of questions aimed at understanding how Falco is used within their respective architectures. These interviews are converted into end-user case studies which we will continue to publish on https://falco.org/about/ecosystem/ as well as linking them from the falco.org home page.
Activities completed include:

Trendyol case study: Highlighting how Trendyol uses Falco to develop a threat detection system by utilizing Kubernetes audit logs and kernel events to monitor user behavior in production clusters. This allows them to detect operational anti-patterns, improve visibility, and identify malicious actors.
R6 case study: Showcasing how R6 Security employs Falco to enhance their moving threat detection platform, Phoenix.

Technical Writing / Documentation
In collaboration with Nate Waddington, we've worked on improving the Falco messaging and enhancing the quality of the Falco documentation, particularly focusing on new user onboarding. Based on his feedback and assessment, we've simplified and updated the Falco getting started guide and provided clear instructions on adopting Falco, including what additional projects users will need to make the most of Falco. Almost all of the website has been revised and improved. For full detail, see the related task list above.
Completed activities worth mentioning include:

Enhancing Falco messaging throughout falco.org and Falco GitHub repositories.
Enhancing clarity about each Falco component.
Improving cross-references between all repositories, plugins, libraries, etc., leading to the main documentation site and the main repository, as well as the evolution repository.
Facilitating a better understanding of Falco content by adding a referenceable glossary on falco.org with frequently used terms. This includes references for individuals to learn more about eBPF, kernel security, syscalls, and more by adding links to third-party official documentation and websites.
Massive reworking on rules documentation with clear instructions on styling and how to use, customize, and fine-tune rules according to various use cases (see the related section below for more detail).

Use Case Development
We have increased clarity on Falco use cases and added two primary use cases on a dedicated falco.org/about/use-cases/ page.
Completed activities include:

Threat Detection use case: Added additional blogs and guides demonstrating how Falco can be used for threat detection. For example:

Defensive Capabilities for Container & Cloud Threats with Tidal

Compliance use case: Added additional blogs and guides showcasing how Falco can be used for Regulatory Compliance. Examples include:

PCI/DSS Controls with Falco
Validating NIST Requirements with Falco

Kernel Version Testing
We've successfully implemented our Driver Kernel Testing Framework proposal. This effort includes the creation of a specific falcosecurity/kernel-testing sub-project featuring Ansible playbooks and Dockerfiles to manage FireCracker microVMs for testing drivers against a range of distros/kernel versions. We've also established CI jobs on the falcosecurity/libs repository to execute tests and generate a markdown table with the results on GitHub pages: https://falcosecurity.github.io/libs/matrix/. The CI is impressively speedy, completing runs in approximately 15 minutes. This accomplishment was made possible thanks to the CNCF Community Infrastructure Lab (CIL), which we leverage to run our CI jobs for kernel testing.
Lastly, we're also considering further enhancements beyond the initial scope of this initiative. So, stay tuned!
Rule Modularity/Assumptions
Building upon the Falco Rules Adoption, Management, and Maturity Framework proposal, we have made substantial progress in the adoption of Falco rules. We have successfully established a rule maturity framework and completed a comprehensive round of tagging and enhancing rules. This includes augmenting descriptions and providing tuning advice for stable rules. The newly introduced style guide has been applied to existing rules.
To streamline the adoption process, we've put forth a thorough style guide and an adoption guide on our website. We have also included a dedicated contributing guide within the rules repository to facilitate participation from contributors. To improve accessibility for adopters, we've introduced an overview document and improved cross-links to official documentation on our website.
Looking ahead, we're working on additional enhancements that are currently in the pipeline, expected to be completed by the Falco 0.36 release.

Awesome this is great!  You all are on track for a Sept release? I want to verify the changes you've put in place with that release to confirm and wrap things up. CC @justincormack

Hey @TheFoxAtWork
Thank you! :pray:
Yes, we're on track for the Falco 0.36 release, due by September 29th. I can confirm that all the items mentioned will be included in this release. Currently, we're in the final stages of wrapping up various sub-components, such as the libs, rules, etc. This phase concludes around the end of August. From September, we'll transition into the "Release Preparation" iteration, focusing on thorough testing, addressing bugs, if any, and minor improvements. You can view a detailed breakdown on our high-level roadmap.

Status update:

We can close this tracking issue since the vote passed.
We're now just waiting for CNCF GB & LC resolutions of a couple of license exception requests (see https://github.com/cncf/foundation/issues/629 and https://github.com/cncf/foundation/issues/645), then we can finally move to the graduation level :partying_face:

/close

@leogr: Closing this issue.

In response to this:

Status update:

We can close this tracking issue since the vote passed.
We're now just waiting for CNCF GB & LC resolutions of a couple of license exception requests (see https://github.com/cncf/foundation/issues/629 and https://github.com/cncf/foundation/issues/645), then we can finally move to the graduation level :partying_face:

/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


iOS plugin installation blocks other plugins installation routine

I'm using cordova in a gulp task and saved cordova-hot-code-push in my config.xml file via cordova plugin add cordova-hot-code-push-plugin --save. If I init my app everything goes fine for android. But all my iOS plugins don't get installed, when your plugin is in the config.xml.
This is part of my gulp task:
return new Promise(resolve => {
  cordova.platform('add', 'android', () => {
  cordova.platform('add', 'ios', resolve);
  });
});

There is no difference between installing Android or iOS version of the plugin. It is handled by Cordova itself, and the only thing is executed from the plugin is a hook, that installs the required node dependencies. If it's working for android - it should work for iOS as well. If it get's blocked - maybe there is something wrong with the gulp task itself.
Haven't used a gulp, so can't really tell right now. Need time to investigate, but not sure that this is a plugin issue.

Ok, thanks! I've build a work around via exec:
exec('cordova plugin add cordova-hot-code-push-plugin', {
    maxBuffer: 500 * 1024,
  },
(err, stdout, stderr) => {
  process.stdout.write(stdout);
  process.stdout.write(stderr);
  if (err) process.stdout.write(err);
  cordova.prepare(resolve);
});

Glad to hear that )


Hello Wold example stuck at ContainerCreating with microk8s

What happened:
I installed microk8s 1.18.0 on Ubuntu 18.04. Enabled dns, storage, ingress.
Then I went through https://github.com/argoproj/argo/blob/master/docs/getting-started.md.
I executed following
kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/install.yaml
kubectl create rolebinding default-admin --clusterrole=admin --serviceaccount=default:default

I installed argo binary, then I did microk8s.kubectl config view --raw > $HOME/.kube/config
Then I tried argo submit --watch https://raw.githubusercontent.com/argoproj/argo/master/examples/hello-world.yaml which is stuck at ContainerCreating
When I run kubectl describe pods, I can see info like this one
  ----     ------       ----              ----                    -------
  Normal   Scheduled    18s               default-scheduler       Successfully assigned default/hello-world-vxd48 to ubuntu-server
  Warning  FailedMount  2s (x6 over 18s)  kubelet, ubuntu-server  MountVolume.SetUp failed for volume "docker-sock" : hostPath type check failed: /var/run/docker.sock is not a socket file

What you expected to happen:
I expected example to run
How to reproduce it (as minimally and precisely as possible):
repeat the steps described in the issue
Anything else we need to know?:
Environment:

Argo version:

$ argo version
argo: v2.7.0
  BuildDate: 2020-03-31T23:35:04Z
  GitCommit: 4d1175eb68f6578ed5d599f877be9b4855d33ce9
  GitTreeState: clean
  GitTag: v2.7.0
  GoVersion: go1.13.4
  Compiler: gc
  Platform: linux/amd64

Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: "2020-03-25T14:58:59Z"
  compiler: gc
  gitCommit: 9e991415386e4cf155a24b1da15becaa390438d8
  gitTreeState: clean
  gitVersion: v1.18.0
  goVersion: go1.13.8
  major: "1"
  minor: "18"
  platform: linux/amd64
serverVersion:
  buildDate: "2020-03-25T14:50:46Z"
  compiler: gc
  gitCommit: 9e991415386e4cf155a24b1da15becaa390438d8
  gitTreeState: clean
  gitVersion: v1.18.0
  goVersion: go1.13.8
  major: "1"
  minor: "18"
  platform: linux/amd64

Other debugging information (if applicable):

workflow result:

argo get <workflowname>

executor logs:

kubectl logs <failedpodname> -c init
kubectl logs <failedpodname> -c wait

workflow-controller logs:

kubectl logs -n argo $(kubectl get pods -l app=workflow-controller -n argo -o name)

Logs
argo get <workflowname>
kubectl logs <failedpodname> -c init
kubectl logs <failedpodname> -c wait
kubectl logs -n argo $(kubectl get pods -l app=workflow-controller -n argo -o name)

Message from the maintainers:
If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

You should checkout your docker sock file path

microk8s uses containerd. I installed version 1.13 which uses docker and then manually applied these
sudo ln -s /var/snap/microk8s/common/var/lib/docker /var/lib/docker
sudo ln -s /var/snap/microk8s/current/docker.sock /var/run/docker.sock

as mentioned in this issue https://github.com/kubeflow/kubeflow/issues/2347
after which it started to work.
My question is how I get it to work with latest microk8s, if possible.

ok, I found a way to make it work with latest microk8s. Seems that the default executor needs to be changed (https://github.com/argoproj/argo/blob/master/docs/workflow-executors.md). The default seems to be docker which doesn't work with latest microk8s.
I added containerRuntimeExecutor: k8sapi to the hello world example (https://raw.githubusercontent.com/argoproj/argo/master/examples/hello-world.yaml
)
for workflow-controller-configmap, so that part looks like this now
apiVersion: apps/v1
kind: Deployment
metadata:
  name: workflow-controller-configmap
data:
  config: |
    containerRuntimeExecutor: k8sapi

It worked with k8sapi


Clarify the unofficial nature of the sped referral packet per Alex's …

…comment on PR #507 @alexsoble

👍 It might even be helpful to make it extra huge, so there's no way someone can miss it.


[Bug]: Simultaneous mm calls lead to permanently degraded performance.

Your current environment

The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64) - WSL
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.11.10 (main, Oct  3 2024, 07:29:13) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.5.119
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4080 SUPER
Nvidia driver version: 555.99
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      39 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             28
On-line CPU(s) list:                0-27
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Core(TM) i7-14700KF
CPU family:                         6
Model:                              183
Thread(s) per core:                 2
Core(s) per socket:                 14
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           6835.20
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
Virtualization:                     VT-x
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          672 KiB (14 instances)
L1i cache:                          448 KiB (14 instances)
L2 cache:                           28 MiB (14 instances)
L3 cache:                           33 MiB (1 instance)
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==<IP_ADDRESS>
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==<IP_ADDRESS>
[pip3] nvidia-cufft-cu12==<IP_ADDRESS>
[pip3] nvidia-curand-cu12==<IP_ADDRESS>
[pip3] nvidia-cusolver-cu12==<IP_ADDRESS>
[pip3] nvidia-cusparse-cu12==<IP_ADDRESS>
[pip3] nvidia-ml-py==12.560.30
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] nvidia-nvjitlink-cu12==12.6.77
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] pyzmq==26.2.0
[pip3] torch==2.4.0
[pip3] torchvision==0.19.0
[pip3] transformers==4.45.1
[pip3] triton==3.0.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-cublas-cu12        <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
[conda] nvidia-cudnn-cu12         <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cufft-cu12         <IP_ADDRESS>                pypi_0    pypi
[conda] nvidia-curand-cu12        <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusolver-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusparse-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-ml-py              12.560.30                pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.77                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
[conda] pyzmq                     26.2.0                   pypi_0    pypi
[conda] torch                     2.4.0                    pypi_0    pypi
[conda] torchvision               0.19.0                   pypi_0    pypi
[conda] transformers              4.45.1                   pypi_0    pypi
[conda] triton                    3.0.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.6.1.dev238+ge2c6e0a82
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X                              N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

Model Input Dumps
No response
🐛 Describe the bug
I don't know whether this is user error, or a bug. But I'm doing multimodal inference and I noticed I could have multiple instances of my script making calls to the model and each would get the same throughput, leading to a higher avg generation throughput. Having noticed this I modified my script to make it asynchronous. But when I run it the avg generation throughput is now worse than if I was just running one instance. And when I go back to running just a single synchronous instance I'm now getting even worse throughput. The only way to get it to return to it's original performance is to restart the model. I have however noticed that if the calls are still asynchronous, but have a staggered delay to them I have no problems.
To clarify with rough numbers:

setup
avg gen throughput (tokens/s)

A single synchronous instance
~50

3 manually started synchronous instances
~150

1 asynchronous instance making 3 calls at a time
~45

A single synchronous instance after running async
~15

Restarted Server
N/A

1 asynchronous instance making 3 staggered calls
~150

Looking at these it appears that something about the way I'm running the asynchronous version hits some sort of "resource limit" and so the amount of resources each call is assigned are limited (15 * 3 = 45). But when running again with a smaller load this "call limit" is still in place. Maybe?
Possibly supporting this theory is the fact that when I run the staggered version the vram usage doesn't get as high. But I don't know why the vram usage is lower because in the end it's still running 3 requests at once. And I don't have enough headroom to truly test it.
You can reproduce with with the below code. Just update the open ai connection to point it at your own model. You can achieve each of the different behaviours by adjusting the num_parallel and delay parameters.
Serving Command:
vllm serve $MODELS/openbmb/MiniCPM-V-2_6 --enforce-eager --max-model-len 4096 --max-seq-len 2048 --served-model-name hello --trust-remote-code --limit-mm-per-prompt image=2 --gpu-memory-utilization 0.95 --quantization fp8

Code:
import asyncio

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import SystemMessage
from langchain_core.runnables import Runnable

from langchain_openai import ChatOpenAI

def get_transcribe_chain() -> Runnable:
    chat = ChatOpenAI(model="hello", base_url="http://localhost:8000/v1", api_key="EMPTY", temperature=0.0)
    prompt_template = ChatPromptTemplate.from_messages([
        SystemMessage(content="You are an expert long format writer."),
        ( "user", [
            { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" } },
            { "type": "text", "text": "Write me a really really long and beautiful description of the image, it should be a least 4000 words." }
        ]),
    ])
    return prompt_template | chat

async def _transcribe_page( semaphore: asyncio.Semaphore, transcribe_chain: Runnable, delay: float) -> None:
    async with semaphore:
        # VLLM seems to complain if you hit it with too many at once.
        # So each of the pages is staggered.
        await asyncio.sleep(delay)
        await transcribe_chain.ainvoke(input={"input": "hi"})

async def _transcribe_pages_async( transcribe_chain: Runnable, num_parallel: int, delay: int) -> None:
    semaphore = asyncio.Semaphore(num_parallel)

    tasks = []
    for page_number in range(1, 101):
        tasks.append(_transcribe_page(
            semaphore=semaphore,
            transcribe_chain=transcribe_chain,
            delay=delay * (page_number % num_parallel),
        ))

    await asyncio.gather(*tasks)

def transcribe_pages( transcribe_chain: Runnable, num_parallel: int=2, delay=2) -> None:
    try:
        loop = asyncio.get_event_loop()
        if loop.is_closed():
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
    except RuntimeError:
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)

    loop.run_until_complete(_transcribe_pages_async(transcribe_chain, num_parallel, delay))

if __name__ == "__main__":
    # Set this to 1 for a synchronous solution and higher for async.
    num_parallel = 3

    # Set this to 0 for true async or provide a float for n seconds staggering.
    delay = 2

    while True:
        transcribe_pages(get_transcribe_chain(), num_parallel, delay)

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Thanks for reporting this! Can you profile the code and see where the bottleneck is occurring?

cc @ywang96

Here are the profiler results. However I got a warning when starting the profiler
INFO 10-11 17:01:53 api_server.py:361] Profiler started.
INFO:     <IP_ADDRESS>:60518 - "POST /start_profile HTTP/1.1" 200 OK
WARNING:2024-10-11 17:01:53 905874:905874 init.cpp:174] function cbapi->getCuptiStatus() failed with error CUPTI_ERROR_NOT_INITIALIZED (15)
WARNING:2024-10-11 17:01:53 905874:905874 init.cpp:175] CUPTI initialization failed - CUDA profiler activities will be missing

I think I've run the profiler. At least I followed the documentation here and found the start_profile and stop_profile routes.
Unffortunately I haven't been able to view the logs. I've tried both perfetto and tensorboard. perfetto keeps throwing a segmentation fault when I try to load the log file and tensorboard was throwing an error TypeError: MessageToJson() got an unexpected keyword argument 'including_default_value_fields'. I tried again after installing tensorboard from source and it just says No dashboards are active for the current data set..
I don't know if this is something I'm doing wrong, a bug with vllm or a bug with the analysis engines. Either way I can't actually view the traces.
If you want to have a look I've uploaded the traces here

That worked, thanks.
You've probably already seen this then, but in case it's useful to anyone else:

Staggered / Baseline:

Async / Experimental:

Also cc @youkaichao regarding this, I don't think this has anything to do with broadcasting though since TP/PP isn't being used here.

Assuming I've done the right thing using VLLM_TRACE_FUNCTION=1, the stack traces have been uploaded as well, though I'm afraid I have no idea how to pull anything useful from them.
I've also had a play around with v0.6.3 and the problem is still largely still present (with both openbmb/MiniCPM-V-2_6 and Qwen/Qwen2-VL-7B-Instruct). However, this version seems to have introduced lower memory usage (at least in this usecase). And under the original conditions with 3 async calls the slow down is no longer present, but after increasing the number of async calls it eventually returns. Looking at the memory usage it appears that the slow down only appears when vllm starts making use of the shared gpu memory (system ram pretending it's vram). But as before, the curious thing is that staggering the calls uses significantly less ram, to the point that I actually run out of kv_cache before it can start using the shared memory.

Sorry, I meant the trace information in the original logs via start_profile and stop_profile. Let me try extracting a subset of the logs and see if they can load on TensorBoard.

For pytorch profiler, you can pass with_stack=True argument to also capture the stack trace for non-Pytorch functions,

Had a go using py-spy, think that might be more what you were after. Uploaded the results with the rest. The actual svg files are interactive so they provide a bit more information than what's shown below. But the speedscope file's are probably more informative.
To look at the speedscope files you'll either need to install speedscope or use the web app

Thanks for providing info! This shows that sampling is taking much longer in the async case. Perhaps there is an issue in the multi-step scheduler? @comaniac @afeldman-nm are you available to look into this?

@SeanIsYoung do you get a similar issue if you don't input any multimodal data?

@DarkLight1337 Using the same model and text only input I can't get it to do this. The async and staggered versions both appear to have similar memory usage.
I'm also unable to push it to the point that the GPU starts using it's shared memory, the KV cache runs out of space first limiting the number of active calls, and I have that set as high it will allow on my system.

Based on the command I don't think multi-step scheduling is enabled, and AFAIK async output processor is disabled when enforcing eager mode. The huge sampling time in the profile may not be accurate, because we launched CUDA kernels in non-blocking fashion, and the sampler has a sync point (aka barrier).
For "3 manually started synchronous instances", did you launch 3 endpoints and adjust the GPU memory utilization to about 30% per instance, or you have 3 GPUs?
In the case that staggered version works, I have an impression that vLLM request queue is not that efficient (probably due to single thread asyncio event loop). So I implemented another queue outside vLLM to prevent my script from sending too many requests at once.

For "3 manually started synchronous instances", did you launch 3 endpoints and adjust the GPU memory utilization to about 30% per instance, or you have 3 GPUs?

No, sorry if I didn't make that clear. All of them were run with only a single instance of the vLLM server running.
What I meant here was that I had a script that would iterate through a directory making calls one at a time to vLLM for each image (The script itself having no async or anything, just a for loop). I then opened another 2 terminal windows and started another two instances of the script pointing at different directories.

I see, so 3 sync instances are actually having 3 processes sending requests, and each of them sends requests sequentially. A more common term for this use case is "concurrency". Then to me it's hard to make a comparison between your async and sync, because from the engine's point of view, the only difference is "QPS" (query per second). I suspect your issue comes from the overhead caused by too high QPS. One thing you can try:

Launch a server. Have 3 processes sending requests sequentially. Monitoring server log and observe Running: xx reqs and Pending: yy reqs.
Re-launch a server. Have 3 processes sending requests asynchronously. Monitoring server log and observe Running: aa reqs and Pending: bb reqs.

Then you can compare these values over time, where the number of running requests is the batch size, and the number of pending requests is the queue length. Basically we need to know which number changes significantly when moving from sync to async.

Yeah you're right, in the end there's not much difference between the 3 sync, staggered async and async. As far as the server is concerned it still ends up running 3 requests concurrently. It's just whether or not it is passed the 3 requests all at the same time, or if they are staggered (as an aside the 3 sync is essentially the same as the staggered, it's just that the delay is the time it takes me to swap to a different terminal and start up the next one).
Consequently the only difference with the running group is that you might see it at 1 or two initially when the calls are staggered, but it will still eventually end up running the 3 requests concurrently, they just won't have started at the same time. That I've seen, the only time anything is put into the pending queue is when the kv cache is full, otherwise they go straight to the running group.
I don't know enough about how vLLM works under the hood to be certain. But I'm guessing that maybe there are a couple things combining here. Firstly, when I send the requests asynchronously / all at the same time, I'm guessing that all the preprocessing is done at the same time maybe, using the same shared resource space, and because they're done at the same time this resource space grows larger than when they are staggered. This guess comes from the fact that for the same number of requests the VRAM usage is higher when the calls are completely async than when they are staggered. And that by looking at speedscope it seems as though all the preprocessing is done in one go for the async situation whereas the staggered seems to have a separate preprocessing call for each request (it could be this is wrong because due to the monitoring of an async process, but I don't know).
Staggered:

Async:

Having run it with 0.6.3 where memory usage is a bit lower I saw that I could run them completely asynchronously with no slow-down, so long as there few enough requests to keep memory requirements within the VRAM. But when those requests push the memory usage into the shared memory it starts to slow down (Which seems reasonable / plausible given that the RAM is slower and may require copying data between it and the GPU).
But when you go back to making fewer requests the slow-down is still present, indicating that either some flag has been set, or maybe now that the cache has expanded into the shared memory, it won't shrink back out of the shared memory.
But I could also be entirely wrong.

So your "sync" is not really "sync"...it's really confusing.
Then what I can think of in summary is batch size 3 has lower throughput than batch size 1, because when you send requests with some delays (you called it "staggered"), vLLM prefills (and encodes images) one request at a time; while when you send requests at once (you called it "async"), vLLM processes them together (batch size = 3). If your requests have a long prompt length, this may be the case. For text models, we solved this problem with chunked prefill, but this is not merged yet for MM models.

So your "sync" is not really "sync"...it's really confusing.

Sorry about that, I was talking in terms of the individual scripts.

Then what I can think of in summary is batch size 3 has lower throughput than batch size 1, because when you send requests with some delays (you called it "staggered"), vLLM prefills (and encodes images) one request at a time; while when you send requests at once (you called it "async"), vLLM processes them together (batch size = 3). If your requests have a long prompt length, this may be the case. For text models, we solved this problem with chunked prefill, but this is not merged yet for MM models.

I think this makes sense.


[Cherry-pick][branch-3.1][Enhancement] Support refresh hive view (#31557)

Fixes #issue
What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[x] Yes, this PR will result in a change in behavior.
[ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5

[FE Incremental Coverage Report]
:heart_eyes: pass : 25 / 30 (83.33%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/server/GlobalStateMgr.java
0
3
00.00%
[3720, 3729, 3730]

:large_blue_circle:
com/starrocks/connector/hive/CacheUpdateProcessor.java
8
9
88.89%
[99]

:large_blue_circle:
com/starrocks/connector/hive/CachingHiveMetastore.java
15
16
93.75%
[386]

:large_blue_circle:
com/starrocks/catalog/HiveView.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/connector/hive/IHiveMetastore.java
1
1
100.00%
[]

[BE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)


[0.35] Build esmodules for some projects

This is a port of PR #5612 to the 0.35 release branch. This will let us publish 0.35 packages with esmodules without waiting for the 0.37 release.
Main branch: #5612
0.36 branch: #5613
0.35 branch: #5614

/azp run Build - client packages


<EMAIL_ADDRESS>Name / Description unclear, what exactly is this binary?

I recently needed to install OpenVPN Connect and by running scoop search I found the manifest called "openvpn". However, I was not able to determine whether this artifact referred to OpenVPN Access Server (which is the selfhosted VPN Server application) or OpenVPN Connect (which is the client application). The description "A flexible virtual private network (VPN) solution to secure data communications." does not really shine light on that either.
I then tried to determine it via the current version on scoop, which is 2.4.11. The OpenVPN Access Server release notes show that a version 2.4.11 didn't even exist (or atleast it was not documented). I could not find any release notes for OpenVPN Connect, but the most recent version is 3.2.3, which does also not match the version in scoop, 2.4.11.
So what exactly is the openvpn manifest? Access Server, Connect, or another application (if yes, which)? As soon as this is determined, I think the name or the description of this manifest should be updated to be clearer about what exactly it represents.

I am not familiar with the software, but it says OpenVPN 2.5.5 on the website. Is there a better name for the package?
Also we should definitely add openvpn-connect.

I've installed OpenVPN-connect v<IP_ADDRESS>00. and run it BUT it was blocked by my antivirus because of malwares. I have to ask is it safe to use the software via scoop?

@Xelbayria
SAFE in most cases, i mean up to 99%.
and NO program can guarantee the security for your OS.
which file(s) are reported?

shim? (<SCOOP>\shims*)
file(s) in the APPDIR (<SCOOP>\apps\openvpn-connect*)

ok. Here's the information below:
C:\Scoop\apps\openvpn-connect\<IP_ADDRESS>00(Command line parameters: "C:\Scoop\apps\openvpn-connect\<IP_ADDRESS>00\OpenVPNconnect.exe" -- relaunch --alow-file-access-from-files "C:\Scoop\apps\openvpn-connect\<IP_ADDRESS>00\OpenVPNConnect.exe")
That is the reported file.  The Detection ID: SuspiciousBehavior.50F31B667742D14A
I'm not sure if ID helped at all.

just over-sensitive.

just over-sensitive.

what do you mean by that?

The reported file is OpenVPN itself, so if you think the software is safe, it is safe. Typically it is reported since it is a VPN.

The reported file is OpenVPN itself, so if you think the software is safe, it is safe. Typically it is reported since it is a VPN.

ah, that's what "over-sensitive" means. Got it

@Xelbayria
i mean the antivirus you are using is over-sensitive, not you.

@Xelbayria i mean the antivirus you are using is over-sensitive, not you.

You really don't need to explain any further.  I already understand what niheaven is saying.


How to work with Slots having thresholds that change decisions ?

Rasa Core version: 0.9.0a3
Python version:  Python 2.7
Operating system (windows, osx, ...): Linux
Issue:
In the below link:
https://core.rasa.com/master/patterns.html
I want to include the below the code into my rasa-bot :
Using Slot Values to Influence Which Actions are Predicted
Custom Slots
from rasa_core.slots import Slot

class NumberOfPeopleSlot(Slot):

  def feature_dimensionality(self):
      return 2

  def as_feature(self):
      r = [0.0] * self.feature_dimensionality()
      if self.value:
          if self.value <= 6:
              r[0] = 1.0
          else:
              r[1] = 1.0
      return r

I have a slot named 'num_people' in my '.yml' file which has limitations on the value. How do I connect the above class "NumberOfPeopleSlot" to my slot 'num_people' ? I couldn't find how to do that in the documentation ?

Can someone tell me what is the variable 'type_name' in line 14 in the below link ?
https://github.com/RasaHQ/rasa_core/blob/0.9.0/rasa_core/slots.py

if you make this code part of a module, you can refer to it in your domain yml by its path:
you can e.g. save it as a file : my_slots.py and in your yaml write my_slots.NumberOfPeopleSlot

Consider the below to be my current 'yaml' file. Can you tell me where to write my_slots.NumberOfPeopleSlot if I want it work on the slot "people" ??
intents:
 - greet
 - default
 - goodbye
 - affirm
 - thank_you
 - change_bank_details
 - simple
 - hello
 - why
 - next_intent

entities:
 - name

slots:
  name:
    type: text
  people:
    type:text

templates:
  utter_greet:
    - "hey there {name}!"
  utter_goodbye:
    - "goodbye :("
    - "bye bye"
  utter_default:
    - "default message"

actions:
  - utter_default
  - utter_greet
  - utter_goodbye

Thanks in advance,

you would set the type of the people slot to my_slots.NumberOfPeopleSlot, so:
people
    type: my_slots.NumberOfPeopleSlot

Got it.
It is not intuitive. So, I think you should add it in the documentation as well.
Thanks,

Thanks for the feedback! I'll close this issue for now, let us know if there's other issues.

Can we add this small information into the documentation?

Shouldn't it be possible for the stories to be trained using the class labels rather than individual examples?
If I have 3 classes - 'a' = x < 1000, 'b' = 1000 < x < 5000, 'c' = 5000 < x, I should be able to specify my actions using these class labels instead of giving examples of values x can take like 454, 1023, 6035, etc.

Please post these kind of questions in our forum


catch_ros2: 0.1.0-1 in 'iron/distribution.yaml' [bloom]

Increasing version of package(s) in repository catch_ros2 to 0.1.0-1:

upstream repository: https://github.com/ngmor/catch_ros2.git
release repository: https://github.com/ros2-gbp/catch_ros2-release.git
distro file: iron/distribution.yaml
bloom version: 0.11.2
previous version for package: null

catch_ros2
* Initial Release
* Contributors: Nick Morales

Iron is held for sync https://discourse.ros.org/t/preparing-for-iron-sync-2023-06-30/32099


Example S3_rag broken?

Cloned the repo 8/29/2024, configured the S3 bucket, ran make dev-serve.
Error:
make serve-dev
Makefile:3: *** missing separator.  Stop.

You fixed it! 😄


comments are not preserved

TypeScript Version:  1.9.0-dev.20160217
Code
class Foo {

    // This comment will disappear
    public bar: string | number;
     constructor(abc: string | number) {
this.bar = abc;
}
}

Expected behavior:
This comments should still be there after the code have been transpilled down to ES6 / ES5
Actual behavior:
Comments are removed even if "removeComments": false, in tsconfig.js..

The bar property is not initialized anywhere, so where exactly would you want the comment to appear?

For the record, we do emit the comment if you have an initializer.
Input:
class Foo {

    /** This comment will disappear*/
    bar = 10;

    constructor(abc: string | number) { }
    }
}

Output:
var Foo = (function () {
    function Foo(abc) {
        /** This comment will disappear*/
        this.bar = 10;
    }
    return Foo;
}());


simulator: add lockstep initialized logging

After quite a look of debugging I may have found the problem. https://github.com/PX4/sitl_gazebo/pull/631


(BOLT-794) Unwrap sensitive data

Problem
Currently, when passing a Sensitive string to a task or script, the data is passed as a Sensitive class instead of a string, resulting in an error. For more details, see:
https://tickets.puppetlabs.com/projects/BOLT/issues/BOLT-794
Solution
We need to "unwrap" the underlying string from the Sensitive wrapper before invoking the task or script.
We also need to modify our type check in run_task() to compare against RichData instead of Data because Data does not contain Sensitive in its type alias.
TODO

[ ] Add unit tests
[x] Produce a helpful error message about the type mis-match problems

Added helpful error message for future issues / type mis-matches when calling run_task:
Example (this was a forced error, couldn't actually figure out how to generate a bad type that wasn't accepted by this alias, suggestions welcome!):
{
  "kind": "bolt/pal-error",
  "msg": "Task parameters is not of type RichData (Variant[Scalar,SemVerRange,Binary,Sensitive,Type,TypeSet,URI,Object,Undef,Default,Hash[RichDataKey,RichData],Array[RichData]]) it is of type: Struct[{'key' => String, 'decrypt' => Boolean[true], 'api_key' => Sensitive[String]}] (file: /path/to/bolt/plans/xxx.pp, line: 55)",
  "details": {
  }
}

This looks like it's probably a good idea, but I want to better understand RichData in general to see the ramifications. It looks like the intention is primarily to accept sensitive data and keep it from being logged.

CLA signed by all contributors.

@MikaelSmith yes, the only reason i switched to RichData was to allow for Sensitive types to be accepted

OK i think this is good to go, except for the fact that some transports (winrm and ssh) both log full commands and full outputs when running with --debug.
Few options i see:

Pass in the "structured" data all the way down into the transports so it is still wrapped with Sensitive up until the moment before we send it over SSH / WinRM.
Remove the logging code within the transports that exposes the sensitive data
Leave it and let people see it if they pass --debug

Talking with @MikaelSmith and he liked option "3. Leave it and let people see it if they pass --debug"
If this is the choice then everything is good to go.


Focus first element in scope if nodeToRestore does not exist

In collaboration with @snowystinger. Builds on #3323.
Closes #3171. Closes #3128.
This fixes a few related issues:

In cases like the Modal unmounting trigger story, focus would not move into a Dialog within a DialogContainer. The underlying reason is that DialogContainer is not a child of the Popover in the JSX tree, so the Popover's FocusScope was containing focus within it when the dialog opened (here). This is fixed by using the currently active scope (i.e. the popover) as the parent scope rather than the parent in the JSX tree, but only if the JSX parent is not already inside the active scope. This way, the focus scope within the dialog appears to be nested within the popover, and focus is allowed to move into it.
In cases where the node to restore is deleted from the DOM, and there's no parent focus scope to walk back up to. For example, if you delete a row from a table in the CRUD story, or in the ActionBar examples, focus is lost to the body. This is fixed by wrapping collection components such as Table in a FocusScope, and walking up the scope tree from the dialog until we find a scope that still exists (the table), and then focusing the first item within that scope.
In a similar case to (2), the ActionGroup in the CRUD story removes its delete button when nothing is selected, so when restoring focus from the delete dialog, focus should go back to the plus button instead. The fix is the same (wrapping ActionGroup in a FocusScope), but this also required tracking the active scope when both contain and restore are false.

Also related to #3039: focus now goes to the collection element when an item is removed rather than lost to the body, but still would be good to focus a more adjacent row at some point.
To do:

[ ] Add FocusScope around all collection components where items may be deleted
[ ] Figure out what to do about React Aria examples
[ ] Tests

This seems to resolve #3128, closing #3171, but #2451 to fix focus restoration from the ActionBar to the TableView in this ActionBar Example, still fails intermittently to scroll the element to which focus should be restored within the TableView back into view when restoring focus from the Close/Cancel button within the ActionBar.

@majornista I think this was fixed by merging in main, which includes https://github.com/adobe/react-spectrum/pull/3168. That will now keep the focused row in the DOM even when scrolled out of view, instead of shifting focus to the collection. I believe what was happening before was that row element was reused when scrolled out of view, and focus was restored back to that instead.

@majornista I think this was fixed by merging in main, which includes #3168. That will now keep the focused row in the DOM even when scrolled out of view, instead of shifting focus to the collection. I believe what was happening before was that row element was reused when scrolled out of view, and focus was restored back to that instead.

Seems to be working as expected now.


Test discrepancies between 9.4 and 9.5

I'm not sure if there's a plan to support one or both of 9.4 and 9.5?
When running the tests against 9.4 there are two failures.
  1) Document updates, Save data and update, updates the document:
     Uncaught TypeError: Cannot read property 'vaccinated' of null
      at Object.next (test/document_update_spec.js:30:25)
      at Object.next (lib/document_table.js:131:12)
      at null.callback (lib/runner.js:53:16)
      at Query.handleError (node_modules/pg/lib/query.js:106:17)
      at null.<anonymous> (node_modules/pg/lib/client.js:171:26)
      at Socket.<anonymous> (node_modules/pg/lib/connection.js:109:12)
      at readableAddChunk (_stream_readable.js:146:16)
      at Socket.Readable.push (_stream_readable.js:110:10)
      at TCP.onread (net.js:523:20)

  2) Document updates, Save data and update, updates the document without replacing existing attributes:
     Uncaught TypeError: Cannot read property 'score' of null
      at Object.next (test/document_update_spec.js:37:25)
      at Object.next (lib/document_table.js:131:12)
      at null.callback (lib/runner.js:53:16)
      at Query.handleError (node_modules/pg/lib/query.js:106:17)
      at null.<anonymous> (node_modules/pg/lib/client.js:171:26)
      at Socket.<anonymous> (node_modules/pg/lib/connection.js:109:12)
      at readableAddChunk (_stream_readable.js:146:16)
      at Socket.Readable.push (_stream_readable.js:110:10)
      at TCP.onread (net.js:523:20)

When running the tests against 9.5 there are two different failures.
  1) On spin up loads up functions:

      AssertionError: 21 == 20
      + expected - actual

      -21
      +20

      at Context.<anonymous> (test/loader_spec.js:31:12)

  2) Synchronous Load loads up functions:

      AssertionError: 21 == 20
      + expected - actual

      -21
      +20

      at Context.<anonymous> (test/loader_spec.js:64:12)

Would be good to get the tests passing against one or both of the versions.

@robconery may have some specifics to add, but I think we may need to find a way to support both.
I also think there are some features in 9.5 worth taking advantage of ("upsert" being one of them).
I hope to dig in and see what's what with 9.5 this weekend.
That second test looks like it's simply not finding all of the functions to load. May or may not be related to the version change.

Might be possible to run some tests conditionally depending on the version of Postgres? Then it's just a case of clearly documenting what's supported in each version. Would definitely be good to take advantage of upsert whilst maintaining support for 9.4.

@robinjmurphy My thoughts precisely, on all points.


check pointer if not cast ex: (char*), message print all find ex: if …

I changed the method for misplaced pointers.
Maintained, the pointers in parentheses are no longer marked for the casting.
The message is also more precise, it will display all of them incorrectly placed in the line
examples:
if there is one: << have one Misplaced pointer: 'char *' >>
if there are several: << Misplaced pointer: 'char * | int * '>>
I also added the void * which was not there

Thanks 🚀


[Question]What's the best way to define --mode dev and production?

I've just started using webpack-serve, works great :). However I'm not sure how to define the mode to be used in webpack-serve. I tried a config via a function exported, which seems not supported. When I console log the mode, it says development, but when webpack compiles it warns me that the mode is not defined and defaults to production.
Does webpack-serve default to development mode or not? I think compilation is pretty slow, so I'm guessing it is indeed using production mode. There seems to be no --mode flag for webpack serve to solve this.

I found that it's enough to add "mode: 'production'," between
"module.exports = {" and
"module: {" in webpack.config.js, to make the mode warning go away.
Hope this is clear and helpful...
btw: Webpack (4) defaults to production mode when not configured, not webpack-serve if i'm right.

Sad panda 🐼 😢  You removed the issue template, so we have to close your issue as invalid. Please don't ignore Issue or Pull Request templates.


Fix sync and async auth checker bug

Closes #282, fixes #262, resolve #240.

When version 0.7.3 release? because i really need this feature in my koa project.

it has been released yesterday

@pleerock thanks just saw it


Customizing errors

It would be great to have a way to customize errors.   This could potentially be tied into #124, but having some sort of callback that returns (or throws?) errors would be super useful.  Customizing the error message would be an improvement, but ideally I think throwing custom errors (so that extensions can be added) would be more useful.

Do you think we need sort of global callback - or per plugin callback?

A global callback would make some things easier, but I imagine people will want to customize messages based in how things failed.  For example for complexity errors, they might want to include the complexity and/or the limit.  It would be hard to have a global callback that provides the right details about an error in a type-safe way that works for every potential error in a variety of plugins.   I don't have a great solution here, but I can see a couple of options:  a global callback that receives error instances, plugins throw sub-classes of a specific error.  In a global handler you could do something like error => error instance of ComplexityError ? new Error(complexity limit: error.details.complexity) : error  obviously this is over simplified, but the idea is using specific error classes that are documented and exported from plugins to allow catching/replacing errors.  The alternative is to do something on each plugin like onResult(details: PluginSpecificDetails, error: GraphQLError | null) => GraphQLError | null for each plugin.  This pattern would allow you to combine the use case of #124 to have a callback for logging/analytics.  Each plugin can surface captured information (like calculated complexity) regardless of if the plugin actually failed or not.  This solves a lot of important use cases like tracking complexity for things like rate limiting, writing metrics, etc.
I didn't include this in the snippets above, but callbacks probably should also have access to the request context so that context can be used to access things like current user, request logger instances, or whatever else they might need.

Our main use case for having global (or per plugin) error handler is the original concern raised in https://github.com/Escape-Technologies/graphql-armor/issues/124 (observability). We want to ship graphql-armor to production but are still afraid we might have set our limits too conservatively for some operations that we don't control and thus break them after the rollout. We want to track what kind of operations are rejected via GraphQL armor in order to adjust our limits as required/desired.
Regarding the callback:
onError could write to a remote service and due to the function signature limitations of parse and validate where most of these plugins perform (cannot return a Promise ), onError could only be a non-async function, which makes stuff hard for serverless/edge environments as you would want to write the error to some async storage and wait until that is done. But this is nothing one could workaround, to be honest.
let queue = Promise.resolve();

const errorHandler = (errors: Array<GraphQLError>) => {
  const promise = reportErrors(JSON.stringif(errors));
  queue = queue.then(() => promise);
};

const envelop = getEnveloped({
  plugins: [
    maxTokenPlugin({ onError: error => errorHandler(error) }),
    maxDepthPlugin({ onErrors: errors => errorhandler(errors) })
  ]
});

// << do graphql logic >>

// << finally >>

await queue;

// << tear down handler >>

Example using cloudflare workers:
addEventListener('fetch', event => {
  event.respondWith(handleRequest(event, event.request))
})

/**
 * Fetch and log a given request object
 * @param {Request} request
 */
async function handleRequest(event, request) {
  let queue = Promise.resolve();

  const errorHandler = (errors: Array<GraphQLError>) => {
    const promise = reportErrors(JSON.stringif(errors));
    queue = queue.then(() => promise);
  };

  const envelop = getEnveloped({
    plugins: [
      maxTokenPlugin({ onError: error => errorHandler(error) }),
      maxDepthPlugin({ onErrors: errors => errorhandler(errors) })
    ]
  });

  // << Build response etc >>
  const response = await runGraphQLLogic(envelop)
  event.waitUntil(queue)

  return response
}

One thing I want to call out for the observability side of things is that it will be important to observe non-failure cases as well.  For example, I want to be able to write metrics for complexity of queries so we can get max/average/percentiles for complexity, so we can initially set our complexity limits high, and tune them based in actual usage without having to recalculate the scores outside the plugin

#206 will introduce custom callbacks per plugins along with ability to either throw or report error to the current context (#162)


Forge server crashing on startup

Ah, you beat me to it. Had this same issue on my server. This should be a easy fix.

This is the Fabric repository of Tiered; I'm not associated with the Forge port. I think you'll have to contact the other author on CF.