Implement closures

Description
This is a placeholder issue for implementing closures as described in https://github.com/stan-dev/design-docs/blob/master/designs/0004-closures-fun-types.md
I say placeholder cause that's the main specification, and there will also be associated pull requests in stanc3.
Current Version:
v3.3.0

The stanc3 pull request is open: stan-dev/stanc3#742
Probably has lots of bugs but it's good enough that I can compile Lotka-Volterra model:
data {
  int<lower = 0> N;          // number of measurement times
  real ts[N];                // measurement times > 0
  vector[2] y_init;            // initial measured populations
  real<lower = 0> y[N, 2];   // measured populations
}
parameters {
  real<lower = 0> alpha;
  real<lower = 0> beta;
  real<lower = 0> gamma;
  real<lower = 0> delta;
  vector<lower = 0>[2] z_init;  // initial population
  real<lower = 0> sigma[2];   // measurement errors
}
model {
  functions
  vector dz_dt(real t,      // time
               vector z     // system state {prey, predator}
               ) {
    real u = z[1];
    real v = z[2];

    real du_dt = (alpha - beta * v) * u;
    real dv_dt = (-gamma + delta * u) * v;

    return [ du_dt, dv_dt ]';
  }
  vector[2] z[N]
  = ode_bdf(dz_dt, z_init, 0., ts);

  alpha ~ normal(1, 0.5);
  gamma ~ normal(1, 0.5);
  beta ~ normal(0.05, 0.05);
  delta ~ normal(0.05, 0.05);
  sigma ~ lognormal(-1, 1);
  z_init ~ lognormal(log(10), 1);
  for (k in 1:2) {
    y_init[k] ~ lognormal(log(z_init[k]), sigma[k]);
    y[ , k] ~ lognormal(log(z[, k]), sigma[k]);
  }
}

I converted the sir model over here to practice with this. (@rok-cesnovar thanks for the binaries)
It is an absolute delight to not worry about passing in parameters or data.
A few comments:

I get error when I define a function in the transformed parameters block (example model included down the page and data at the bottom):

sir_negbin_lambda.hpp:150:7: error: redefinition of 'sir_L23C9_cfunctor__'
class sir_L23C9_cfunctor__ {
      ^
sir_negbin_lambda.hpp:105:7: note: previous definition is here
class sir_L23C9_cfunctor__ {
      ^
sir_negbin_lambda.hpp:244:1: error: redefinition of 'sir_L23C9_impl__'
sir_L23C9_impl__(const int& N, const T1__& beta, const T2__& gamma,
^
sir_negbin_lambda.hpp:190:1: note: previous definition is here
sir_L23C9_impl__(const int& N, const T1__& beta, const T2__& gamma,

If there is a parameter y defined in the transformed parameters block, I cannot have an argument to my function named y. I guess to allow this we would need to add overloading to the language, which is out of the scope of this?

I'm not sold on the naming. What about:
function dz_dt =  (real t, vector z) -> vector { ... }
I just copied the C++ lambda return type syntax. I wouldn't mind automatically deduced types there. Would this then make it possible to pass the functions as arguments without first defining them as variabies, i.e., ode_bdf((real t, vector z) -> vector { ... }, ...)?

Here is the sir model:
data {
  int<lower=1> n_days;
  vector[3] y0;
  real t0;
  real ts[n_days];
  int N;
  int cases[n_days];
}
transformed data {
  real x_r[0];
  int x_i[1] = { N };
}
parameters {
  real<lower=0> gamma;
  real<lower=0> beta;
  real<lower=0> phi_inv;
}
transformed parameters{
  vector[3] y[n_days];
  real phi = 1. / phi_inv;

  functions
  vector sir(real t, vector x) {
    real S = x[1];
    real I = x[2];
    real R = x[3];

    real dS_dt = -beta * I * S / N;
    real dI_dt =  beta * I * S / N - gamma * I;
    real dR_dt =  gamma * I;

    return [ dS_dt, dI_dt, dR_dt ]';
  }

  y = ode_rk45(sir, y0, t0, ts);
}
model {
  //priors
  beta ~ normal(2, 1);
  gamma ~ normal(0.4, 0.5);
  phi_inv ~ exponential(5);

  //sampling distribution
  cases ~ neg_binomial_2(y[, 2], phi);
}

generated quantities {
  real R0 = beta / gamma;
  real recovery_time = 1 / gamma;
  real pred_cases[n_days];
  pred_cases = neg_binomial_2_rng(y[, 2], phi);
}

Lotka volterra model above but with rk45 solver:
data {
  int<lower = 0> N;          // number of measurement times
  real ts[N];                // measurement times > 0
  vector[2] y_init;            // initial measured populations
  real<lower = 0> y[N, 2];   // measured populations
}
parameters {
  real<lower = 0> alpha;
  real<lower = 0> beta;
  real<lower = 0> gamma;
  real<lower = 0> delta;
  vector<lower = 0>[2] z_init;  // initial population
  real<lower = 0> sigma[2];   // measurement errors
}
model {
  functions
  vector dz_dt(real t,      // time
               vector z     // system state {prey, predator}
               ) {
    real u = z[1];
    real v = z[2];

    real du_dt = (alpha - beta * v) * u;
    real dv_dt = (-gamma + delta * u) * v;

    return [ du_dt, dv_dt ]';
  }

  vector[2] z[N] = ode_rk45_tol(dz_dt, z_init, 0., ts,
                                1e-5, 1e-3, 500);

  alpha ~ normal(1, 0.5);
  gamma ~ normal(1, 0.5);
  beta ~ normal(0.05, 0.05);
  delta ~ normal(0.05, 0.05);
  sigma ~ lognormal(-1, 1);
  z_init ~ lognormal(log(10), 1);
  for (k in 1:2) {
    y_init[k] ~ lognormal(log(z_init[k]), sigma[k]);
    y[ , k] ~ lognormal(log(z[, k]), sigma[k]);
  }
}

Lotka volterra data: lv.dat.txt
Sir data: sir.dat.txt (used the code here to generate the data file: https://mc-stan.org/users/documentation/case-studies/boarding_school_case_study.html)

@nhuurre does this accurately reflect the todo on this:

[ ] Closure support for ODEs #2094
[ ] Closure support for reduce_sum #2094
[ ] Closure support for algebra solver
[ ] Closure support for integrate 1d (I have a branch doing variadic integrate_1d somewhere, so can do this without much trouble)
Stanc3

[ ] Implementation https://github.com/stan-dev/stanc3/pull/742
[ ] Check against design-doc https://github.com/stan-dev/design-docs/blob/master/designs/0004-closures-fun-types.md (I think we could update the design doc if there's a difference here and we prefer the difference)

Docs

[ ] Language reference
[ ] User's guide

I don't know how we could get this to work with map_rect because I do not know how it works. @wds15 do you have the time or inclination to inspect this?
We're trying to plan out 2.26 (https://github.com/stan-dev/cmdstan/issues/957) and this is a feature that's pretty far into development.

Stan Math has integrate_dae function but it's not exposed in the language so I guess it doesn't need closure support.
I think the only tricky part with map_rect is supporting MPI.
I deviated quite a bit from the design-doc. Main points:

The doc does not mention special suffixes _lpdf/_rng/_lp at all. These would be separate function types. I didn't allow them because they don't seem that necessary and while _rng/_lp are straightforward to implement the _lpdf/_lupdf distinction is a can of worms.
No lambda expressions because I don't think they add anything.
The design doc seems to imply that code like this is valid:

real(real) f;
if (sq) {
  f = (real x) { return x*x; }
} else {
  f = (real x) { return 2*x; }
}

but keep in mind that C++ lambdas do not allow this! Despite the two lambdas having the same signature their C++ types are distinct and incompatible.
Of course it would be possible to work around that but I don't see the point. Stan isn't a functional programming language that encourages passing around lots of small functions. So I'm going with no function reassignments. You write your function definition once and it's not going to change afterwards.

I deviated quite a bit from the design-doc

Cool. Reasons seem good to me. Are the known bugs fixed in the compiler now? I vaguely remember I got an ode and a reduce_sum running, but I might be misremembering the second thing :D.
If it is, I wanna build a cmdstan tarball with the mac binaries so that @bob-carpenter can test. He said he'd be down to check this against the design doc (and presumably update the design doc or suggest changes).

Are the known bugs fixed in the compiler now?

I did push a fix for the redcard model.
Now, looking into special suffixes, I see that this (invalid) model causes a C++ compiler error.
functions {
  real foo(real(real) f) {
    return f(0) + f(1);
  }
  real bar_rng(real s) {
    return normal_rng(s,1);
  }
}
transformed data {
  real z = foo(bar_rng);
}

But models that should compile still do compile correctly as far as I know.

Cool I'll build a tarball then!

Yo @bob-carpenter this contains a cmdstan & stanc3 that can compile closures: cmdstan-closures.zip
The goal here is to check the implementation with the design doc, and update stuff as needed.
To keep the zip file under 10 megabytes so Github would let me upload it I removed the stan and math repos from there. Once you unzip that, you should be able to go  into the folder and run the command bash checkout.sh and it'll check out the stan and math repos you need to make this run.
Implementation of that script is just this:
git clone --depth=1 --single-branch --branch=develop https://github.com/stan-dev/stan stan
git clone --depth=1 --single-branch --branch=feature/ode-closures https://github.com/nhuurre/math.git stan/lib/stan_math

There's a Lotka Volterra one in this comment that you can use to test your build is working.

@nhuurre I'm not sure what to do about map_rect still. I put up a quick pull for integrate_1d (https://github.com/stan-dev/math/pull/2397) and it probably won't be too hard to do a similar one for the algebra solvers.
I guess the worst thing that could happen is we make closures that aren't MPI compatible, or are just really bad in some way. A less worse thing would be that we make sure our closures are MPI-friendly, but just don't implement them for map_rect. Best would be we implement map_rect.
@wds15 I don't want to distract you from adjoint ODEs, so the easiest thing I can think of is we do a Hangouts next week or something (@nhuurre you're invited too ofc. if you wanna come). I can walk you through closures and then we can talk about what it'd take to make them map_rect friendly?

I need to catch up on this. It looks super cool. Can we drop mpi map eect support? Is that an option?

So I am scheduled for a hangout next week?

Pre-Stan meeting work? Or I'm free any day at the Stan time or the hour before.

Can we drop mpi map eect support?

I wouldn't mind doing that. I wanna just give it one try before going that direction, but I also don't wanna keep delaying closures. It would be pretty easy to just throw runtime errors in Math, and presumably stanc3 could have an error on compile ("map_rect does not support closures" or something), so it wouldn't be the worst outcome in the world. But it also might not be that bad to do map_rect, I just don't know how it works.

Today or tomorrow 10h am your time / 16h my time would work. Maybe easiest to email me? CU

I would vote against removing MPI map_rect support.
What we could do if making closures work here for MPI would be a ton of work is make stanc3 aware that its compiling for MPI, like we do for OpenCL (when STAN_OPENCL is set, stanc3 is called with --use-opencl) and then report a semantic error if closures are used with map_rect if MPI is on.

I am not saying to cancel MPI map_rect in its current form. It‘s just that we may not want closures to be delayed significantly due to MPI stuff. Thus, we could roll out closures without MPI support (meaning that map_rect MPI will not work).
(Map_rect with MPI would be quite nice, of course)

It‘s just that we may not want closures to be delayed significantly due to MPI stuff. Thus, we could roll out closures without MPI support (meaning that map_rect MPI will not work).

Oh, I am definitely on board with that.

Talked to @wds15 this morning about map_rect. It looks difficult. I think we would need to be able to serialize the functors somehow and also handle the optimization where data is only shipped once to the worker nodes (which neither thing straightforward).
Let's do integrate_1d first (#2397), and then algebra_solver, and then come back to make a try at map_rect.

Not a biggie but jw, is it possible in the spec to have anonymous lambdas like
  vector[2] z[N] = ode_rk45_tol(functions vector (real t, vector z) {
    real u = z[1];
    real v = z[2];

    real du_dt = (alpha - beta * v) * u;
    real dv_dt = (-gamma + delta * u) * v;

    return [ du_dt, dv_dt ]';
  }, z_init, 0., ts, 1e-5, 1e-3, 500);

If the compiler could handle type checking of the inputs and return in C++ this would just translate to
  std::vector<Eigen::Matrix<scalar_t, -1, -1>> z =
    ode_rk45_tol([](auto&& t, auto&& z) {
      // compiled body same as current impl
  }, z_init, 0., ts, 1e-5, 1e-3, 500);


Add missing languages

Closes #2873.
I'm not entirely sure why we need some of these made-up or extinct languages... but on the other hand if people want 'em, why not?

@jesseallhands could you review when you get a chance please?


Integrations with IFRAMEs: Some attachment types won't play in Brightspace

"Found in release/6.4-QA2 - #8882 / 6.4.r1348
I just tried adding all sorts of attachments to Brightspace as quicklinks.
Most of them played OK, except Googlebooks, Kaltura and Flickr.
(But they work OK if I add them to Moodle.)
Issue Converted from original Equella JIRA

Comments from Original JIRA Ticket
Quite possibly it's a HTTP/HTTPS issue. Looks like Flickr now insists on HTTPS anyway (you enter HTTP and it redirects to HTTPS)
Flickr is preventing the site being displayed in an iframe. The only thing we could do about that is to popup into a new window.
Confirmed that it's the same issue with Googlebooks. The only solution would be to do the following:
Change both of the viewers to render some Javascript that detects if it's in a frame or not. If it is, popup the real viewer URL in a new window, otherwise redirect to the real viewer URL.
In any case, the issue is not new. It would affect all integrations that embed the Equella content in an iframe.


I get garbage in HelloFX (Mac OSX Catalina)

Just built HelloFX per the instructions and I get line of unreadable garbage instead of the expected messages. Screen shot here:
http://blackcap.name/blog/new/wp-content/uploads/2020/01/Screen-Shot-2020-01-04-at-17.34.11.png
This is with javafx-sdk-13.0.1.
java -fullversion says:
openjdk full version "13.0.1+9"

JBS is JDK Bug System
The OpenJFX-docs issue page is to deal with an issue with the sample/documentation itself, and not with an issue in JavaFX
You have a reproducible example. Shoot a mail to OpenJFX-dev mailing list. The list has most JavaFX platform developers who can help you with any issue with the platform
All platforms can have issues. A quick Google search landed me to this issue JDK-8234916, which shows that this issue with JavaFX 13 has been fixed


Could not find browser on Elastic Beanstalk AWS

Steps to reproduce
Tell us about your environment:

Puppeteer version: 7.0.1
Platform / OS version: Linux on Elastic Beanstalk AWS
URLs (if applicable):
Node.js version: 12.16.1

What steps will reproduce the problem?
Try the demo code to launch puppeteer
Please include code that reproduces the issue.
const browser = await puppeteer.launch({
        headless: true,
        args: ['--no-sandbox', '--disable-setuid-sandbox']
    });

What is the expected result?
It is able to start in my local. When I deployed in my Elastic Beanstalk AWS I am getting this error.
What happens instead?
It throws error
Error: Could not find browser revision 848005. Run "PUPPETEER_PRODUCT=firefox npm install" or "PUPPETEER_PRODUCT=firefox yarn install" to download a supported Firefox browser binary.

at ChromeLauncher.launch (/var/app/current/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:80:27)

@nvquangth Did you manage to fix that issue on your end? How did you do it if you don't mind sharing?


It's impossible to zoom in on items on the right or bottom of the diagram.

The zoom "+" and "-" buttons scale the diagram with the top left corner as an anchor point, so zooming in pushes items on right and bottom out of the viewport. For bigger diagrams, this may become a usability issue.
Proposals for improvement:
Preferred : Add a "hand" icon that allows you to drag the view to add pan functionality (most intuitive)
Alternatively : When an element is selected, use a point on the selected element to serve as anchor point for the zoom operation instead of the top-left corner.

Hello @meetinthemiddle-be
Very good suggestions, it would be good to have both the 'hand' and also to centre the zoom on the selected element (if one is selected).
We are looking for contributors, and this enhancement would make a good self-contained task

transferring this issue to repo:
https://github.com/mike-goodwin/owasp-threat-dragon-core/issues
as this is the main diagram repo for both desktop and web application versions


Add ability to schedule notifications about upcoming events

Text notifications are by default sent out a month before, week before, day before, and on the day of the event. We should allow users to choose which days or on which schedule that notifications should be sent.

I'm thinking that we'll restructure the documents for events too have their sentNotifications look something like this:
sentNotifications {
  [
    {
      hoursBefore: 24, // day of
      usersNotified: [<array of user IDs>]
    },
    {
      hoursBefore: 48, // day before
      usersNotified: [<array of user IDs>]
    },
    {
      hoursBefore: 168, // week before
      usersNotified: [<array of user IDs>]
    },
    {
      hoursBefore: 720, // month before
      usersNotified: [<array of user IDs>]
    },
  ]
}

This could be how the default notification schedule will look, but the user should have the option to define their own.


FR: Return Track Numbers

Last one for now, I'm already convinced this is better than Spotify :). I'm pretty sure Youtube Music can return Track Numbers when the track is from an album - but obviously this isn't always relevant for YouTube videos. Is it possible to return them so that they're present when we do a search or retrieve Mopidy's tracklist?

Ok, I can have a look at that. Shouldn't be too hard.
One of the tricks for me is to continue to have the three ways of accessing YouTube (YouTube API, scraping YouTube, and YT Music through ytmusicapi) working (and working at an acceptable speed).
The plugin predates YouTube Music, but I've had in mind for a while to properly support ytm albums and songs - I started doing some of the work, but life got busy. Track numbers and artists are two good examples of things that ytm brings that are absent (or, at best, unreliable) in normal YouTube.
Anyway, I'll see what I can do.
Not difficult to be better than Spotify when Spotify is irreparably broken! (Hopefully Google doesn't break this for us - it's happened before...)

Ok, I can have a look at that. Shouldn't be too hard.
One of the tricks for me is to continue to have the three ways of accessing YouTube (YouTube API, scraping YouTube, and YT Music through ytmusicapi) working (and working at an acceptable speed).
The plugin predates YouTube Music, but I've had in mind for a while to properly support ytm albums and songs - I started doing some of the work, but life got busy. Track numbers and artists are two good examples of things that ytm brings that are absent (or, at best, unreliable) in normal YouTube.
Anyway, I'll see what I can do.
Not difficult to be better than Spotify when Spotify is irreparably broken! (Hopefully Google doesn't break this for us - it's happened before...)

I couldn't see where track numbers are returned by ytmusicapi, but YTMusic.get_album does return a list of tracks: assuming that the tracks are always in the right order, the track number could be inferred from the position in that list.
This would require changes to:

yt_listitem_to_playlist: where the list item includes tracks, add track numbers
ytm_item_to_video: to add the track number to the video item
extend_fields: to make sure track numbers are picked up when mopidy-youtube objects are created
_set_api_data: to set the track number field of a mopidy-youtube video where track number is known
a tracknumber() method in Video objects, with a fallback to None.
convert_video_to_track: to add the track number to the mopidy Track when it is created from the mopidy-youtube Video object, if a track number is available

I've just noticed that when I use the 'file file yt:playlist:....' addition you put in the other day, I DO get Track Numbers.
OK MPD 0.19.0
find file "youtube:playlist:MPREb_J8oBiEZMFNu"
file: youtube:video:eAOvikzmuGQ
Time: 210
Artist: Ed Sheeran
Album: Celestial
Title: Celestial
Track: 1
X-AlbumUri: yt:playlist:MPREb_J8oBiEZMFNu
OK

But I don't get them in search results. That track had already been found by a search for 'any "Ed Sheeran"' and it came back with no 'Track' field.

I think it was adding a track number for all playlists, regardless of whether it was an album or not. I've made some changes; would be interested to hear if it is working for you.
I think a track number will only be returned if the track is known to be from a known album.  Anyway, play around, let me know.

Sorry, not had a chance to try it for a few days.
In the testing I've done so far I find I am getting track numbers when I add random Uris to the play qeue.
But, they're often missing from search (which were before) but now when I do the lookup using 'find file' for those albums it doesn't return track numbers like it used to.


Enum default argument value?

How can I provide the default argument value for an enum?
I tried this: @GraphQLArgument(name = "direction", defaultValue = "ASC") final Direction direction but I get: io.leangen.graphql.metadata.strategy.value.InputParsingException: Value: ASC could not be parsed into an instance of org.springframework.data.domain.Sort$Direction
What am I doing wrong? I tried quoting it but it doesn't help.

For non-string values, the expectation is the value will be valid JSON, as Jackson/Gson is used to deserialize it (by default). In some cases, like yours, it ends up looking a bit excessive as you need to duplicate the quotes.
Good you got it working!
P.S.
Rekao bih da smo iz istog komšiluka 😁

Blizu 😄  Btw, GJ, zakon library 😄


[feat]: Adding i18n feature

Adding internationalization using Next.js i18n sub path routing feature

All articles translated.

All articles translated.


Avoid pr-bot state desync

In https://github.com/apache/beam/pull/17285 the PR bot exhibited 2 pieces of broken behavior:

It requested a second review from me after having already requested one from @riteshghorse
Once I approved, it requested an extra review from me again.

These were caused by 2 underlying issues:

Git failed to correctly checkout the pr-bot-state branch causing the first issue. This led the bot to believe nobody had been assigned, and it tried to assign me, but it wasn't able to persist that state back to the state branch (See this broken run). To fix this, I added an extra git push step to make sure we're actually able to push to the pr-bot-state branch before we try to do any state manipulation.
Because the state got out of sync, the bot didn't realize I had been assigned and thought Ritesh was still assigned. When I approved, it revealed a bug that will cause a new reviewer to get assigned when a non-reviewer approves. To fix this, I added an early return in those cases.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

R: @jrmccluskey

R: @lostluck


Banis seem to speed up after extreme usage

The banis benchmark slowly the first usage after a build, but then speed up.
Perhaps something is being generated to aid querying?
To try: disable index auto creation, and preload the DB with analyse stats, based off a benchmarked DB

Is it just a dev thing?

https://vincit.github.io/objection.js/api/objection/#initialize
?


Updated GDSC-Lead Map

Fixes #
Proposed Changes

I submitted Jamsher Shekh details,
This code is not submitted by me
On Wed, Aug 11, 2021, 9:44 PM Aravind Nair @.***> wrote:

@.**** requested changes on this pull request.
In src/components/json/worldData.js
https://github.com/abhinavsri360/GDSC-Lead-Map/pull/117#discussion_r686979793
:

@@ -1046,7 +1047,22 @@ const data = [

     "linkedin": "https://linkedin.com/in/sanjaydevtech",

     "twitter": "https://twitter.com/SanjayDevTech",

     "portfolio": "https://sanjaydevtech.github.io/"

}

},

⬇ Suggested change

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/abhinavsri360/GDSC-Lead-Map/pull/117#pullrequestreview-727684924,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AONBLYHLEHL3OAVPVTLA2NTT4KOVXANCNFSM5B6F2I7A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

photo added


Add a way to clear Likes

Files:

src/services/utils.js
src/pages/Likes/Likes.jsx

Likes are saved in local storage.

Add a function that clears the local storage and removes all the saved likes.
Add a "clear likes"-button somewhere in src/pages/Likes/Likes.jsx that calls the function

Maybe with a prompt "are you sure?"

I'd like to do it.

Thank you!

I've just opened a pull request for this feature. Would appreciate it if you could review it.

Issue resolved in PR #12


Tutorial

Hi,
just wondering if there are any tutorials for this library.
I wanna use it to filter users as that is allow user access to a page based on his session.
can you show some examples as to how your library works

If I find time, I'll create an "examples" page in the wiki, with a user authentication section. However, if you just need some code to get you started:
$klein = new Klein\Klein();

$klein->with('/admin', function () use ($klein) {

    // this will be run when anyone
    // navigates to a route beginning
    // with "/admin".
    $klein->respond(function () {
        Auth::restrict();
    });

    $klein->respond('/?', function () {
        return 'Admin Home Page!';
    });
});

$klein->dispatch();

Instead of redirecting, it might be better to use $klein->abort(403), and set up a 403 block in the onHttpError handler:
$klein->with('/admin', function () use ($klein) {
    $klein->respond(function ($request, $response) use ($klein) {
        if( !Auth::isAuthorized()) {
            $klein->abort(403); // 403 = Forbidden
        }
    }
}

$klein->onHttpError(function ($code, $router) {
    switch ($code) {
        case 403:
            $router->response->body('You do not have access to this page!');
    }
}

I do wish the abort method were on the $response object instead of the $klein object, though.

Nice spot. :) I keep forgetting about the onHttpError() handler.

In onHttpError(), is there a way to access $app?

$klein->onHttpError(function ($code, $router) {
    $app = $router->app();
});

That works, thanks @nbish11


Issue with k8s.io/docs/tutorials/stateful-application/basic-stateful-set/ possible mistake in partition value expected behaviour explanation

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
In the section Staging an Update the partition value was 3 and web-2 is not modified because:

the ordinal of the Pod is less than the partition specified by the updateStrategy.

I guess the sentence from the Rolling Out a Canary section is somehow wrong:

When you changed the partition, the StatefulSet controller automatically updated the web-2 Pod because the Pod’s ordinal was less than or equal to the partition.

I think it should be "greater than or equal to the partition." because in this case partition value is 2 and web-2 ordinal is 2 (equal) but it doesn't change the other pods (web-0 and web-1).
Proposed Solution:
Change the "less than" by "greater than":

When you changed the partition, the StatefulSet controller automatically updated the web-2 Pod because the Pod’s ordinal was greater than or equal to the partition.

Page to Update:
http://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/

Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T21:12:46Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"0b9efaeb34a2fc51ff8e4d34ad9bc6375459c4a4", GitTreeState:"clean", BuildDate:"2017-11-29T22:43:34Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"linux/amd64"}

I can open a PR to fix it if you confirm me that I'm not mistaken.
Regards

This is fixed by #7123


Feature Request: Callback function to get progress of index process

Looking for a callback function in embeddings.index to get the progress of large data sets.
So far I did it by adding a callback function in embeddings.index and passing the function to eight functions in:
embeddings/base.py,
embeddings/index/transform.py,
vectors/base.py,
vectors/hugghingface.py,
models/pooling/base.py
I know, it looks ugly.
I already saw the solution with tqdm but I want to get the progress and send it to the front so I did it my way.
I think there must be a better way, and probably a persistent one to updates, but for now it works. I'm sorry for my lack of experience in python, the library and my poor English.
Thanks

Sorry, didn't see that, i'll take a look. Thanks

Feel free to re-open if there are further questions.


MGMT-13467: CIM: Create InfraEnv secret as a first resource

included in https://github.com/stolostron/console/pull/2778


Instead of passing std::string to handler objects, pass mqtt::string_view

    using publish_handler = std::function<bool(std::uint8_t fixed_header,
                                               mqtt::optional<packet_id_t> packet_id,
                                               std::string topic_name,
                                               std::string contents)>;

For handlers that really need to modify the passed in string object, they can allocate a new string from the passed in string_view.

I think that it is good modification.
Maybe topic_name and contents are created as mqtt::string_view at  https://github.com/redboltz/mqtt_cpp/blob/master/include/mqtt/endpoint.hpp#L5099


adding some missing package infos. #task

Pull Request Test Coverage Report for Build 4027

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 49.375%

Totals

Change from base Build 4026:
0.0%

Covered Lines:
11887

Relevant Lines:
24075

💛  - Coveralls


cutoff prompt when its too long for clip encode

When the input prompt is too long it results in two clip_encoded token vectors of len 77 instead of just one. This causes the final_conditioning = torch.cat(cond_out, dim=0) line to crash when other prompts only have one [77,768] token vector.
This pr simply detects when this happens and drops the second encoded part of the input prompt (this effectively ignores the end of the prompt that's too long).
This fixes this bug

after many tests without animatediff, I can assure you this isn't an issue with my nodes but is actually a problem with the AnimateDiff Evolved repo. To change the repo for another's issue isn't the best way forward. This would be better as a bool option for the batch schedulers only. You can test for yourself without animate diff and see that it isn't a problem with this repo.

the actual fix has been added in pr #64 . Thanks for your work, it helped during the size issue errors for a time.


[SUGGESTION] unInstall XanMod Kernel

Is your feature request related to a problem? Please describe.
for any reason a user might want to remove the installation. For example I tested on Debian 11 and 12. Later for any apt update -y it hangs
Describe the solution you'd like
Adding a remove/uninstall option to the bash script
Describe alternatives you've considered
otherwise a user has to do it manually
Additional context
A software without a Ctrl-Z option is incomplete
Regards

Thanks.
Yes, you are correct but the main challenge is that after the kernel is changed to Xanmod, the old kernel might be removed with some commands (like apt autoremove and such). A rollback feature could be implemented but it can cause some security and/or compatibility risks (As it would be necessary to download a default kernel from somewhere and install it on the server then remove the Xanmod kernel).
But the problem you are experiencing is being connected with DNS or a bad link from your server to Xanmod repositories.
However, I'm implementing a way to roll back to a default Linux kernel.

actually testing on both Ubuntu and Debian , I did not face issues on Ubuntu but had issue with Debian 11.
No it was tested outside Iran.
Tested two times on two Debian 11, both had  the same issue


Updated and fixed model docs and examples.

Addresses Issue #614

Added GPU memory requirements to docs.
Fixed some requirements issues for some of the training and
tuning scripts.

@dagardner-nv Can you review the doc changes in this PR?

/merge


Apparently the keyboard chip has events now

Can someone investigate how it works and update the page?

https://docs.retrogadgets.game/api/modules/KeyboardChip.html

To use any event of a module (if it has any), you first need to make the CPU listens for any event coming from the module you want to use.
For that, you need to go to the EventChannels page in the properties menu in the multitool when selecting the CPU. There, you select the index in the array you want to set the listener for the module, and then select the module itself.
EventChannels > 1 > KeyboardChip0 (for example)
Then, in the CPU code itself (the one that has the listener set, obviously), you HAVE TO DEFINE A FUNCTION that will be called when an EventChannels receives a *MESSAGE.
How's that?
Every event needs to be settled in a eventChannel function, that will be automatically called when the module linked sends a message.
In this case, when you have a module settled in index 1 in the EventChannels of your CPU, you define a function like this:
function eventChannel1(sender, response) end
This HAVE to be the head of the function for make it work.
Notice the use of lowercase at the first character, and the 1 that represents the index of the EventChannels that we're refering to.
When a message is received by an EventChannel, it sends two arguments to the function:
sender: the module class of the object that sent rhe event
response: the object that has the body of the event itself, with the properties and such...
So, if you define an eventChannel1 that listens for a KeyboardChip module, the only event that could be received is an KeyboardChipEvent.
The response body will have the properties shown on the doc, like:
— response.Buttondown: boolean
— response.InputName: InputName (a string with the name of the key being pressed)
— etc...
Hope this helps!!
Srry if you don't understand some parts, I'm bad explaining, as I don't speak english natively

Yes, we already have an explanation of events (under CPU), I meant this specific case of the keyboard chip event.

We have new info! You do not need to have a key bound, and key presses do not backsend events. We just need someone to write these.


stand up morning session 2

deployment
This project was deployed.
check it here  ➡
Thnaks for any suggestion

Hey Jorge, great job! It looks like you put an enormous effort into this project...If I may say something about you could try other kinds of animations to give to your web more interaction and get your end user emotions...So you can try this animation
animation-duration: 3s;
animation-name: slidein;
}
@keyframes slidein {
from {
margin-left: 100%;
width: 300%;
}
to {
margin-left: 0%;
width: 100%;
}
}
and give movement to the text in your intro...good luck with your everything!

Thank you @baezluise and @JuanDBta for your contributions to this project.
I'll take it into account.

thanks for your suggestions. All Done!


Hmc, Nuts gaussian chain tests are flaky and slow

These tests are flaky and slow and are impeding development, e.g.
FAILED tests/infer/mcmc/test_nuts.py::test_nuts_conjugate_gaussian[dim=5_chain-len=9_num_obs=1]
FAILED tests/infer/mcmc/test_hmc.py::test_hmc_conjugate_gaussian[dim=10_chain-len=4_num_obs=1]

Can we replace some of these with less flaky tests?

Is the issue that these tests are giving non-deterministic results (we had a few instances of this, but that has been resolved), or that they are extremely sensitive to small changes in the algorithm? I think at this point there is very little signal that we get from these conjugate gaussian tests, so I would be inclined to skip them altogether if I am unable to make them faster. They are still useful for profiling and checking if inference has improved due to a better chosen mass matrix etc., so we can keep them around.

@neerajprad FYI, pyro NUTS is 20x slower than pymc3 NUTS. This is unacceptable :(

I updated pytorch to lastest version and turned on jit_compile. Things are faster. Now, pyro is 4x slower than pymc.
You can get profiles file here: https://1drv.ms/f/s!AiPrcBQmpf6EgSW6A-7-QD77Y5Ax

@fehiepsi can you clarify what models you're using and what you mean by "compiling in pymc" and "without compiling, pyro is a little bit faster than pymc"?  Are you saying your model in Pyro is slower with the JIT than without in the latest version of PyTorch?

@eb8680 FYI, here are scripts to replicate results in pyro (1.0 branch) and pymc3:
# Source: Statistical Rethinking with Python and PyMC3
import numpy as np
import pandas as pd
import theano
# uncomment the following line to disable jit
#theano.config.mode = 'FAST_COMPILE'
import pymc3 as pm

np.random.seed(0)
rugged_df = (pd.read_csv('rugged.csv', sep=';')
             .assign(log_gdp=lambda df: np.log(df.rgdppc_2000))
             .dropna(subset=['log_gdp']))

with pm.Model() as model:
    a = pm.Normal('a', 0., 100.)
    bR = pm.Normal('bR', 0., 10.)
    bA = pm.Normal('bA', 0., 10.)
    bAR = pm.Normal('bAR', 0., 10.)
    mu = (a + bR * rugged_df.rugged + bA * rugged_df.cont_africa
          + bAR * rugged_df.rugged * rugged_df.cont_africa)
    sigma = pm.HalfCauchy('sigma', 2.)
    log_gdp = pm.Normal('log_gdp', mu, sigma, observed=rugged_df.log_gdp)

with model:
    trace_8_1 = pm.sample(1000, tune=1000, chains=1)

import math
import pandas as pd
import torch

import pyro
import pyro.distributions as dist
from pyro.infer import EmpiricalMarginal
from pyro.infer.mcmc import MCMC, NUTS

pyro.set_rng_seed(0)

rugged = pd.read_csv("rugged.csv", sep=";")
d = rugged
d["log_gdp"] = d["rgdppc_2000"].apply(math.log)
dd = d[d["rgdppc_2000"].notnull()]
dd_rugged = torch.tensor(dd["rugged"].values, dtype=torch.float)
dd_cont_africa = torch.tensor(dd["cont_africa"].values, dtype=torch.float)
dd_log_gdp = torch.tensor(dd["log_gdp"].values, dtype=torch.float)

def m8_1stan_model(rugged, cont_africa, log_gdp):
    a = pyro.sample("a", dist.Normal(0, 100))
    bR = pyro.sample("bR", dist.Normal(0, 10))
    bA = pyro.sample("bA", dist.Normal(0, 10))
    bAR = pyro.sample("bAR", dist.Normal(0, 10))
    mu = a + bR * rugged + bA * cont_africa + bAR * rugged * cont_africa
    sigma = pyro.sample("sigma", dist.HalfCauchy(2))
    with pyro.plate("plate"):
        pyro.sample("log_gdp", dist.Normal(mu, sigma), obs=log_gdp)

num_samples = 1000
warmup_steps = 1000
num_chains = 1
nuts_kernel = NUTS(m8_1stan_model, jit_compile=False)  # change to True to use jit
posterior = MCMC(nuts_kernel,
                 num_samples=num_samples,
                 warmup_steps=warmup_steps,
                 num_chains=num_chains)
posterior.run(rugged=dd_rugged, cont_africa=dd_cont_africa, log_gdp=dd_log_gdp)

Link to the data
PyMC3 uses Theano to compile the code under the hood. With theano.config.mode = 'FAST_COMPILE', Theano will use Python implementation instead (with some small optimization). What I mean is Pyro (with jit_compile=False) is a little bit faster than PyMC3 (with FAST_COMPILE).

Great work, and thanks for starting to dig into the performance aspects of NUTS! I will take a closer look at your benchmarks.

Without compiling, pyro is a little bit faster than pymc.

I would have thought the same since there are few opportunities for making further optimizations in our python code, and for the models that I have seen, most of the time taken is in PyTorch. But it is great to know that we are actually a bit faster than pymc in this regard. :)

With compiling, pyro is 4x slower than pymc (compiling in pymc gives it 8x improvement). This suggests that theano does better job in compiling (small models) than pytorch.
potential_jit is 4x faster than potential but grad is not improved much.

This might improve with any improvements in trace.jit, but I doubt if the performance gap will close by much just with the JIT tracer. 4X isn't that bad though. :)

Can we compile backward pass too?

torch.jit.script will be able to compile the backward pass too but I am not sure if that's slated for the 1.0 release. I would expect that models compiled with torch.jit.script should be faster, and we might be able to close the gap if we are able to JIT the integrator step itself (we may need to use a combination of jit.trace and jit.script though).

we might be able to close the gap if we are able to JIT the integrator step itself (we may need to use a combination of jit.trace and jit.script though).
yes, totally agree! (and some other methods such as kinetic_energy and is_turning too). I seems that jit.script allows OrderDict, so we might use OrderDict instead of dict (I am not so sure, I get problems while playing around with jit.script).

@fritzo @neerajprad Are the slowness and flakiness fixed? If not, I’ll take a look. We might just change random seed, or num steps, step size though.

This is closed because Neeraj already marked skip for these tests.


Stop using deprecated RC API

This quickstart was failing to build with the latest firebase-ios-sdk master

cc: @christibbs @karenyz


Add Code Quality and security doc

Signed-off-by: John Mertic<EMAIL_ADDRESS>

Thank you @cmgrote  - incorporated your feedback now.

@planetf1 Thanks for point that out - would you recommend we roll this into those docs, those docs into this, or something else?

I actually like the filename you chose, and the conciseness of the information which is especially suitable for anyone coming along to find out high level info about the security of egeria.
In most cases the other docs get into more detail - but even then there are overlaps between those two.
One other thing that's missing is pointing to this file from the overall README in developer resources
If it makes it easier I'd be happy we merge this, and then I can try and consolidate the three files with a view to keeping the principle of

the high level security overview/reports
more detailed process info for developers

Note though that currently you're missing the required header to pass our license check. Can you add
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
<!-- Copyright Contributors to the ODPi Egeria project. -->

to the top of the file?

Thanks @planetf1 - added the headers now :-)
I'll leave it to you whether to merge now or fix the other docs at the same time...

Thanks @planetf1 - added the headers now :-)
I'll leave it to you whether to merge now or fix the other docs at the same time...

Will merge and open up new issue


Add update for Github releases

Fritter on F-Droid is updating with a delay (there is still 2.2.0).
NewPipe has its internal system for updating Github releases.

It should be great add this to Fritter if it requires urgent update (e.g. when Twitter changes its work dramatically).

I'm accelerating this, as Twitter just made a breaking change that requires an app update (#77). It happened!


inter-gpu communication

Hi, I have a question about the multi-gpu training.
The original SimCLR paper uses a large batch size and the good performance is greatly contributed to the large batch size. This is critical when computing the loss. However, in your implementation, it seems you didn't consider other sub-batches in the other gpus. Specifically, if you set the batch size as 1024 and use 8 gpus, then in each gpu, the sub-batch size is 128. According to your implementation, it seems that when computing the nt_xent loss, the number of samples used is just 128, not 1024. I think the correct implementation should use all_gather operations to get all the 1024 samples from all the gpus.
I am not sure about the actual implementation. If I understand something wrong, please correct me.

Hi Shijian Xu,
Thank you for opening the issue. It is correct that the sim. matrix constructed in the loss function is done on the sub-batches on each GPU, and the loss is thus computed for every sub-batch and then aggregated in the way DataParallel/DistributedDataParallel aggregates losses. After taking a good look at the original implementation, this is indeed different: the original author first aggregates the data from every TPU ("replicas"), before calculating the loss.
Feel free to submit a PR if you have the right solution for this.

Hi Shijian,
I've looked into the matter more closely. For DistributedDataParallel training, you are right about gathering tensors from all GPU's. I've changed it in this commit so samples from all GPU's are gathered in ./modules/NT_Xent.py
With DataParallel training, everything seems to go fine, since only the model is wrapped in the parallel module. For convenience, I added a flag in the config for DP training.


Feature request - Check that QueueRunner enqueue ops use the correct queue

The documentation for tf.train.QueueRunner.__init__ states:
"The enqueue ops do not have to all be the same op, but it is expected that they all enqueue tensors in queue."
Would it be possible for tensorflow to check that the given ops do in fact enqueue tensors in the correct queue, and throw an error if they do not?  I encountered a problem recently where I mistakenly passed the wrong queue to my QueueRunner, and as a result, the clean shutdown behavior described here wasn't working as expected.  Changing to the correct queue solved the problem.  This was definitely user error on my part, but it was a bit tricky to track this problem down, and it seems like the kind of thing that would be possible to check automatically.

There was no check preventing people from doing that, so there may be people using single queue runner to enqueue to multiple queues successfully out there. So starting to enforce that now will break those people

OK, that sounds like a reasonable argument for not doing this. :)


torch.compile(sync_float8_amax_and_scale_history) not working with triton latest main

[rank0]:   File "/opt/venv/lib/python3.10/site-packages/lightning_fabric/wrappers.py", line 411, in _capture
[rank0]:     return compile_fn(*args, **kwargs)
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/__init__.py", line 2448, in compile
[rank0]:     return torch._dynamo.optimize(
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 716, in optimize
[rank0]:     return _optimize(rebuild_ctx, *args, **kwargs)
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 790, in _optimize
[rank0]:     compiler_config=backend.get_compiler_config()
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/__init__.py", line 2238, in get_compiler_config
[rank0]:     from torch._inductor.compile_fx import get_patched_config_dict
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 49, in <module>
[rank0]:     from torch._inductor.debug import save_args_for_compile_fx_inner
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_inductor/debug.py", line 26, in <module>
[rank0]:     from . import config, ir  # noqa: F811, this is needed
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_inductor/ir.py", line 77, in <module>
[rank0]:     from .runtime.hints import ReductionHint
[rank0]:   File "/opt/venv/lib/python3.10/site-packages/torch/_inductor/runtime/hints.py", line 36, in <module>
[rank0]:     attr_desc_fields = {f.name for f in fields(AttrsDescriptor)}
[rank0]:   File "/usr/lib/python3.10/dataclasses.py", line 1198, in fields
[rank0]:     raise TypeError('must be called with a dataclass type or instance') from None
[rank0]: TypeError: must be called with a dataclass type or instance

Do we know which torch version should we use to make compile work?


why does the 'start' event be triggered only once?

Version information

fluent-ffmpeg version:newest version
ffmpeg version:newest version
OS:windows10

I have two camera and want to transport the rtsp to rtmp stream and put the decoded video to the electron client.But why does the start event only be triggered once?Is there any other way to let the start event be triggered every time when i initialize a ffmpeg instance?
    ffmpeg('rtsp://admin:123456abc@<IP_ADDRESS>:554/11')
            .outputOptions([
                "-vcodec",
                "copy",
                "-r",
                "25",
                "-an"
            ])
            .inputFPS(50)
            .noAudio()
            .aspect("4:3")
            .format("flv")
            .save('rtmp://localhost:9090/live/123')
            .on("start", function (e) {
                _this.notify("[ffmpeg start]" + e);
            })

Are you using the same  .save('rtmp://localhost:9090/live/123') on both cameras?

change .save() to .output() will be ok.

You should register event handlers before calling save(), because the processing methods check for registered event handlers at the beginning. That would probably explain unpredictible results when registering events after having called save().
#674


Docker Private Registry Configuration Question related to redis

Does Docker Private Registry support Redis cluster or Redis sentinel? Reference: https://docs.docker.com/registry/configuration/#overriding-the-entire-configuration-file.
After reviewing this link, I understand that redis can be used as a blobdescriptor. Further by defining the redis server & port details under redis: section, one can configure registry to use redis for layer metadata caching. But what is not clearly mentioned is the support for redis advanced features like sentinel or cluster. As per redis documentation, client libraries should be able to understand the redirection and connect to the redis server in case of failovers (from sentinel when master fails and a slave is promoted as new master) or when data hash is found in a different redis node ( than the currently connected node in the redis pool). Is registry is coded to understand these redirections and work? I am looking forward to understand how registry behaves in these mentioned scenarios to architect a high available setup for Docker Private Registry. Thanks in advance for any help or suggestions.

Hello All, Can some one please review this issue and share your views

I'm not super familiar with sentinel or overall Redis HA.
We do not do anything specific in the registry about them though, and rely on this client library: https://github.com/garyburd/redigo/
There is limited interest at this point into redis caching - so I would suggest you do a couple of tests with cluster and sentinel, and share the results here with the community.

Looking at redigo, I can see:
Related Projects
[...]
FZambia/sentinel - Redis Sentinel support for Redigo
mna/redisc - Redis Cluster client built on top of Redigo

from where I'm infering that "if redigo had that support, there wouldn't be those "extensions" to the library. In fact, even "FZambia/sentinel" suggests: «Alternative solution - You can alternatively configure Haproxy» (which is basically what everyone seems to be doing for backwards compatibility with non-sentinel clients.
So my understanding is that docker registry does not support a sentinel-managed Redis setup (nor a Redis Cluster setup).

No, the current implementation does not support Redis [sentinel] cluster.
We're open to accepting PRs if you fancy adding support for them.
Given this question has been answered I'll close it now.


Question regarding task naming / service discovery

I was trying to find an intelligent way to receive ip:port information for tasks launched by dcos-commons. I use the following service definition currently:

https://github.com/tobilg/dcos-commons/blob/mongodb-replicaset/frameworks/mongodb-replicaset/src/main/dist/svc.yml

While it's possible to derive the ip:port info for task by matching the (partial) task name, e.g. _server, from the output of the Master's /master/state endpoint, which is not ideal IMO.
It seems to be impossible to get this info from

The REST API

There is a GET /tasks/connection/{taskName}, which can be used for specific taskNames. When queried like /v1/tasks/connection/mongodb-2-server in my case, I see {"dns":"mongodb-2-server.inf-mongodb-replicaset.mesos","ports":""} as result. The port info is empty.
Also, there is no endpoint to query the overall connection data (per pod/task). This makes it impossible to use the REST API at the current state to discover the service/task endpoints.

Mesos DNS

As dcos-commons seems to apply a {podName}-{instanceId}-{taskName} naming pattern, it isn't possible to use Mesos DNS SRV records to get all endpoints of a specific pod's tasks, but only one-by-one:
$ dig _mongodb-0-server._tcp.mongodb-replicaset.mesos SRV

; <<>> DiG 9.10.2-P4 <<>> _mongodb-0-server._tcp.mongodb-replicaset.mesos SRV
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8591
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;_mongodb-0-server._tcp.mongodb-replicaset.mesos. IN	SRV

;; ANSWER SECTION:
_mongodb-0-server._tcp.mongodb-replicaset.mesos. 60	IN SRV 0 0 27017 mongodb-0-server-69857-s0.mongodb-replicaset.slave.mesos.

;; ADDITIONAL SECTION:
mongodb-0-server-69857-s0.mongodb-replicaset.slave.mesos. 60 IN A <IP_ADDRESS>

;; Query time: 1 msec
;; SERVER: <IP_ADDRESS>#53(<IP_ADDRESS>)
;; WHEN: Mon Jan 09 14:34:06 UTC 2017
;; MSG SIZE  rcvd: 160

Ideally, IMO, once could do the following:
$ dig _server.mongodb._tcp.mongodb-replicaset.mesos SRV

so the naming convention would be _{taskName}.{podName}._tcp.{serviceName}.mesos
Maybe I did miss the idea on the task/pod service discovery, I'm not sure. It'd be interesting to learn the "official recommended way" to deal with this.
Thanks in advance! cc @gabrielhartmann @mrbrowning

Maybe take a look at EndpointsResource which is meant to replace the /tasks/ retrieval: https://github.com/mesosphere/dcos-commons/blob/master/sdk/scheduler/src/main/java/com/mesosphere/sdk/api/EndpointsResource.java
It depends on DiscoveryInfo being populated in the tasks.

Thanks @nickp for the feedback. As I'm using the YAML definitions solely, I'm not using any Java code. So, the YAML doesn't seem to populate the ports?

Ah ok. Yes I think that could be it. Once that's fixed then the /endpoints/ endpoint should start working as well.

According to https://mesosphere.github.io/dcos-commons/swagger-api/index.html there doesn't seem to be a /endpoints endpoint... Should there already be one?
I guess the Mesos DNS discoverability of all endpoints for all specific task instances of a pod wouldn't work if I'm not mistaken

Whoops, I should've updated that when adding the endpoint. That swagger listing is automatically generated from this manually written source (have been meaning to figure out how to generate this as well...): https://github.com/mesosphere/dcos-commons/blob/master/docs/swagger-api/swagger-spec.yaml
Filed issue for myself: #435

I tested this today: GET http://<IP_ADDRESS>/service/mongodb-replicaset/v1/endpoints, which yields to
{
	"mongodb-replicaset": {
		"direct": ["mongodb-2-server.mongodb-replicaset.mesos:27017", "mongodb-1-server.mongodb-replicaset.mesos:27017", "mongodb-0-server.mongodb-replicaset.mesos:27017"],
		"vip": "mongodb-replicaset.mongodb-replicaset.l4lb.thisdcos.directory:27017"
	}
}

So, it seems the final VIP is mongodb-replicaset.mongodb-replicaset.l4lb.thisdcos.directory:27017. That's kind of unexpected for me though... Would it mean if I use an empty VIP prefix in the YAML, or omit it completely, that the VIP would be changed to mongodb-replicaset.l4lb.thisdcos.directory:27017?

The REST API endpoint for http://<IP_ADDRESS>/service/mongodb-replicaset/v1/tasks/connection/mongodb-2-server still doesn't return a port number though:
{
	"dns": "mongodb-2-server.mongodb-replicaset.mesos",
	"ports": ""
}

The format of a vip is $VIP_PREFIX.$SERVICE_NAME.l4lb.thisdcos.directory -- in this case, the first two happen to be set to the same value, but they don't have to be. It's not possible to omit the prefix, since it's the only parameter that DC/OS lets you set. DC/OS adds everything after the VIP prefix. I'm not sure what you mean about discoverability via DNS -- this should be taken care of by the /connection, and we're not able to specify arbitrarily nested DNS domains -- it's task.service.mesos only.

@mrbrowning Thanks for the reply! I think it would be great if the VIP format could be "officially" documented somewhere easy to find.
Concerning /v1/connection, this seems to be replace now with /v1/endpoints? At least I can't seem to access to path, results in a 404 (see also #443).
After reading and trying to understand the sources of the EndpointsResource class, I figured that I could do a query like /v1/endpoints/mongodb?format=nativewhich will return "real" endpoint information:
{
	"direct": ["<IP_ADDRESS>:27017", "<IP_ADDRESS>:27017", "<IP_ADDRESS>:27017"],
	"vip": "mongodb.mongodb-replicaset.l4lb.thisdcos.directory:27017"
}

I guess that's what I needed, but this is also not really documented somewhere.
Still, another issue is /v1/tasks/connection/{taskName} not returning port numbers IMO.

Is this resolved and can we close the issue?

Haven't checked in a while if the port is still empty. Is there a way to create a test for that eventually?


Compatibly with latest version of Alamofire (5.1)

Hello.
I'm trying to update my pods libraries but I'm getting an error from cocoa pods indicating that 'PromiseKit/Alamofire' requires an older version of Alamofire to work?, is that correct? or maybe I'm doing something wrong in my Podfile, I'll leave an example of how it looks like.
pod 'Alamofire', '~> 5.1'
 pod 'AlamofireImage', '~> 4.1'
 pod 'Moya', '~> 14'
 pod 'PromiseKit', '~> 6'
 #comment pod 'PromiseKit/Alamofire'
 pod 'RxSwift', '~> 5.1'
 pod 'RxCocoa', '~> 5.1'
 pod 'Moya/RxSwift', '~> 14'

if I remove the " #comment" tag I receive this error:
PromiseKit/Alamofire (~> 6) was resolved to 6.13.1, which depends on
      Alamofire (~> 4.0).

I want to double check if it is possible to have PromiseKit/Alamofire with Alamofire 5 and PromiseKit 6, and if that's the case, what could be wrong in the example above?.
Sorry for my English, it is not my native language.
I appreciate your help, thanks.

Hello @MiltonPacheco89
I think this issue is not related to mxcl/PromiseKit and you already discussed it in PromiseKit/PMKAlamofire. You can close the issue.


Restore custom param types over messaging system

@piotrmaslanka , would you mind trying this out with your work with jaeger. You can now add do
# add span
messageBuilder.withCustomParam("JAEGER", myJaegerSpan);

and
# get span
myJaegerSpan = message.customParams.get("JAEGER");


Can you push to packagist?

Thank you for all your work! Can you push/setup sync to get the latest version to packagist?

Just following up here, thank you!


feat(home):  adjust homepage with new design

Closes #21

Adjust font family to use Open Sans
Use existing Button component

cc @bandungpy/frontend

for < md screen size, something feels off to me. what do you think if we make the align to be center for < md screen size?

Yes need to adjust a bit for the font sizing consistency
Large screen:

Medium screen:

Will do, thanks @ridhof @mhaidarhanif

Updated, I've updated the font to Quicksand and adjusted the font consistency @mhaidarhanif @ridhof
However @ridhof , when I tried to align the hero content to the center, it seems to be more off than it was before so I keep it as is. Any suggestion?

Alright it's better
Let's keep it as is for now and merge this PR


Hide emulator settings

Hi I was wanting to make a retropie console for my little cousins and want to disable the emulator settings when the game boots up by pressing buttons on the controller while booting as I don't want them to accidentally go in that menu and mess up anything.

Could this be merged with https://github.com/Aloshi/EmulationStation/issues/518 ?

Would really like to find out how to do this


Build run error

I ran a build with success.
Maybe 5 minutes later I run another build.
I had this error

I restarted my VM, re run the build, and no more error ...

Thanks @BrunoJunior.

@BrunoJunior I am closing this issue for now because this problem appeared once. Feel free for reopen this issue if problem will repeat again.


Long pooling using @SqsListener

Type: Feature
Is your feature request related to a problem? Please describe.
Enable long pooling using @SqsListener
Describe the solution you'd like
Long pooling for SQS is crucial feature. I was wondering if it's possible to make it configurable from @SqsListener annotation?

Long polling is enabled by default as the waitTimeout is set to 20 seconds.
Timeout can be controller through a property: cloud.aws.sqs.listener.wait-timeout.
We do want to bring more flexibility to configuration per-listener, not through annotation but rather in a configuration file. Look at this POC how this may look like in Spring Cloud AWS 3.0: https://github.com/maciejwalkowiak/reactor-sqs-poc/blob/master/src/main/resources/application.yml

Thanks for answer


issue with attribute deprecation in Numpy < 2.0?

I was attempting to plot a dataset using Altair and ran into the issue of having > 5000 rows. The documentation associated with the error suggested installing vegafusion via conda install -c conda-forge "vegafusion-python-embed>=1.5.0" "vegafusion>=1.5.0", so I did.
This resulted in a downgrade of my Numpy installation as follows:
The following packages will be DOWNGRADED:

numpy                               2.0.0-py312h22e1c76_0 --> 1.26.4-py312heda63a1_0

The next time I executed the relevant cell, I got the error that I was missing the vl-convert dependency, so I installed it via conda install -c conda-forge "vl-convert-python>=1.3.0"
The cell now errors out, but seems to be pointing the finger at the Numpy downgrade. The full trace is as follows:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:977, in MimeBundleFormatter.__call__(self, obj, include, exclude)
    [974](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:974)     method = get_real_method(obj, self.print_method)
    [976](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:976)     if method is not None:
--> [977](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:977)         return method(include=include, exclude=exclude)
    [978](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:978)     return None
    [979](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/IPython/core/formatters.py:979) else:

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2576, in TopLevelMixin._repr_mimebundle_(self, include, exclude)
   [2574](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2574)     return {}
   [2575](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2575) else:
-> [2576](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/altair/vegalite/v5/api.py:2576)     return renderers.get()(dct)

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:304, in curry.__call__(self, *args, **kwargs)
    [302](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:302) def __call__(self, *args, **kwargs):
    [303](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:303)     try:
--> [304](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:304)         return self._partial(*args, **kwargs)
    [305](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:305)     except TypeError as exc:
    [306](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/toolz/functoolz.py:306)         if self._should_curry(args, kwargs, exc):

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:16, in vegafusion_mime_renderer(spec, mimetype, row_limit, embed_options)
     [15](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:15) def vegafusion_mime_renderer(spec, mimetype="html", row_limit=None, embed_options=None):
---> [16](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:16)     return spec_to_mime_bundle(
     [17](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:17)         spec,
     [18](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:18)         mimetype=mimetype,
     [19](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:19)         row_limit=row_limit,
     [20](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:20)         embed_options=embed_options
     [21](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:21)     )

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:38, in spec_to_mime_bundle(spec, mimetype, row_limit, embed_options, html_template, full_html, scale)
     [35](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:35) vega_spec = vegalite_compilers.get()(spec)
     [37](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:37) inline_datasets = transformer.get_inline_datasets_for_spec(vega_spec)
---> [38](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:38) tx_vega_spec, warnings = runtime.pre_transform_spec(
     [39](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:39)     vega_spec,
     [40](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:40)     local_tz.get_local_tz(),
     [41](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:41)     row_limit=row_limit,
     [42](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:42)     inline_datasets=inline_datasets
     [43](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:43) )
     [45](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:45) for warning in warnings:
     [46](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/renderer.py:46)     if warning.get("type", "") == "RowLimitExceeded":

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:364, in VegaFusionRuntime.pre_transform_spec(self, spec, local_tz, default_input_tz, row_limit, preserve_interactivity, inline_datasets, keep_signals, keep_datasets, data_encoding_threshold, data_encoding_format)
    [362](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:362) else:
    [363](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:363)     local_tz = local_tz or get_local_tz()
--> [364](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:364)     imported_inline_dataset = self._import_or_register_inline_datasets(inline_datasets)
    [366](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:366)     # Parse input keep signals and datasets
    [367](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:367)     keep_signals = parse_variables(keep_signals)

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:217, in VegaFusionRuntime._import_or_register_inline_datasets(self, inline_datasets)
    [214](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:214)         except ValueError:
    [215](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:215)             pass
--> [217](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:217)     imported_inline_datasets[name] = PandasDatasource(value)
    [218](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:218) elif hasattr(value, "__dataframe__"):
    [219](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:219)     # Let polars convert to pyarrow since it has broader support than the raw dataframe interchange
    [220](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:220)     # protocol, and "This operation is mostly zero copy."
    [221](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/runtime.py:221)     try:

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:37, in PandasDatasource.__init__(self, df, sample_size, batch_size)
     [35](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:35)         field = pa.Schema.from_pandas(pd.DataFrame({col: cat.categories})).field(col)
     [36](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:36)     else:
---> [37](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:37)         candidate_schema = pa.Schema.from_pandas(df.iloc[::sample_stride][[col]])
     [38](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:38)         field = candidate_schema.field(col)
     [39](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:39) except (pa.ArrowTypeError, pa.ArrowInvalid):
     [40](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/vegafusion/datasource/pandas_datasource.py:40)     # If arrow fails to infer the type, fall back to string

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/types.pxi:2620, in pyarrow.lib.Schema.from_pandas()

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:100
     [83](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:83)             return 'decimal'
     [84](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:84)         return 'object'
     [87](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:87) _numpy_logical_type_map = {
     [88](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:88)     np.bool_: 'bool',
     [89](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:89)     np.int8: 'int8',
     [90](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:90)     np.int16: 'int16',
     [91](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:91)     np.int32: 'int32',
     [92](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:92)     np.int64: 'int64',
     [93](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:93)     np.uint8: 'uint8',
     [94](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:94)     np.uint16: 'uint16',
     [95](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:95)     np.uint32: 'uint32',
     [96](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:96)     np.uint64: 'uint64',
     [97](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:97)     np.float32: 'float32',
     [98](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:98)     np.float64: 'float64',
     [99](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:99)     'datetime64[D]': 'date',
--> [100](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:100)     np.unicode_: 'string',
    [101](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:101)     np.bytes_: 'bytes',
    [102](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:102) }
    [105](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:105) def get_logical_type_from_numpy(pandas_collection):
    [106](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/pyarrow/pandas_compat.py:106)     try:

File ~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:397, in __getattr__(attr)
    [395](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:395) if len(w) > 0:
    [396](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:396)     for _wn in w:
--> [397](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:397)         if _wn.category is exceptions.RankWarning:
    [398](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:398)             # Ignore other warnings, they may not be relevant (see gh-25433).
    [399](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:399)             error_message = f"{_wn.category.__name__}: {str(_wn.message)}"
    [400](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:400)             msg = (
    [401](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:401)                 "Polyfit sanity test emitted a warning, most likely due "
    [402](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:402)                 "to using a buggy Accelerate backend."
   (...)
    [405](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:405)                 "\nOtherwise report this to the vendor "
    [406](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/tedals/projects/tangerine-temp/tangy-temp/~/miniconda3/envs/ttemp/lib/python3.12/site-packages/numpy/__init__.py:406)                 "that provided NumPy.\n\n{}\n".format(error_message))

AttributeError: `np.unicode_` was removed in the NumPy 2.0 release. Use `np.str_` instead.

...is there a workaround here, short of trying to dip into the innards of these packages and do some monkey-patching or something?

Hi @TSSlade, thanks for the report. The stack trace is confusing, since the error seems to be coming from numpy 2, but conda claims that numpy was downgrade from 2 to 1. I also don't see any dependency in the VegaFusion or vl-convert packages that would cause a numpy downgrade, so I don't know why conda decided that was necessary.
From the error itself, it looks like the issue is with pyarrow's use of numpy, so the you could try updating your pyarrow package.
If that doesn't help, could you try reproducing the issue from a fresh conda environment? Thanks!

I completely failed to notice the inconsistency between the stack trace's error message and the downgrade warning. You're absolutely right.
In the existing env:
$ conda list | grep pyarrow
pyarrow                   16.1.0          py312h9cebb41_4    conda-forge
pyarrow-core              16.1.0          py312h70856f0_4_cpu    conda-forge

In a new, clean, env (generated via $ conda create -n ttclean jupyterlab altair pandas vegafusion numpy vl-convert-python):
$ conda list | grep pyarrow
pyarrow                   13.0.0          py312h176e3d2_8_cpu    conda-forge

...but it turns out that in either case -- both new env and old -- we're able to get things working properly. The issue wound up being
import vegafusion as vf
vf.enable(row_limit=50000)

# alt.renderers.enable("jupyter")
# alt.JupyterChart.enable_offline()
# alt.data_transformers.enable("vegafusion")

when vegafusion is imported directly and enabled as above, everything executes properly even with the downgraded Numpy. It appears to be some combination of the various commented-out attempts at invoking the vegafusion-powered altair features that was the problem.


can't not resolve 'canvas' in ......./pdf.js

Attach (recommended) or Link to PDF file here:
Configuration:

PDF.js version:3.7
vue version:2.6

Steps to reproduce the problem:

download pdf.min.js
2.'import pdfjs from 'pdf.min.js'' in a specific file of vue project
3.throw a error 'dependency not found ' in vscode terminal ,and browser throw module not found
ps: in other project doesn.t require this dependency

What is the expected behavior? (add screenshot)
normally import

PDF.js version:3.7

Note that that's not a complete version number, did you actually mean 3.7.107!?

vue version:2.6

Sorry, but we unfortunately cannot provide help/support for the myriad of JS frameworks that exists.
Generally, please also note https://github.com/mozilla/pdf.js/blob/master/.github/CONTRIBUTING.md (emphasis mine):

If you are developing a custom solution, first check the examples at https://github.com/mozilla/pdf.js#learning and search existing issues. If this does not help, please prepare a short well-documented example that demonstrates the problem and make it accessible online on your website, JS Bin, GitHub, etc. before opening a new issue or contacting us in the Matrix room -- keep in mind that just code snippets won't help us troubleshoot the problem.


Implement rustfmt CI checks

The code got formatted according to the existing .rustfmt.toml. Additionally, I've added a CI check for code formatting.

@seqre in the future, if you don't mind, please check allow edits by maintainers on your Nosey Parker PRs — sometimes this can speed up the review/merge process instead of me asking for a small change.

I made one followup change to opt out of rustfmt formatting for one function, which ends up much less readable after auto-formatting: 06cc1fa2b33f10dcb22e652f9817fc5d44247b94.
I'm curious to see what pre-commit integration for this project would look like, specifically for the   rustfmt check available here.

See #135 for pre-commit support.


React Native Example start error.

Hello,
When trying to open the react native example in either web, android, or ios, I get hit with an error.

ERROR in ./node_modules/expo/build/launch/withDevTools.web.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
SyntaxError: /Users/giovannicoakley/Documents/demo/connectrpc examples-es main react-native/node_modules/expo/build/launch/withDevTools.web.js: Duplicate __self prop found. You are most likely using the deprecated transform-react-jsx-self Babel plugin. Both __source and __self are automatically set when using the automatic runtime. Please remove transform-react-jsx-source and transform-react-jsx-self from your Babel config.
4 |     function WithDevTools(props) {
5 |         return (<>
->6 |         <AppRootComponent {...props}/>
|         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
7 |
8 |       </>);
9 |     }
at File.buildError [as buildCodeFrameError] (/Users/giovannicoakley/Documents/demo/connectrpc examples-es main react-native/node_modules/@babel/core/src/transformation/file/file.ts:249:12)

This is after downloading the repo, running npm install and then npm start.

@gcoakleyjr This should hopefully be fixed now. Please pull latest and give the react-native commands a try.


Bulk check API

Hello,
I'm evaluating spicedb (very nice product btw !).
I'd like to handle the following use case:

My front-end app calls my rest API to display the list of objects returned by the API.
The last column of the list is an 'Actions' column.
This column contains the list of actions that can be executed by the current user on the given object.

I'd imagine calling SpiceDb API to evaluate a [list of permissions] for a given user and for a [list of objects].
This way, I'd know the buttons I need to display in each row of the actions column.
I could of course do that by calling the checkPermission API in a loop, but it won't scale
Is there something like a bulk checkPermission API ?
Thanks !

Hey there!
Feel free to jump into Discord if you'd like to discuss your particular example more often.
In the past, when we considered bulk checking, we figured that the frame multiplexing in HTTP2 would handle this reasonably well. We're totally open to re-evaluating that, if folks are find otherwise.

@gonzalad would it be sufficient to be able to check for the same permission on different object IDs, or do you need different object types + permissions as well?

Hi @josephschorr , my use case is to check a list of permissions on different object Ids (all object Ids being of the same type)
For instance, my use case could be covered by a request parameter like this (it's just a sample - just to give a better idea):
{
   objectIds: [ id1, id2 ],
   permissions: ["perm1", "perm2"],
   user: "myuser"
}

The previous request would be generated by the front end when it displays a table containing 2 lines (object id1 and object id2), and when the user can potentially perform 2 actions for a given line (the two actions being represented by perm1 and perm2 permission).
Depending on the response from the API, the actions for each line will be enabled or disabled.
Thanks

@gonzalad Okay. Supporting batch checks of multiple objects for the same permission will be doable from an internal perspective after https://github.com/authzed/spicedb/pull/777 is merged; I suppose we could look into a BulkCheck API that allowed for a set of permissions as well, and just dispatched in parallel

#777 is now merged

Is this available as a public API now？Thanks

@pathing #777 was an internal-only change for performance improvements. There is no externally available bulk checking API as of right now

Any plans for this？

@pathing #777 was an internal-only change for performance improvements. There is no externally available bulk checking API as of right now

#777 is now merged
NOTE: This is not available as a public API yet; this is simply an internal change that could facilitate a bulk check API publicly in the future

Any plans for this？

@pathing Tentative plans, yes, but still determining the best way to do so

Questions for discussion:

Maximum size of the batch?
Should the batch check API be streaming in response, such that we return results as they are computed?

On point 2, that sounds sane to me. It'd match the behavior of other bulk-ish endpoints (e.g. ListResources) and support a larger set of check results.

This is now in active development here: https://github.com/authzed/spicedb/pull/1453

@josephschorr thanks for the update 👍 just to verify: this is currently implemented for different objects, not different permissions, right?
I ask bc I think of a meta-authz case where different permissions could become handy when you e.g. want to check "user is able to delete file in folder, but only if they have write rights on the folder" or sth

This is great, when can we see this in a release tag?

Great job!
Is there any sort of benchmark or any idea of the perf improvement brought by bulk-checking? :)

This is great, when can we see this in a release tag?

Went out in v1.25.0

Is there any sort of benchmark or any idea of the perf improvement brought by bulk-checking? :)

The major perf improvement today will be if using the bulk check to check the same permission, on the same subject, for different resources: in that case, the system will auto-batch those checks into a single check of up to 100 checks at once, and run them as a single dispatch tree. This is already done internally for subproblems, but is now accessible via bulk check


Redux Video Tutorial

[ ] Traversy Media Video Tutorial on Redux visit this link

This video tutorial specifically uses Redux in the same destructured way that we are seeing inside of the current code base. Will be nice to see it built up from scratch.

FE has gone over the redux videos and have a better idea of how it will work inside of this project. Closing this ticket.


new theme for personal blogs - ghazal

New portfolio theme ghazal
https://github.com/darshanbaral/ghazal
I made sure that...

[x] the repository contains a good README.md describing my theme
[x] an open source license has been added to LICENSE.md
[x] all metadata have been added to theme.toml
[x] screenshots have been added in the images/ folder with the required dimensions
[x] in case I'm using a customized demo via the exampleSite folder that

[x] https://example.com is set as base url in exampleSite/config.{toml, yaml, json} to avoid the abuse of unused domains

[ ] I tested that my theme's demo works with the content directory of gohugoio/HugoBasicExample
[ ] I tested my theme against the gohugoio/HugoBasicExample
[x] I've checked the developer tools' console in my browser for error messages

Hello @darshanbaral
While reviewing your theme's demo I noticed that on the Posts list page the HTML markup is rendered as a string in the post summaries.
The culprit is line 20 of _default/list.html.
You need to sanitize the post content by using the plaintify function to remove the HTML markup.
Furthermore you need to provide a valid Hugo version in line 11 of theme.toml see here for details: https://github.com/gohugoio/hugoThemes#themetoml
Once you do the above please let me know

@onedrawingperday, I fixed the two issues..

@darshanbaral
I added the theme to the repo in https://github.com/gohugoio/hugoThemes/commit/51911c3fb2fd5fe6fd3a8021c5ca92dfb0fb1c94

@digitalcraftsman
This is a simple and responsive blog theme with an intro.

@digitalcraftsman/@onedrawingperday, could you please approve the content for this theme. The demo doesn't show the archive and about pages.


Search_Dates Error Report

The following code crashes:
part = search_dates("Edmonton, AB T5J 2Z2 Sent: November-15-18 4:44 PM")
It gives me the following error:
File "kv_extraction.py", line 51, in extract
part = search_dates("Edmonton, AB T5J 2Z2 Sent: November-15-18 4:44 PM")
File "/usr/local/lib/python3.6/dist-packages/dateparser/search/init.py", line 50, in search_dates
text=text, languages=languages, settings=settings
File "/usr/local/lib/python3.6/dist-packages/dateparser/conf.py", line 84, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/dateparser/search/search.py", line 230, in search_dates
settings=settings)}
File "/usr/local/lib/python3.6/dist-packages/dateparser/search/search.py", line 162, in search_parse
original=original, translated=translated, settings=settings)
File "/usr/local/lib/python3.6/dist-packages/dateparser/search/search.py", line 125, in parse_found_objects
parsed_item = self.parse_item(parser, item, translated[i], parsed, need_relative_base)
File "/usr/local/lib/python3.6/dist-packages/dateparser/search/search.py", line 111, in parse_item
parsed_item = parser.get_date_data(item)
File "/usr/local/lib/python3.6/dist-packages/dateparser/date.py", line 418, in get_date_data
locale, date_string, date_formats, settings=self._settings)
File "/usr/local/lib/python3.6/dist-packages/dateparser/date.py", line 196, in parse
return instance._parse()
File "/usr/local/lib/python3.6/dist-packages/dateparser/date.py", line 200, in _parse
date_obj = self._parsersparser_name
File "/usr/local/lib/python3.6/dist-packages/dateparser/date.py", line 223, in _try_parser
self._get_translated_date(), settings=self._settings)
File "/usr/local/lib/python3.6/dist-packages/dateparser/conf.py", line 84, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/dateparser/date_parser.py", line 26, in parse
date_obj, period = parse(date_string, settings=settings)
File "/usr/local/lib/python3.6/dist-packages/dateparser/parser.py", line 72, in parse
raise exceptions.pop(-1)
File "/usr/local/lib/python3.6/dist-packages/dateparser/parser.py", line 66, in parse
res = parser(datestring, settings)
File "/usr/local/lib/python3.6/dist-packages/dateparser/parser.py", line 443, in parse
dateobj = po._correct_for_time_frame(dateobj)
File "/usr/local/lib/python3.6/dist-packages/dateparser/parser.py", line 403, in _correct_for_time_frame
if self.now < dateobj:
TypeError: can't compare offset-naive and offset-aware datetimes
I installed dateparser with pip quite recently
Is there something I messed up?

Which version of Dateparser are you using? Could you try Dateparser from the master branch?

I tested it for versions 0.7.4 (pip) and 0.7.6 (github master branch).

Hi @valmendil,
This is a known issue, and I will try to fix it before the next version.
Thank you for comenting
Closing as duplicate of this: https://github.com/scrapinghub/dateparser/issues/679 (and related to this: https://github.com/scrapinghub/dateparser/issues/491)