explicitly apply dragonberry patch

What is the purpose of the change
The dragonberry patch may or may not be implicitly applied from Osmosis dependencies.  This PR explicitly applies the dragonberry patch to osmosis in go.mod.
There is an upcoming update to ics23 v0.9.0 which will replace this, but I think this is a good idea to have in v13.0.1 (and should probably be what validators upgrade to)

https://github.com/cosmos/ibc-go/pull/2875

The above PR may require changes to the osmosis-sdk, so for now, it likely makes sense to explicitly apply the dragonberry fix in the replace section of go.mod.
Brief Changelog
	// dragonberry
	github.com/confio/ics23/go => github.com/cosmos/cosmos-sdk/ics23/go v0.8.0

to go.mod, and make a few other small cosmetic changes to go.mod
Testing and Verifying
"Dragonberry active" should be visible in node logs.
Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? no
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? no
How is the feature or change documented? (not applicable   /   specification (x/<module>/spec/)  /  Osmosis docs repo   /   not documented)

TYSM, yikes that this was missed


Set microphone and timeline plugins to work with require

Coverage remained the same at 4.2% when pulling 8fa41674b772efe664fc906e540bd27bc05ac8bf on dbunker:master into a58729c410f2516fd440ffbeab38749555416b02 on katspaugh:master.

@thijstriemstra review please, this PR removes AMD definition from the microphone plugin.

@dbunker what are the other changes?

Github has ordered the changes strangely, the only change was to match these two plugins up to the others. If AMD is preferable, is there a reason the other plugins are not set up this way, and examples don't seem to use require + AMD? Thanks.

Oh, I see. The diff is better viewed like this: https://github.com/katspaugh/wavesurfer.js/pull/439/files?w=1
I would prefer plain files, but the microphone, timeline and spectrogram were contributed by different people, and I just went with their preferences.

I quickly tested this with my own app which uses dist/wavesurfer.amd.js. With this change that AMD version no longer works (throws a ReferenceError: cant find WaveSurfer) but the plugins look like they do work with the dist/wavesurfer.min.js. So whether this change is correct or not, I can't say for sure, AMD puzzles me. When I added the microphone plugin I made a copy of the timeline plugin, hence the AMD header in both plugins. I also prefer plain files but I wonder what @dbunker thinks of that wavesurfer.amd.js, do you also use this one or the minified version?

Maybe we could add plugins to the AMD task in the gruntfile (https://github.com/katspaugh/wavesurfer.js/blob/master/Gruntfile.js#L38)?

I guess all we need is a unit test that proves it works or not

@thijstriemstra I actually just use the raw unminified files. These and all my other raw js get loaded using require.js, then webpack spits them into a single minified bundle.js for me. It looks like the only change is that wrapper for amd files, would it make sense to have a separate amd-dist directory where all the plugins have this wrapper? This would be pretty easy to add as a grunt task.

Coverage remained the same at 4.17% when pulling f9c28ce14fad0ef20fb5360c0a33190a49342164 on dbunker:master into 18a57105873f4fc2fa24122b985a4daffd178090 on katspaugh:master.

Coverage remained the same at 4.17% when pulling c5f07363175220b86556c7cdb6bf472810eb9c8a on dbunker:master into 18a57105873f4fc2fa24122b985a4daffd178090 on katspaugh:master.

@dbunker could you update this PR?


stage1: use RootDirectory to chroot instead of diagexec

stage1: use RootDirectory to chroot instead of diagexec
systemd has the option "RootDirectory" to start an app in a chroot, so we use it. Then, diagexec does not need to chroot to start an app, so it can run as an unprivileged user. This fixes the options "user" and "group" in the ACI manifest: https://github.com/coreos/rocket/issues/539
diagexec is still used to diagnose the binary when the execution fails.
Since diagexec is a stage1 tool, it is not in the app's chroot directory. To be able to execute it, diagexec is bind mounted to /.diagexec in stage2.
Since the rkt environment file is outside the chroot, diagexec cannot use it to setup the environment. Instead, the "EnvironmentFile" systemd option is used. diagexec still has the code to setup the environment: it is used for "rkt enter". Systemd's EnvironmentFile needs variables to be separated by '\n' instead of '\0'. Rocket is updated to generate the environment file accordingly.
stage1: create /etc/passwd with systemd-sysusers
Stage1 now creates a configuration files in /usr/lib/sysusers.d/ for each application if the application specifies a non-root user in the image manifest:
    "app": {
        "user": "1000",
        "group": "1000",

With this patch, "user" and "group" in the image manifest can only contain uid and gid (not username and groupname).
A new systemd unit file "sysusers" calls systemd-sysusers before the app is started. This will read the /usr/lib/sysusers.d/ configuration files and generate /etc/passwd and /etc/group. With the example above, the username and groupname will be "u1000".
systemd-sysusers is already available in stage1. systemd requires the libnss_files library in order to open /etc/passwd, so it needs to be added in stage1.
The systemd unit file of the app contain the username and groupname that was created previously.

Very interesting approach, cc @vcaputo @philips

I'm not a fan, diagexec was implemented the way it's implemented for specifically the reasons @jonboulle points out.
However, now that we're building systemd from source in one flavor of rkt, we could add a patch to systemd which includes the diagexec functionality for handling exec() failures.
But that would only be available in the src stage1 flavor, we'd continue using diagexec in the coreos flavor.

systemd already prints some diagnostic errors based on errno after execve() returns and the output of the program when execve() does not return. By testing a systemd unit file manually without Rocket, here are some errors we could get:
testscript[9951]: Failed at step EXEC spawning /testscript: Permission denied
testscript[9959]: Failed at step EXEC spawning /testscript: No such file or directory
testscript[10137]: /testscript: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
testscript[10186]: Failed at step EXEC spawning /testscript: Exec format error

Rocket prints the message from systemd when it is invoked with rkt --debug run .... At the moment, systemd does not print the error messages above because it does not exec the binary directly but through diagexec, so systemd cannot get the errno from execve(). If diagexec is removed, then systemd will be able to print the error message.
I think the error messages above are good enough most of the time. The additional value of diagexec is on errors "ENOENT No such file or directory" or "EACCESS Permission denied" because diagexec can say which file it refers to: it can be the binary itself or the intepretor in the ELF file (ld.so) or the interpretor in the shebang. I like the idea of adding a patch in systemd for that.
systemd.exec has most of the features we want (chroot, setuid, environment, chdir), except the details on the ENOENT/EACCESS errors. I prefer to add the missing features in systemd rather than duplicating the other features (chroot, setuid, environment, chdir) in diagexec. For the same reasons, I have similar thoughts about prepare-app: in the long term, I would prefer to use systemd.exec's options like PrivateDevices and some mount unit files instead of having prepare-app reimplementing that.
About the coreos flavor and the src flavor: I thought that it was good in the short-term to have several flavors to experiment, but in the long term, we will switch to the src option: it allows us to bring our Rocket-specific patches or upstream security patches into stage1's systemd faster than in the Fedora PXE image. So if I am correct, does it matter much that the error messages are less detailed in the coreos flavor?
I see rkt enter as a debug tool, mostly to start a shell in the container. Therefore I think rkt enter should run the shell as root rather than adding calls to setuid()/setgid(). The error message can be improved by printing strerror(errno) after the execv() call. To have the error details on the ENOENT/EACCESS errors without diagexec, I am wondering if the shell can be started from a<EMAIL_ADDRESS>systemd unit file somehow...
I can write a patch to call setuid()/setgid() in diagexec. It will probably be a shorter patch. But I prefer the previous solution because I prefer to use more systemd features and to write less code in stage1 outside of systemd.

systemd already prints some diagnostic errors based on errno after execve() returns

That's interesting because the whole reason we introduced diagexec in the first place was for this very reason - discussion in https://github.com/coreos/rocket/issues/286 . Have systemd's messages just become much less obtuse since the version we were using then?

About the coreos flavor and the src flavor: I thought that it was good in the short-term to have several flavors to experiment, but in the long term, we will switch to the src option

In the long term we need to stabilise the stage0/stage1 interface (https://github.com/coreos/rocket/issues/572) so that we (as upstream Rocket) don't necessarily prescribe any particular stage1 flow; although I would expect we (as CoreOS) would potentially maintain two (built-from-source and derived-from-host)

This approach seems fine to me. As far as I remember #286 was the primary motivation for diagexec and if systemd now provides a good enough error output it is fine with me. The only thing I would like to see is a consistent way of stashing bind mounted internal use dentries that doesn't involve dotfiles in /.

There needs to be an actual clear definition of what rkt enter is supposed to do.
When diagexec learned how to set the environment, and did so consistently for both rkt run and rkt enter, I was already considering renaming it to something unrelated to diagnostics, as it was becoming apparent it would probably be doing even more soon for the sake of consistency between rkt run and rkt enter.
If you see rkt enter as a debugging tool, doesn't its value as such largely depend upon producing an execution environment identical to the app you're debugging?  In my opinion, the rkt enter use case argues against doing more via systemd.exec and instead doing everything affecting the execution environment within a common piece shared by rkt run and rkt enter. Today that piece is called diagexec, but I think it should probably just be renamed to appexec and do everything necessary for consistency between rkt run and rkt enter.
That's my $.02.

@vcaputo is it not conceivable/realistic that this consistent environment for run vs. enter could be achieved through some permutation of systemd tooling though? (nspawn/systemd-run/whatever). (I am just in favour of minimising what we have to maintain wherever possible)

@jonasrosland not AFAIK, but I'm no systemd expert.  If systemd provides what's required for correctly implementing rkt enter then I agree with you.

@alban any thoughts on this last point specifically ^ ?

@jonboulle About this error message:
testscript[10137]: /testscript: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory

That detailed error message about the missing libc.so.6 comes from ld-2.19.so. So if ld-2.19.so exists, the error message about other dynamic libraries will be correctly detailed. But if ld-2.19.so itself is missing, systemd will just print a basic ENOENT No such file or directory without more details. So systemd didn't get better at printing the errors, sadly.
Unfortunately, the systemd team does not want to add code in systemd to guess why execve() returned errno=ENOENT. But they would accept to do that through an external tool with a new option ExecFail=/usr/bin/diagexec /testscript in the unit file to call diagexec when execve() failed to start. I opened a bug for that:
https://bugs.freedesktop.org/show_bug.cgi?id=89840
In this case, diagexec would not exec the application but just inspect a binary. It would be called either through the systemd ExecFail option, or by stage1's enter when execve() fails.
Today, the option ExecFail is not implemented in systemd.exec. In order to fix #539 (run applications as non-root) without waiting for a ExecFail patch to be ready, I could do the setuid in diagexec as @vcaputo suggested, and remove it later? I was hoping to do everything in one PR, but this starts to depend on more things...

@alban ok, good to know, sounds like ExecFail=diagexec could work for us. But actually what I was asking was around the rkt enter stuff: can systemd/systemd-nspawn be used to recreate the same environment as rkt run?

@jonboulle : I tried to use a new systemd unit file for rkt enter in order to start a shell as a child of systemd instead of a child of rkt enter. The pseudo-terminal needs to be reconnected so that the user can interact with the shell. In my patch, rkt enter dynamically allocates a pseudo-terminal in the pod's devpts filesystem and I used the TTYPath option to select it.
But this solution has the following drawbacks:

rkt enter needs to manage an event loop to read and write through the newly allocated pseudo-terminal, in the same way as nspawn does it.
the command to execute is hardcoded to /bin/sh. I already use a template file with one parameter to select the TTYPath. But systemd does not allow template files to have several parameters.

So it does not look too good... any thoughts?
The experimental code is on: https://github.com/endocode/rkt/commit/6053d27b98adb46cd9545a79e7cbbb4ea85592a8

the command to execute is hardcoded to /bin/sh. I already use a template file with one parameter to select the TTYPath. But systemd does not allow template files to have several parameters.

Hmm, could we use a drop-in to achieve the same thing? I was hoping that systemd-run (or nspawn even) would be able to help, but I suppose neither of them support invocation from template/unit files.

rkt enter needs to manage an event loop to read and write through the newly allocated pseudo-terminal, in the same way as nspawn does it.

I don't have any great suggestions for this at this stage

@jonboulle nspawn doesn't do anything enter-like, unless you're implying modifications to nspawn I don't see it bringing anything to the table here.

I overlooked systemd-run... Since v219, systemd-run has an option -t:
       --pty, -t
           When invoking a command as service connects its standard input and
           output to the invoking tty via a pseudo TTY device. This allows
           invoking binaries as services that expect interactive user input,
           such as interactive command shells.

It's not in systemd-v215 currently used by rkt. But I should check if it would work for rkt enter.

I would prefer to use systemd.exec's options like PrivateDevices and some mount unit files instead of having prepare-app reimplementing that

It's not really ready but I have updated my patch to make PrivateDevices work with RootDirectory.

Do you guys think this diagexec binary in Stage1 has anything to do with an error I am seeing on my runit based distro with rocket , a problem in Stage1 with "isolation" and "systemd".   Issue #1076 has the exact error message and just some overall questions on how to proceed.  Thanks.

@bitshark : I don't think the issue you are seeing is related to diagexec. I have replied on #1076.

The current usage of appexec is:
Usage: appexec /path/to/root /work/directory /env/file uid gid[,gid...] [-e] /to/exec [args ...]

appexec
systemd option

chroot
RootDirectory= (systemd >= v220, https://github.com/systemd/systemd/commit/ee818b89f4890b3a00e93772249fce810f60811e)

chdir
WorkingDirectory=

setenv
EnvironmentFile=

setresgid, setresgid
User=, Group=

setgroups
SupplementaryGroups=

execvp
ExecStart=

$PATH lookup
none, needs to implement it when writing the unit file

ELF diagnostic
not implemented, see bfo#89840

appexec might still be needed for rkt enter.

Closing in favor of https://github.com/coreos/rkt/pull/2493


fix(client): 🥅 handle errors for verifyExpiration

the main benefit of this PR is that Too early to make request is no longer an error printed to the console so it is easier to diagnose problems

just run the app and look in the browser console. it should show info instead of error (grey not red)


TypeScript?

Been meaning to have an issue about TypeScript. I personally really want some sort of type checking in the repo, also because IntelliSense makes coding a lot nicer on VS Code, but also because PS has had at least a few crashes that could have been caught by stricter type checking.
Supposedly, research says that even in the absence of good test coverage and/or TDD, strict typing rarely leads to fewer bugs. I don't normally like going against research, but either way it still has plenty of benefits.
This issue is mostly to see what other contributors think of TypeScript.

I tried this with pokemon showdown. Here's are my frustrations when trying to convert battle-engine.js:
https://twitter.com/ARedHerring_/status/833344587183828992
https://twitter.com/ARedHerring_/status/833330440811446273
TypeScript is great, but it'll be hard to get working just due to how incorrect some assumptions are about types in that particular file.

I can't find where I read that static typing doesn't catch bugs, but I've definitely had an Amazon engineer claim it, as well as an article I read online. Both of which seemed to be intended to push me towards TDD.

Static typing won't catch business logic bugs, but it will catch bugs to do with not checking if something is null/undefined, highlight confusing business logic and so forth :smile:

based on the herpderp in #3272, I'll see if I can't get something done with TypeScript over the weekend.

@Zarel is there a chat room that I can communicate with you / other contributors about questions I have? I noticed that there's a Code/Tech room on PS, but I didn't know if that's the right one.

@danpantry try play.pokemonshowdown.com/dev

Yeah, the Dev chatroom is listed in several places, including CONTRIBUTING.md which of course you read, right? ;) ;) ;)

One of my co-workers researched to see if unit testing obviates static typing. This was their Master's Thesis. The conclusion was that static typing would have caught bugs that made it into popular opens source python libraries. The thesis did use Haskell, which has a fundamentally different type system than TypeScript. The thesis can be read here.
This does go against many opinions on the internet and this should not be sufficient to make any decision, but I wanted to provide counter research.

I feel like the biggest argument for TypeScript is that every time we have new OMs, they crash PS due to type errors that could very easily have been fixed... :|

Poring over the handbook, I'm pretty impressed by what it offers. Anything stricter than JS' type system is godsend, let alone the sugar for features JS lacks from other languages, though I don't think TDD should be neglected since the extra insurance of more easily knowing your types are doesn't protect you from screwing up your types. I'm probably not saying anything new, but regardless this is something that looks interesting

@danpantry, anyway, my recent refactor to decisions should at least fix this problem:
https://twitter.com/ARedHerring_/status/833344587183828992

Well, TypeScript 2.3 is a game-changer:
https://github.com/Microsoft/TypeScript/wiki/Type-Checking-JavaScript-Files
It's probably overall better for our use-case, too.

@Zarel if you are planning to use ES2017 as output for TypeScript compilation we should make sure that the node version is the appropriate to support such features. I'm making a PR for this. Please check (http://node.green/)
My background has been mostly in Haskell, Ruby and TypeScript so in my short experience i'd say that: You can add tests to cover some cases, but you'll never get to cover all of the branches of possible scenarios (not even with mutation tests). But, if you have a good type system which can represent accurately your domain model then it should not allow an invalid representation of that model and therefore all cases would be already covered by your type system. Though, you can still add tests but mostly integration tests
That said, I'd love to help with moving the application to TypeScript. Is there already a plan? Could I propose one?

PS's usual plan is support the latest Node LTS version (and avoid using any features not supported by the latest Node LTS version). Usually. If there's a more modern feature we really want, we'll write a polyfill, or as a really last resort, increase our required Node version beyond LTS (so far, this hasn't happened yet). async/await are nice, but we don't use Promises often enough to drop LTS support.
So, in that context, it's probably a bad idea to set TypeScript to ES2017. I'll change it to ES2016 since I believe that's the latest version currently supported by Node 6 LTS.
...actually, currently we use a polyfill for Object.values in a few places, so ES2017 might be necessary to make sure TypeScript doesn't complain about those uses.

As for moving the application to TypeScript: Currently, the plan is to instead move it to TypeScript-jsdoc-compatible JavaScript. This will be easier on third-party server mod writers, and skip the need for a compile step.

Thanks for the link to node.green by the way, it's looks really useful!

...actually, currently we use a polyfill for Object.values in a few places, so ES2017 might be necessary to make sure TypeScript doesn't complain about those uses.

You can include libraries by doing the following for any polyfills:
{
  "compilerOptions": {
    "lib": [
      "es2017.asyncIterable" // as an example
    ]
  }
}

In general, polyfilling with TypeScript is a little messy, but it works

Oh, thanks! That's really useful.

We need to add style guidelines for how we write with Typescript. e.g.
class Foo {
    bar: string
    baz: number
    constructor(bar: string, baz: number) {
         this.bar = bar;
         this.baz = baz;
    }
}

vs.
class Foo {
     /**
      * @param {bar=} string
      * @param {baz=} number
      */
     constructor(bar, baz) {
         this.bar = bar;
         this.baz = baz;
     }
}

as well as
const fs = require('fs');

vs
import fs = require('fs');

vs
import 'fs';

The current plan is to use checkJs, which limits us to JavaScript.
The main style issue is that we probably want to use @param {string} [bar] rather than @param {string=} bar for optional arguments.

How should arrow function parametres be typed when needed? I'm assuming we can't do it with (foo: type) => {} type syntax

TypeScript syntax works inside @param {...} tags.

It does, with the exception of callbacks like foo.on('bar', msg => { /* ... */ }); from what I find. The only workaround I've managed to find is defining the callback outside the function it's being passed to and using @param then, otherwise TypeScript ignores that or @ts-ignore. This isn't a very big issue though, since JSDoc works fine for typing in every other case.

You could just use function statements in this example. It's a few more lines, but the stack trace would be nicer in the event of a failure anyway as you will retain the method name.

foo.on('bar', onBar)
......
return foo

function onBar() {
 ...
}

I think we need to add some rules to the style guide pertaining to Typescript, such as:

types that can also be null: ?type or type??
spacing for object types: {[k:string]:any}, {[k: string]: any} or { [k: string]: any }?
arrays: any[], [any], or Array<any>?
multidimensional arrays: any[][], [any][], [[any]], or Array<Array<any>>?
functions: when would including the parametre and return values like in (any) => any be preferable to simply using Function, or vice versa?

As well as which JSDoc comments are acceptable;

@document is not
@param, @type, @typedef, @return are
what about @class, @extends, @constructor, @property, @name?

?type
{[k: string]: any}
any[]
any[][]
(any) => any
Ask when they come up.

For style with nullable/possibly undefined types, I think it'd be better to write them as type?/type= instead of ?type. Typescript's picky about how those are written, and =type is not valid syntax for function parametres

I guess...

TypeScript is currently partially implemented.
I'm still thinking about actually transitioning to .ts files. It would fix approximately half of our struggles with TypeScript. And I can make the launcher automatically compile them (incrementally, even! thanks to the incremental compiler I just implemented for the client). Our error messages would preserve the line number, and I could probably even silently rewrite the error messages to correspond to the original .ts file lines.
Thoughts?

One of the questions I have is how different is a .ts file from a .js file? Is it just JS with some add-ons and a build step? If its not really different I would be more comfortable with a swap, regardless I should look into this more when I get some time in a few days.

It's a build step and makes writing TypeScript a lot easier, in particular, a not-null assertion is !, and casts and type definitions are also a lot more concise.
A few of the ugliest hacks in dev-tools/global.d.ts and dev-tools/globals.ts would no longer be necessary.

On the other hand, we'll have to use type | null instead of type?, but that's a small price to pay.

Our error messages would preserve the line number, and I could probably even silently rewrite the error messages to correspond to the original .ts file lines.

This is pretty trivial with 'source mapping'. Simply adding require('source-map-support').install();. to the ./pokemon-showdown script or test runners etc should be enough. Its also supported by Chrome etc so you can debug the .ts code directly.

There's no work started on TypeScript migration yet, but the plan is pretty straightforward: just rename the files to .ts one at a time, and add a compiler to ./pokemon-showdown.

Originally posted by @Zarel in https://github.com/Zarel/Pokemon-Showdown/pull/5181#issuecomment-465403979
I'm happy to get the ball rolling on this, however, I'm running into problems  that maybe you have the answer to?. I've converted projects to Typescript before (eg. I extracted and rewrote the damage calc in Typescript and hope to eventually upstream it after some polish), but I did that as one big chunk instead of piecemeal.
Say I take one file (sim/prng.js, because its short) and rewrite it in Typescript (a723e0a). I then run into problems because I want to make tsc emit code (currently our tsconfig.json has it set not to), but by default it will do it in the same place. However, this causes errors regarding the JS files:
error TS5055: Cannot write file 'Pokemon-Showdown/config/config.js' because it would overwrite input file.

...

error TS5055: Cannot write file 'Pokemon-Showdown/sim/side.js' because it would overwrite input file.

error TS5055: Cannot write file 'Pokemon-Showdown/sim/team-validator.js' because it would overwrite input file.

Found 147 errors.

You can configure tsc to ouput to an "outDir", say "./build/". However, it only does this to the files which we "include". Because only part of the project is typed, we now have half the JS in one tree and half in another. One solution would be to have a secondary step for the build to copy the compiled JS files out build and into the main source tree - I think this is what the client does (to get src/*.ts into lib/*.js)? This is kind of ugly IMO, with the build artifacts intermingling with source, but I guess the client sets precedent (I don't think its a problem with like object files in other languages, because one format is binary and one is text, that's not the case with 'compiled' Typescript). We could alternatively copy the source files into build and run ./pokemon-showdown against the build directory.
Apologies if I'm missing something obvious.

I'm still thinking about actually transitioning to .ts files. It would fix approximately half of our struggles with TypeScript. And I can make the launcher automatically compile them (incrementally, even! thanks to the incremental compiler I just implemented for the client).

Can you point me to this incremental compiler? Is it in build-tools and I'm just missing it?

Hm, what's the fastest compiler that supports TypeScript and either source maps or preserving lines?
https://github.com/swc-project/swc is a recent one I saw, I don't see support for either feature, though.
I would probably have tsc emit code to the same directory, at least until an entire directory is converted. After the entire directory's converted, then we can split up source and build directories.

I also have questions about how much performance overhead source-map-support has. We use stack traces in prod, after all.

Hm, what's the fastest compiler that supports TypeScript and either source maps or preserving lines?
https://github.com/swc-project/swc is a recent one I saw, I don't see support for either feature, though.

By 'fastest' do you mean compile times, or the performance of the generated code? And do we care about the former - sure tsc is slow, but is ./pokemon-showdown startup time that concerning? (Or is this about tests? The npm run test script already includes tsc? Is this so npx mocha is fast? We could configure incremental compiles?)

I would probably have the compiler emit code to the same directory, at least until an entire directory is converted. After the entire directory's converted, then we can split up source and build directories.

That's the problem - this doesn't seem to be an option for the project.  I'm assuming you know something I don't, because you already did this on the client?

I also have questions about how much performance overhead source-map-support has. We use stack traces in prod, after all.

I could be mistaken, but I believe source maps work after the fact. ie/ you get a stack trace from prod and combine it with your source map locally to determine where the actual location is. Thus zero performance overhead outside of perhaps minor differences at compile time. I'll dig around and try to find links for it, but thats how it works internally at my company with other tools (I haven't used Typescript at work, but have used/worked on other transpilers).

I'll go comb through the tsconfig.json docs.

Nothing jumps out at me from http://json.schemastore.org/tsconfig and https://www.typescriptlang.org/docs/handbook/compiler-options.html. Maybe we could use --outFile to put everything into a single file?

I'll dig around and try to find links for it.

https://www.html5rocks.com/en/tutorials/developertools/sourcemaps/ seems to be a pretty good explanation. I think the node-sourcemap-support package I linked probably would add overhead as it rewrites the stacktraces, but as the article outlines there are different alternatives which wouldnt impose that overhead at runtime.

By 'fastest' do you mean compile times, or the performance of the generated code?

I mean compile times. Generated code should be roughly equivalent at this point.

And do we care about the former - sure tsc is slow, but is ./pokemon-showdown startup time that concerning?

Yes; fast feedback loops are important for quick development. Especially since there's no real tradeoff. I'm no fan of using tsc for compilation; it's not particularly featureful as a compiler, as you seem to be discovering.

That's the problem - this doesn't seem to be an option for the project.

Sounds like a good reason to use a different compiler. :p

Can you point me to this incremental compiler? Is it in build-tools and I'm just missing it?

https://github.com/babel/babel/pull/8877
In the client, it's just shoved into the repo:
https://github.com/Zarel/Pokemon-Showdown-Client/tree/master/build-tools/babel-cli
Anyway, the server doesn't need to compile down to ES3, so there's no need to screw around with Babel like this. I'd rather just use an off-the-shelf compiler designed for speed, like swc.

I'm assuming you know something I don't, because you already did this on the client?

The client puts TypeScript in src/ and JavaScript in js/, and sets the TypeScript outdir to js/. In hindsight, a similar trick might work here, too.

The client puts TypeScript in src/ and JavaScript in js/, and sets the TypeScript outdir to js/. In hindsight, a similar trick might work here, too.

Oh, perhaps more importantly, the client uses globals for everything, there's no require/import paths to worry about.

Yep. The paths all become bundled.
I'm also having trouble with hybrid the hybrid js/ts requires. I can do something like 043d4f8 for importing types, but I haven't figured out yet how to require sim/prng.ts in sim/battle.js for instance (where we need to actual constructor, not just the type).
My take from Googling is that theres not really a blessed path forward for an already jsdoc typed project to convert piecemeal (let alone a partially typed project like PS). It seems you can either type in jsdoc for js files or type in ts files, but mixing the two is not really that clearly supported.

You might need to convert the entire sim/ directory at once. At least it's comparatively small.

You can also remove --strict, so it's easier, and then slowly bring it back one file at a time.

You might need to convert the entire sim/ directory at once. At least it's comparatively small.

This would only work if none of the other typed-JS code depends on things within sim/ (which they don't because they definitely want sim/dex.js). And even then I'm not sure it would work because I don't know about how clever Typescript is with ordering (will it know to compile and emit code from sim-ts/ to sim/ so that all the other JS requires work? I highly doubt it). I also think we'd need to convert sim/s dependencies (data/, lib/streams.js) for it to compile.
Even if this approach did work, it would still be forcing us to effectively be converting directories at a time, and while sim/ is relatively small, its not an ideal approach.

You can also remove --strict, so it's easier, and then slowly bring it back one file at a time.

Yeah, I kind of wanted to avoid that. It seems really garbage to have to basically lose the benefits of type checking during the period of migration between typed-.js and .ts. :(
What do you think of using --outFile to merge all of the source files (.js and .ts into a single file)? It might also be a non-option because of the ".js doesn't seem to be able to require from .ts" issue I'm encountering, but if i can resolve that issue then --outFile might let us allow us to do files one by one

Ugh. TBH, at this point after wrestling with trying to get typed-JS and ts to play together I'm debating whether it might just be better to bite the bullet and create a feature branch where all 150 files are migrated to .ts in one go. "it shouldn't be too hard" given everything is already typed, the main concern would be doing the migration during a period of time where master isn't being updated too frequently and conflicts could be kept to a minimum.

This would only work if none of the other typed-JS code depends on things within sim/ (which they don't because they definitely want sim/dex.js). And even then I'm not sure it would work because I don't know about how clever Typescript is with ordering (will it know to compile and emit code from sim-ts/ to sim/ so that all the other JS requires work? I highly doubt it). I also think we'd need to convert sim/s dependencies (data/, lib/streams.js) for it to compile.

It's unclear to me why this would be the case. Most of our dependencies are currently globals, anyway (for hotpatch compatibility), so we have some of the affordances that allow us to intermix things in client.

I'm trying to convert sim/. I don't think it will work, but YOLO.
Very interesting the types of things Typescript catches with .ts filles that checkJs seems to miss:

Unless I'm mistaken and {Pokemon} [pokemon] in the JSDoc should not be translated as pokemon?: Pokemon in Typescript, it seems like the compiler is also better at actually noticing things that could be undefined (which, if we know they can't be in this particular circumstance we can trivially fix with action.pokemon!.position etc). My point being, it seems like we get better type checking in .ts

Did a rough conversion for all of sim/ except for battle.js - I'll tackle it in the morning and start actually trying to compile things to prove why I'll actually probably need to convert all typed-JS in one go (I'd of course love to be wrong).
dex-data.js is the worst with all the fields, and Typescript really hates the Object.assign pattern being used - had to use tons of // @ts-ignore for error TS2565: Property 'Foo' is used before being assigned.. I have an idea for a Typescript-friendlier pattern, but just converting over the existing code as-is is my main goal - I plan to // @ts-ignore most new potential bugs identified by tsc with the .ts files and then go back later and try to eliminate as many as possible.

You may be interested in my TypeScript-friendlier pattern:
https://github.com/Zarel/Pokemon-Showdown-Client/blob/master/src/battle-dex-data.ts
Although unlike server, client explicitly discards unrecognized data.

Yup, thats basically it. Just do the Object.assigns to merge the data and moreData etc and come up with an any object, then pull fields off of it. Pretty easy, but I'm trying not to make any logic changes whatsoever, just rote conversion to limit the scope.

I then run into problems because I want to make tsc emit code (currently our tsconfig.json has it set not to), but by default it will do it in the same place. However, this causes errors regarding the JS files
Originally posted by @scheibo in https://github.com/Zarel/Pokemon-Showdown/issues/3278#issuecomment-466565545

Thought of a workaround for this - we'll have two separate configs:

the current one which is strict, checkJs/allowJs and is used for type checking, not compiling/emitting
a new configuration, which includes just the .ts and doesn't support .js, but emits

The emitting configuration will be used by swc/sucrase/tsc or whatever #5199 lands on for ./pokemon-showdown/build runner in package.json.

It's unclear to me why this would be the case. Most of our dependencies are currently globals, anyway (for hotpatch compatibility), so we have some of the affordances that allow us to intermix things in client.
Originally posted by @Zarel in https://github.com/Zarel/Pokemon-Showdown/issues/3278#issuecomment-466705089

Yes and no. I dismissed this original, but  you're correct that globals will buy use some affordances provided that they are defined (and not just declared) in globals.ts (not server/index.js, which can't require the .ts files to new up the global instances`).

Here we go: https://github.com/pkmn-cc/Pokemon-Showdown/tree/tsc/sim - fully converted. The problem is sim/ is not perfectly encapsulated - some random-team.js files rely on being able to require and call new PRNG, and data/mods/ssb/moves.js uses new Pokemon thanks to something called 'wondertrade'.
I'm going to dive into data/ now...

Alright: converting all the data/**/random-team.js and data/mods/ssb/moves.js seemed to do it, as npx tsc now compiles without errors. npx mocha on the emitted code fails catatrophically - I think maybe because I tried to change the import pattern from export = ... to export class ... to be more idiomatic in a lot of places. Who knows. Will try again tonight.

I'd rather spam @ts-ignore in the random team and mods than convert them, tbh.

Alright, noted. We can evaluate that approach once I get things working. Right now I'm just doing whatever I can to get a working prototype.

BOOM! We're in business! I've got npm run test working with all tests passing on my branch! :D
Here's what's left:

[ ] keep synced to HEAD until it can be reviewed and land (*gulp*)
[ ] revert data/**/random-teams.ts and data/mods/ssb/moves.ts conversions and try to use// @ts-ignore to get around the problems instead as per https://github.com/Zarel/Pokemon-Showdown/issues/3278#issuecomment-4668248284828
[ ] set up build script which runs npx tsc -p emit.json 2>/dev/null and use it in ./pokemon-showdown
[ ] see if its possible to resolve the complaints in the above step that have me piping to /dev/null, purely because it would be cleaner
[ ] try to set up swc or sucrase as the compile step instead of swc. This is tracked in #5199. @Zarel: is using tsc as the compiler for the time being a blocker that would prevent you from merging my PR?

Follow up after landing:

[ ] remove as many // @ts-ignores as possible and clean up types

From https://github.com/Zarel/Pokemon-Showdown/issues/5199#issuecomment-466859039 I actually think using sucrase renders two of those bullets obsolete because it doesn't give any warnings/errors and is mad fast and probably even simpler to use than tsc plus a special emit.json config. (Y)

My branch is currently synced to HEAD (there were only 3 small merge conflicts, all related to 3 of my previous commits, doh). I might find time before/after work tomorrow to make ./pokemon-showdown auto build it and then do a pass to polish the changes  as I kind of blitzed through this and it definitely needs a sanity check. Still, happy with how it turned out! It was simultaneously a bigger and smaller job that I thought it would be.

I notice you're using type where the standard is to use interface. e.g. type ChosenAction = { should probably be interface ChosenAction {.

Sucrase seems fine to me. The only part that would be an issue is initializing properties where they're declared, but I guess we can just avoid doing that.

Given sim and lib are now native Typescript, and work on server is underway, I don't know that this issue still needs to be open. We are not fully on Typescript yet, but its hard to see us going back now.


Actions in sidebar open modals inside of sidebar instead of center of screen

Package
filament/filament
Package Version
3.1.43
Laravel Version
10.3.2
Livewire Version
3.3.5
PHP Version
8.3
Problem description
Some apps have actions at the top and/or bottom of the sidebar. For example a single full-width button at the top or
"Get help" and "Send feedback" icon buttons at the bottom.
However actions added to the Filament sidebar open their modals inside of sidebar instead of in the center of the screen:

Expected behavior
Action triggers in the sidebar should open modals in the center of the screen.
Steps to reproduce
Launch the reproduction repository, then click the button at the top of the sidebar or the icon button at the bottom of the sidebar.
Reproduction repository
https://github.com/binaryfire/filament-action-bugs
Relevant log output
No response

This wasn't really the intention of the component, so can you please debug this and work out what's happening? Maybe the modal component needs to live in the main page content, and you just emit an event to open it from the sidebar?

Dispatching an event from the sidebar buttons to open a modal in the main page works. Thanks.

@binaryfire
Hey, I have the same use case. Can you please tell me how you fix it?

Hi @mwaqasiu. Check out the code in the reproductin repo. I updated it with a working example.


Adds Prowlarr app

What this PR does / why we need it:
This PR adds Prowlarr, an indexer aggregator similar to Jackett, but includes NZB indexers
Prowlarr Wiki

Feels good to help out!


postgres: Native upsert detection broken

From #133:
For some reason, when I wrote this change my database was correctly detected as supporting native upsert, but at least on my current setup, the detection is broken:
ueberdb=> EXPLAIN INSERT INTO store(key, value) VALUES ($1, $2) ON CONFLICT (key) DO UPDATE SET value = excluded.value;
ERROR:  there is no parameter $1
LINE 1: EXPLAIN INSERT INTO store(key, value) VALUES ($1, $2) ON CON...
                                                      ^

When filled with real values, however:
ueberdb=> EXPLAIN INSERT INTO store(key, value) VALUES ('key', '{"value": true}') ON CONFLICT (key) DO UPDATE SET value = excluded.value;
                     QUERY PLAN
-----------------------------------------------------
 Insert on store  (cost=0.00..0.01 rows=1 width=250)
   Conflict Resolution: UPDATE
   Conflict Arbiter Indexes: store_pkey
   ->  Result  (cost=0.00..0.01 rows=1 width=250)
(4 rows)

I’ll open a separate issue for that.
Originally posted by @qsuscs in https://github.com/ether/ueberDB/pull/133#issuecomment-689521070

I reverted these commits as they were breaking.


fsObjectProcessor - OnBeforeDeserialize and "$type"

Hello Jacob,
After updating to the latest FS, I am encountering an issue with OnBeforeDeserialize of the fsObjectProcessor and a trick I used that modifies the "$type" json entry of the object proccessed.
As it seems the json deserialized is not being modified at all now. This used to work great a couble of versions back (about 3 weeks ago).
Here is the critical part of the processor code:
	public override void OnBeforeDeserialize(Type storageType, ref fsData data){

		if (data.IsNull)
			return;

		var json = data.AsDictionary;

		if (json.ContainsKey("$type")){

			var serializedType = ReflectionTools.GetType( json["$type"].AsString );
			if (serializedType == null){
				json["$type"] = new fsData( typeof(MissingNode).FullName );
			}
		}
	}

Thanks a lot!
Gavalakis Vaggelis

I'll take a look this weekend, but this is almost certainly related to 414e21ee0b4ccf33af773a93fdad8bd9a84c9cf0.

What does the CanProcess function look like?
If it looks like typeof(BaseNode) == type, can you try changing it to return typeof(BaseNode).IsAssignableFrom(type) instead?
I made a change so that processors are resolved based on the actual instance type instead of the type passed in - still thinking about the proper behavior. I may revert it to be based on the declared deserialization type and not the instance type.

Also, is OnBeforeDeserialize actually getting invoked?

Hey Jacob,
Thanks for taking a look into it.
Sorry. I forgot to mention it initialy, but OnBeforeDeserialize is called normaly yes. That's the most weird part actually.
CanProcess is also "declared" like typeof(BaseNode).IsAssignableFrom(type).
Right after OnBeforeDeserialize is called, the error "Cannot create an instance of an interface or abstract type for BaseNode" is thrown, which is normaly the case when the serialized $type is missing. That's what I am trying to handle in OnBeforeDeserialize by altering the serialized information.
This used to work a couple of version back. I can't really say why it doesn't now :)
If you need more information, please let me know.
Thanks a lot!

I believe the given commit should fix the issue. Let me know.

Hey Jacob,
Everything works fine again with the latest commit.
Thanks a lot!


Use Vector for size() in Renderer

A size() is more accurately represented by a Vector than by a Point.

Huh, looks like I requested review after you already reviewed it.
Well, thanks for the quick review.


watch any python extension add and don't request a reboot

For #405

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[x] Title summarizes what is changing.
[x] Has a news entry file (remember to thank yourself!).
[x] Appropriate comments and documentation strings in the code.
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated.
[ ] Test plan is updated as appropriate.
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).

As @rchiodo mentioned on Teams we should poke at this more. But in my basic testing it was just fine to hook up the python extension and not reboot. The API hooks up ok and calls to it work. Tried stuff like Dataviewer and run by line.

Only test failures were widget failures, so merging.


Add missing feature alert if recent feature data is missing

Issue #, if available:
Description of changes:
Add missing feature alert if recent feature data is missing
If latest 2 feature data points of any feature are missing, we show yellow warning like below

If latest 3 feature data points of any feature are missing, we show red alert below:

Regardless of yellow or red alert, as long as there exists missing feature data, there is annotation on the feature chart.

Meanwhile, I will check with tech writer on wording change >> Done
When data point is missing, below is shown:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

If latest 2 feature data points of any feature are missing, we show yellow warning

Why don't show yellow warning if 1 feature data point missing? So user can know the data ingestion may have problem earlier.

If latest 2 feature data points of any feature are missing, we show yellow warning

Why don't show yellow warning if 1 feature data point missing? So user can know the data ingestion may have problem earlier.

It may be too sensitive if latest 1 data point is missing, and ingestion is recovered soon. This may happen if data ingestion is delayed occasionally. If latest 2 consecutive data points are missing, it can be considered as consistent behavior and we should let user be aware of this.

Issue associated with this: https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin/issues/249

Also, general question, is there an easy way to remove the line connecting the feature data points where data was missing in between? Feel like it shouldn't be included.

Also, general question, is there an easy way to remove the line connecting the feature data points where data was missing in between? Feel like it shouldn't be included.

Actually this is a corner case. My local ES process somehow stopped itself during that period. Ideally it should not happen. And the connecting line is automatically drawn by Elastic chart library. Let me explore if it is possible to skip it.

Also, general question, is there an easy way to remove the line connecting the feature data points where data was missing in between? Feel like it shouldn't be included.

Actually this is a corner case. My local ES process somehow stopped itself during that period. Ideally it should not happen. And the connecting line is automatically drawn by Elastic chart library. Let me explore if it is possible to skip it.

After exploring, I didn't find good way to remove the connecting line. Issue created: https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin/issues/250

Also, general question, is there an easy way to remove the line connecting the feature data points where data was missing in between? Feel like it shouldn't be included.

Actually this is a corner case. My local ES process somehow stopped itself during that period. Ideally it should not happen. And the connecting line is automatically drawn by Elastic chart library. Let me explore if it is possible to skip it.

After exploring, I didn't find good way to remove the connecting line. Issue created: #250

Yeah, not a big deal, thanks for looking into it!


Create template for the custom environment in StableBaselines3

Using the information at this link we need to create an environment that mirrors Romina's existing Gym environment, adapted for input from OpenCV, the servo motors, and outputs to the servo motors.
`import gym
from gym import spaces
class CustomEnv(gym.Env):
"""Custom Environment that follows gym interface"""
def __init__(self, arg1, arg2, ...):
	super(CustomEnv, self).__init__()
	# Define action and observation space
	# They must be gym.spaces objects
	# Example when using discrete actions:
	self.action_space = spaces.Discrete(N_DISCRETE_ACTIONS)
	# Example for using image as input (channel-first; channel-last also works):
	self.observation_space = spaces.Box(low=0, high=255,
										shape=(N_CHANNELS, HEIGHT, WIDTH), dtype=np.uint8)

def step(self, action):
	...
	return observation, reward, done, info
def reset(self):
	...
	return observation  # reward, done, info can't be included
def render(self, mode='human'):
	...
def close (self):
	...`

Dependent on #2 but can be worked in parallel with assumption that the APIs which are created follow the format specified in #2.


Updated missing types in help and docs

Updated valid image types based on the current supported options

Ah we were doing the same thing at the same time, see also https://github.com/osbuild/bootc-image-builder/pull/269


Linux kernel updates 2023-02-25

Description of changes
Previous merge request: https://github.com/NixOS/nixpkgs/pull/217699
Things done
Built and tested 6.1.14-hardened1

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

From: @alyssais on this comment on #218660

To whoever merges this: please merge https://github.com/NixOS/nixpkgs/pull/218694 at the same time (to group the rebuilds together), assuming OfBorg is happy with it by then.

Currently testing...


Add a docker container

I'll add a docker container to reduce the overhead of setting up an environment.

Sweet. Fixes #14.

@bparish628 Just a heads up.  I created a project so we have a kanban to look at.  I don't know if you get notifications when I add cards to the TODO column, but you can view the project and move the cards around in the projects tab.


fixed the error message for a user to open the crate

What does this PR try to resolve?
This PR is resolving an issue where cargo doc --open --examples does not give a clear message when the examples file is given to it.
I changed the message from no crates with documentation in the cargo_doc.rs and doc.rs to requested crate documentation is not available to open. Now it becomes easy for the user to understand what they're missing.
Fixes #14776
How should we test and review this PR?
Here is the command through which you can test this change:
cargo test SNAPSHOTS=overwrite -- doc::open_no_doc_crate

@ibilalkayy The fmt check failed. You need to run cargo fmt --all.

Thank you, @Rustin170506! I was not aware of this. Once I get the clarification message from @epage, I will push again.

Would you be ok cleaning up the commit history?


Upgrading to DFX 0.23.0 failing to build canisters - No crate X found

Although there aren't any breaking changes, when upgrading from 0.22.0 to 0.23.0 my project is no longer able to build:
Error: Failed while trying to build all canisters.
Caused by: The build step failed for canister 'bkyz2-fmaaa-aaaaa-qaaaq-cai' (zk-rust-vue-demo-backend)
Caused by: Failed to build Rust canister 'zk-rust-vue-demo-backend'.
Caused by: Failed to create <Type>CanisterInfo for canister 'zk-rust-vue-demo-backend'.
Caused by: No crate `zk-rust-vue-demo-backend` found

Here is my DFX file:
{
	"canisters": {
		"internet-identity": {
			"type": "pull",
			"id": "rdmx6-jaaaa-aaaaa-aaadq-cai"
		},
		"zk-rust-vue-demo-backend": {
			"dependencies": [
			  "internet-identity"
			],
			"candid": "src/zk-rust-vue-demo-backend/zk-rust-vue-demo-backend.did",
			"package": "zk-rust-vue-demo-backend",
			"type": "rust"
		},
		"zk-rust-vue-demo-frontend": {
			"dependencies": [
				"zk-rust-vue-demo-backend"
			],
			"source": [
				"src/zk-rust-vue-demo-frontend/dist"
			],
			"type": "assets",
			"workspace": "zk-rust-vue-demo-frontend"
		}
	},
	"defaults": {
		"build": {
			"args": "",
			"packtool": ""
		}
	},
	"output_env_file": ".env",
	"version": 1
}

Meta
dfx --version:
0.22.0 upgrading to 0.23.0

Further log output

Building canisters...
Checking for vulnerabilities in rust canisters.
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 659 security advisories (from /home/kairus/.cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (79 crate dependencies)
Audit found no vulnerabilities.
Building frontend...
Running cd "/mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo" && CANISTER_CANDID_PATH="/mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/.dfx/local/canisters/zk-rust-vue-demo-frontend/assetstorage.did" CANISTER_ID="bd3sg-teaaa-aaaaa-qaaba-cai" CANISTER_ID_INTERNET_IDENTITY="rdmx6-jaaaa-aaaaa-aaadq-cai" CANISTER_ID_ZK_RUST_VUE_DEMO_BACKEND="bkyz2-fmaaa-aaaaa-qaaaq-cai" CANISTER_ID_ZK_RUST_VUE_DEMO_FRONTEND="bd3sg-teaaa-aaaaa-qaaba-cai" DFX_NETWORK="local" DFX_VERSION="0.23.0" "npm" "run" "build" "--workspace" "zk-rust-vue-demo-frontend"...
WARN: Generating type declarations for canister zk-rust-vue-demo-frontend:
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-frontend/zk-rust-vue-demo-frontend.did.d.ts
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-frontend/zk-rust-vue-demo-frontend.did.js
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-frontend/zk-rust-vue-demo-frontend.did
Generating type declarations for canister zk-rust-vue-demo-backend:
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-backend/zk-rust-vue-demo-backend.did.d.ts
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-backend/zk-rust-vue-demo-backend.did.js
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/zk-rust-vue-demo-backend/zk-rust-vue-demo-backend.did
Generating type declarations for canister internet-identity:
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/internet-identity/internet-identity.did.d.ts
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/internet-identity/internet-identity.did.js
  /mnt/c/Users/Kairus/git/IC/zk-rust-vue-demo/src/declarations/internet-identity/internet-identity.did

Error: Failed while trying to build all canisters.
Caused by: The build step failed for canister 'bkyz2-fmaaa-aaaaa-qaaaq-cai' (zk-rust-vue-demo-backend)
Caused by: Failed to build Rust canister 'zk-rust-vue-demo-backend'.
Caused by: Failed to create <Type>CanisterInfo for canister 'zk-rust-vue-demo-backend'.
Caused by: No crate `zk-rust-vue-demo-backend` found

This seems to be a problem associated with the following changes
https://github.com/dfinity/sdk/pull/3836
This can be worked around by changing hyphens in name of the crate to underscores.
Since hyphens are not used as identifiers inside Rust, I assume this is due to the conversion of hyphens to underscores, but if this side effect is unintentional, it needs to be fixed.


Add missing supported OSes from the test matrix

Description

Add missing supported OSes from the test matrix (Fixes: #194)

Issues Resolved
That tests are carried out on more supported OSes.
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[ ] New functionality includes testing.
[ ] New functionality has been documented in the README if applicable
[x] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

Missed adding entries to kitchen.vagrant.yml, that's been done and more CI should pass now.

Not sure 🤔 if this is viable, taking a very long time to run.
I'll see if it's possible to make this functional.

@jjm the repo used to run with kitchen-dokken to workaround the lengthy testing time, but with the last release, I had to switch it back to kitchen-vagrant due to a limitation with systemd and docker. The jobs are running in parallel somewhat; perhaps multiple workflows that run in parallel?

I guess that could help, I know that in terms of billing GitHub actions for macos use 10x the number of minutes as for Linux builds. So it could very well be that there's just less capacity to call on and these builds spend later being queue up.
I've raised another PR to correct the issue that caused me to notice not all platforms were being tested and add a new feature.
I've successfully tested centos-6 locally with dokken, what was the issue you had with systemd? Would it be possible to split out the tests so that some happen on macos and some on linux and dokken? For example disabled could possibly run for all platforms on dokken.

@jjm I couldn't find the right combination of settings to run in kitchen-dokken while running splunk as a non-root user with systemd

I've never tried configuring Test Kitchen to run tests under different provisioners. I suppose that we could create two separate Actions workflows: one where the ENV VARs uses the kitchen.dokken.yml file and the other uses the kitchen.vagrant.yml file. The test matrix would need to changed.

I've never tried configuring Test Kitchen to run tests under different provisioners. I suppose that we could create two separate Actions workflows: one where the ENV VARs uses the kitchen.dokken.yml file and the other uses the kitchen.vagrant.yml file. The test matrix would need to changed.

Given how the workflow with each suite and OS being it's own matrix step, running some of the tests under dokken and only those that require a change to a non-root user as different jobs is totally possible.

If you would like I can do the split out into sections tomorrow (it's a little late for doing PRs now here).

sounds good. Thanks for contributing!

sounds good. Thanks for contributing!

Your welcome, happy to help out.

Well the tests are quicker for sure, but there's a huge wait for them to kick off that I've not seen before. But that's either under my Pro account or in a paid up Team org.
The single suite that failed above passed on my local machine.
Profile: tests from {:path=>"/Users/jonmiddleton/public-github/chef-splunk/test/integration/inspec/server_test.rb"} (tests from {:path=>".Users.jonmiddleton.public-github.chef-splunk.test.integration.inspec.server_test.rb"})
Version: (not specified)
Target:  docker://de69e57eb684c5126992aa70385f16aa5eef4abbbb06a3a3db865a1c3cb0ba22

  ✔  Enterprise Splunk: Verify Enterprise Splunk server installation
     ✔  chef-splunk::server listening ports Port 8089 is expected to be listening
     ✔  chef-splunk::server listening ports Port 8089 protocols is expected to include "tcp"
     ✔  verify inputs config File /opt/splunk/etc/system/local/inputs.conf is expected to be file
     ✔  verify inputs config INI /opt/splunk/etc/system/local/inputs.conf default.host is expected not to be nil
     ✔  verify inputs config INI /opt/splunk/etc/system/local/inputs.conf default.host is expected not to be empty
     ✔  System Package splunk is expected to be installed
     ✔  Service Splunkd is expected to be installed
     ✔  Service Splunkd is expected to be enabled
     ✔  Service Splunkd is expected to be running
     ✔  Processes /splunkd.*-p 8089 _internal_launch_under_systemd/ is expected to exist
     ✔  Processes /splunkd.*-p 8089 _internal_launch_under_systemd/ users is expected to include "root"
     ✔  File /etc/systemd/system/Splunkd.service is expected to exist
     ✔  File /etc/systemd/system/Splunkd.service is expected to be file
  ✔  Splunk admin password validation: Splunk admin password
     ✔  File /opt/splunk/etc/system/local/user-seed.conf is expected not to exist
     ✔  File /opt/splunk/etc/system/local/.user-seed.conf is expected to exist
     ✔  Command: `/opt/splunk/bin/splunk login -auth admin:notarealpassword` stderr is expected to be empty
     ✔  Command: `/opt/splunk/bin/splunk login -auth admin:notarealpassword` exit_status is expected to eq 0

Profile Summary: 2 successful controls, 0 control failures, 0 controls skipped
Test Summary: 17 successful, 0 failures, 0 skipped

I think I've got it, there was the following in the dokken config:
    lifecycle:
      post_converge:
        - local: sleep 120

Which I removed as it was not in the vagrant config, however the 2 server runas suites need a little more time to start as those are the ones that need Splunkd to have a little more time to complete. So moved the above lifecycle blocks onto the those 2 suites so that they have more time to be ready for inspec tests.

Happy dance! All steps are passing!

Nice work, @jjm !


Readers might like the Concourse Tutorial

Great post. Perhaps readers might like the Concourse Tutorial to learn more.

Thanks @drnic, sorry about the delay :)


Added serializability functionality

Some users might need an import/export functionality for the current LFUCache data-structure for portability/transmission over get/post requests.
Added:
import(): To import and set an already existing LFUCache
export(): To export an already existing LFUCache as string
Additional dependencies: circular-json to parse and stringify Circular referenced JSON Objects.

@kapouer Can you please review the request?

Well i've asked a question above !

I'm not able to view any other comments except for this... Can you please post it again?

Oops, sorry i'm not that used to the github review gui.

@kapouer is the checking needed?

Well i miss some time to fully review your changes - and more importantly, to write some tests to make sure they won't break anything.
I'm also not sure casting unconditionally is a good idea.


Swipe

Is there any way to get when user swipes and have the "index" of the current image?

If you get the latest commit from the master branch there is a OnPageChangeListener (see demo). #100
If you are using Gradle, it's not in the 1.0.9 release but hopefully we'll get a new release soon.


ghcide needs -fexternal-interpreter?

We're getting this error in ghcide.
ghcide: ghcide: this operation requires -fexternal-interpreter

https://github.com/hercules-ci/ghcide-nix/pull/18#issuecomment-555127101
Shouldn't ghcide hardcode this flag if it depends on it, or how do we deal with this?

I think -fexternal-interpreter implies other things that we don't do, so I'm not sure it's as simple as adding the flag in.

To add a bit more: We definitely don’t always depend on -fexternal-interpreter. This sounds like something that might be caused by a project using template haskell?

I did some research earlier today and it turns out it can also be thrown when ghc is compiled without ghc support, which seems to be the case with haskell.nix at the moment.
As far as I can tell, -fexternal-interpreter is only needed for cross-compilation, ghcjs and those kinds of things, whereas my setup  did not have any of that.
So this does not seem to be a problem in the ghcide project.


how to insert some data in collection?

collection.add_row() is just adding new row..
Can I insert some data into some specific row?
so I can update numbers in same row -

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

I do not get it ..
your code does not seem to work

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

I do not get it ..
your code does not seem to work

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

Thank you I got it!

from notion.client import NotionClient
token = 'your token'
client = NotionClient(token_v2=token)
list_url = 'table url'
cv = client.get_collection_view(list_url)
def update_kanban_states():
for row in cv.collection.get_rows():
if row.Done is True:
row.Kanban = 'Done'
if row.Done is False:
row.Kanban = 'To Do'
update_kanban_states()
´´´
just use get_rows() on collection_view method and iterate over it in order to access each row.

Thank you I got it!


Support JDK 17 from Gradle 6.9.*

I'm using Java 17 almost exclusively and have had to update numerous projects to Gradle 7 so that I can target JDK 17 as the minimum version. However, there are some projects which use plugins that have not yet been updated to support Gradle 7 and are stuck on version 6. This means that we cannot target JDK 17 on those projects, it would be really helpful if the latest releases of Gradle 6.9 could support the more recent JDK so that I can target Java 17 without needing to replace the currently used plugins.

I've not but will take a look into it thanks


Indicate dates & versions

Hello Martin,
At the moment, the comparison is massively out-of-date - as it seems to refer to doctest when it was less than half its current age, IIANM.
But be that as its may - I believe it is imperative, so as not to confuse people, to do the following:

Indicate the release version (or commit hash) used for each column of the comparison table.
Indicate the date the comparison was made (which may not be the date of the last editing of README.md)

Hi Eyal,
Thanks for the heads-up, you're quite right.
I'm not inclined to invest much work into this at the moment. Perhaps it's nice to keep the comparison for 'historical reasons' and adding a big fat warning to indicate it is ancient is the appropriate thing to do.

@martinmoene : Indicating versions and the date of comparison should really not take much time :-(


Update javadoc attribute descriptions

We need to update the javadocs for the so that the description of the fields used for Entry, Tool, Workflow, Tag, etc match up with what the fields actually do.
Currently there is some disconnect, as the uses for some of the attributes have changed over time.

Closing, will probably need something more specific to be actionable


options -r, --limit-rate RATE does not work

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Description

does not work options:
-r 256K
--limit-rate RATE 256K
Before the update to version 2<IP_ADDRESS>, speed limit worked well.
Now this option does not work and the download of the video occurs at the maximum speed.

Raise the issue at https://github.com/yt-dlp/yt-dlp/issues.


Remove outdated comment about shell args on different platforms.

Description
I made a reference to this comment in #9509 and @lamont-granquist said it was outdated and should be removed.
Related Issue
https://github.com/chef/chef/pull/9509#issuecomment-600949638
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] I have run the pre-merge tests locally and they pass.
[x] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[x] All commits have been signed-off for the Developer Certificate of Origin.

Thanks for the cleanup @phiggins

👍


Android toolchain single ABI

This series:

Updates the Android toolchain script to specify a single ABI (default is still all)
Specifies a toolchain ABI in the Travis-CI config file
Adds a 64-bit Android job to run in parallel

End result is a much faster Android toolchain build (~3m30s -> ~42s for arm32 locally), and better coverage in our public CI.

cherry-picked to master, thanks!


Editorial Changes

remove prefix IDTA-01001 and IDTA-01xxx from files
update all xref:
merge files from Annex and SharedAnnex to Annex folder

substituted by #441

@zrgt may you please look for the confict , I just merged your PR #441. Thank you

@BirgitBoss the PR is ready to be merged now


add support for SQL Server DateAdd()

the following fails to parse:
select start_date where start_date > DATEADD(HH,-?, GETDATE())
Caused by: net.sf.jsqlparser.parser.ParseException: Encountered " <S_IDENTIFIER> "HH "" at line 1, column 46.
DATEADD() is describe here:
https://msdn.microsoft.com/en-us/library/ms186819.aspx

it seems the problem is with the negated parameter, if you change it (remove the minus) it works:
select start_date where start_date > DATEADD(HH, ?, GETDATE())


Update year in NOTICE-binary

Description
Update year in NOTICE-binary
Ref: https://github.com/apache/spark/blob/master/NOTICE-binary
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

[ ] Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

[ ] Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

[ ] Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes
N/A
Documentation
N/A

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@f1f1c41). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #5547   +/-   ##
=========================================
  Coverage          ?   66.58%
=========================================
  Files             ?     1104
  Lines             ?    56871
  Branches          ?     8505
=========================================
  Hits              ?    37869
  Misses            ?    16210
  Partials          ?     2792

Flag
Coverage Δ

#integrationtests
45.12% <ø> (?)

#unittests
56.85% <ø> (?)

Impacted Files
Coverage Δ

...er/starter/helix/SegmentMessageHandlerFactory.java
69.33% <0.00%> (ø)

...impl/dictionary/FloatOffHeapMutableDictionary.java
59.13% <0.00%> (ø)

.../apache/pinot/minion/metrics/MinionQueryPhase.java
100.00% <0.00%> (ø)

...inot/server/api/resources/HealthCheckResource.java
0.00% <0.00%> (ø)

...index/converter/SegmentFormatConverterFactory.java
20.00% <0.00%> (ø)

...elix/core/relocation/RealtimeSegmentRelocator.java
81.57% <0.00%> (ø)

...he/pinot/core/query/config/QueryPlannerConfig.java
60.00% <0.00%> (ø)

.../core/indexsegment/mutable/MutableSegmentImpl.java
82.90% <0.00%> (ø)

.../pinot/core/query/scheduler/PriorityScheduler.java
80.82% <0.00%> (ø)

...apache/pinot/core/operator/ProjectionOperator.java
94.11% <0.00%> (ø)

... and 1094 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1f1c41...fe1d02f. Read the comment docs.


[0.8] Polymer.Base.chainObject tosses out existing prototype chains

Polymer.Base.chainObject will toss out a prototype chain that already exists on the object being chained to inherited. Since the Polymer helper function uses Polymer.Base.chainObject to chain a provided prototype with Polymer.Base, it is very difficult to apply traditional prototypal inheritance strategies to Polymer elements.

Closing as this API is on the legacy element.


⚠ Samsung has degraded performance

In 5a255bf, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 1924 ms

Resolved: Samsung performance has improved in 157abb4.


timeout is buggy in JRuby

First I thought it was celluloid-io sockets, but then I realized that in Jruby, timeouts will not always be triggered. Here is a simple script:
require 'time'
require 'celluloid/io'

class Tester
  include Celluloid::IO
  def initialize
    @sock = TCPSocket.new("localhost", 80)
  end

  def test_timeout
    begin
      timeout(2) do
         # option 1
         # @sock.wait_readable
         # option 2
         # sleep 3
         # option 3
         loop do
            sleep 1
         end
      end
    rescue Celluloid::Task::TimeoutError
      puts "successfull timeout fetch: #{Time.now}"
      retry
    ensure
      puts "#{$!.class} #{$!.message}" if $!
    end
  end

end

puts "starting the actor..."
t = Tester.new
t.async(:test_timeout)
sleep

# Option 1
# Both MRI 2.1 and jruby <IP_ADDRESS>
....
successfull timeout fetch: 2016-01-27 18:50:47 +0100
successfull timeout fetch: 2016-01-27 18:50:49 +0100
successfull timeout fetch: 2016-01-27 18:50:51 +0100
successfull timeout fetch: 2016-01-27 18:50:53 +0100
successfull timeout fetch: 2016-01-27 18:50:55 +0100
successfull timeout fetch: 2016-01-27 18:50:57 +0100
successfull timeout fetch: 2016-01-27 18:50:59 +0100
successfull timeout fetch: 2016-01-27 18:51:01 +0100

(hangs at some point...)

# Option 2
# MRI 2.1 and JRUBY-<IP_ADDRESS>
starting the actor...
successfull timeout fetch: 2016-01-27 18:49:12 +0100
W, [2016-01-27T18:49:15.799990 #328595]  WARN -- : Attempted to resume a dead task: type=:call, meta={:dangerous_suspend=>false, :method_name=>:test_timeout}, status=:dead

Option 3 works in MRI (apparently), but also hangs at some point.
This can seriously hinder https://github.com/celluloid/celluloid-io/pull/170.

Yeah, this has been reported several times...
Any ideas @ioquatix?

I will investigate the issue, is the above test case failing on MRI 2.3?

didn't test with all possible mri versions. will continue to test today, I'll come up with more results.

maybe this is old news, but I'll state them nonetheless: replacing the Celluloid::IO with a Celluloid actor seems to fix the issues. My bet is now on the evented mailbox.

I've been investigating option 3, which consistently happens in jruby. Something's wrong with the timeout...
I've added a few messages before the reactor selects, and also inside the jruby extension, before select is called on the NIO channel. Here's an example of a successful select:
# on celluloid/io/reactor.rb:59
selecting for 0.9849073000000006 seconds...
# on ext/nio4r/org/nio4r/Nio4r.java:291
timing out for<PHONE_NUMBER>000006 milliseconds...

Here's where it gets blocked:
selecting for 0.0009740119999985808 seconds...
timing out for 0.9740119999985808 milliseconds

The doSelect() method has a condition stating that if timeout is 0, one calls selectNow() on the NIO channel.
I've patched it by calling selectNow() when the timeout is  smaller than 1 millisecond, and it doesn't hang anymore:
/* inside doSelect() */
                        long tim = (long)(t * 1000);
                        if (tim < 1) {
                          result = this.selector.selectNow();
                        } else {
                          result = this.selector.select(tim);
                        }
...

Would this be an acceptable patch?
I think the real "problem" still comes from timer, as an interval triggering a message handling is probably so small that the java Runtime can't handle it properly. Or are java NIO channels not able to handle that kind of precision?

This could be an okay patch, but I have a request, could you print out values of tim where the bug happens? Is this bug also solved if you change the condition to (tim <= 0)?

@ioquatix I just did:
timing out for 0.9740119999985808 milliseconds

I'm digging a bit more, problem comes from Celluloid::Actor#run, particularly the wait block (line 157).
Timers is passing the wait_interval to the block, and this will be used to check a message from the mailbox. This wait_interval can be apparently very small!
Were select system calls ever able to deal with such an interval? Is this just the java way?

I wrote the code which generate those intervals, but it was a while ago. Yes, it's possible to generate small intervals.
Did you try with the condition I suggested?
Are you referring to this code in Nio4r?
https://github.com/celluloid/nio4r/blob/8cc48508a7cf3bb8497ba65f15febd691cf5d2a6/ext/nio4r/org/nio4r/Nio4r.java#L284-L291
That code is a bit different from what you quoted above. It uses double rather than long so avoids rounding issues, handles negative values correctly.

Can you link me to the code you are looking at w.r.t wait_interval, thanks :)

Just FYI, in some cases timers will provide a -ve wait interval if a timer should have fired already (it was late) https://github.com/celluloid/timers/blob/41145ed260e4f061c6b93604c5e0ba86192f37af/lib/timers/group.rb#L81-L90 but I'm not sure if this is used in the code you are talking about or not.

it casts it to a long before calling select, check a few lines below. That's what tim was, the long already converted to milliseconds value that the channel must interpret on the selec call .
I don't think anymore that the interval is the issue, as timers are supposed to be that precise, right? It's as if the NIO channels can't handle that low of an interval.
The code I meant: https://github.com/celluloid/celluloid/blob/master/lib/celluloid/actor.rb#L152-L156

When it gets cast to a long, perhaps the result is 0. In that case, some API have sleep indefinitely behaviour when called with 0 (or sometimes -1).

So you can see here exactly what I said:
https://docs.oracle.com/javase/7/docs/api/java/nio/channels/Selector.html#select(long)
timeout - If positive, block for up to timeout milliseconds, more or less, while waiting for a channel to become ready; if zero, block indefinitely; must not be negative
So if you call it with 0, clearly it will block forever :D

IF you call that API with a negative number, it will return the meaning of life.

The result is not 0, is the one that was printed out in the example above. Are you saying the channel will interpret it as 0 and block forever?

selecting for 0.0009740119999985808 seconds... timing out for 0.9740119999985808 milliseconds
If you cast 0.97 to an int, I'd expect truncation?

yes, but the API expects a long as argument. Unless it is truncating it inside. If so, the API is misleading as it doesn't accept positive in milliseconds, but > 1. I missed java...

result = this.selector.select((long)(t * 1000));
If t < 0.001 this will result in select being called with 0.

Actually, it blocks even with 0.1

Are you saying t = 0.1 or t * 1000 = 0.1? Because 0.1 * 1000 should result in select(100) which should definitely be okay.

t is already (long) milliseconds. so, 0.1 milliseconds.

Ok, I think this is java:
import java.io.IOException;
import java.nio.channels.Channel;

public class SelectorText {
  public static void main(String[] args) {
    System.out.println("starting the test...");
    java.nio.channels.Selector selector = null;
    try {
      selector = java.nio.channels.Selector.open();
      System.out.println("selecting now...");
      selector.selectNow();
      System.out.println("selecting later...");
      selector.select((long)2);
      System.out.println("selecting small...");
      selector.select((long)0.9999);
      System.out.println("that's all");

    }  catch(IOException ie) {
      System.out.println("balls");
    }

  }
}

try to run this.

Dude,
 selector.select((long)0.9999);

is the same as
 selector.select(0);

long x = (long)0.9999;

x will contain 0.

jesus, what a mental block... of course.

well, I guess it's settled then. I'll come up with a patch in nio4r.

sweet, let me know and I'll check it out.

I was reading long and thinking double...

@ioquatix , can you reopen this? option 1 still hanging...

Problem is here, wait_interval at some point is nil, and the selector will block. This is behaviour counts for MRI as well (tested in ruby 2.1 and 2.3). I think this time it's really timers...
When calling select with timeout set to nil, what is the expected behaviour? Both NIO as libev seem to have an inner threshold, as at some point I see the reactor calling back. But my timeout block is not retried anymore...

nil means there are no timers. So, in theory, wait indefinitely.

Yes we can certainly reopen this issue, what does @mailbox.check(interval) do if interval is nil?

Here is the non-everted mailbox:
https://github.com/celluloid/celluloid/blob/master/lib/celluloid/mailbox.rb#L48-L70
Timers::Wait.for(nil) loops indefinitely:
https://github.com/celluloid/timers/blob/41145ed260e4f061c6b93604c5e0ba86192f37af/lib/timers/wait.rb#L7-L17
Here is for evented mailbox:
https://github.com/celluloid/celluloid/blob/b19e19bf201787248e5b19e47ba99d1b7be6fd44/lib/celluloid/mailbox/evented.rb#L41-L54
Calling reactor.run_once(nil):
https://github.com/celluloid/celluloid-io/blob/cf249b9a1baf4e16e765a5c8e43f8531ceeb1319/lib/celluloid/io/reactor.rb#L56-L67
That code looks like it is designed to handle nil, but passes it to @selector.select(nil), which does seem to be handled correctly:

JRuby nio4r: https://github.com/celluloid/nio4r/blob/8cc48508a7cf3bb8497ba65f15febd691cf5d2a6/ext/nio4r/org/nio4r/Nio4r.java#L281-L283
MRI nio4r: https://github.com/celluloid/nio4r/blob/8cc48508a7cf3bb8497ba65f15febd691cf5d2a6/lib/nio/selector.rb#L61
C nio4r: https://github.com/celluloid/nio4r/blob/8cc48508a7cf3bb8497ba65f15febd691cf5d2a6/ext/nio4r/selector.c#L274-L290

It looks like C expects timeout to be non-nil which isn't consistent with the other implementations:
https://github.com/celluloid/nio4r/blob/8cc48508a7cf3bb8497ba65f15febd691cf5d2a6/ext/nio4r/selector.c#L282-L284
Not sure about libev, perhaps @tarcieri can comment on this since he wrote most of that code (?)

In theory, the behaviour of run_once(nil) should be to block indefinitely and run_once(t >= 0) should be to block for at most t seconds, and run_once(0) should check for events and return immediately.

humm, I don't know... when is it safe to run the reactor indefinitely? The main problem for me is, the nil is passed to the reactor while the timers are still handling a timeout. It seems that one is blocking the other from triggering.
The way I see it, either the timers should not yield nil, or the reactor should not be passed nil (which would be in accordance to what libev expects). I just don't understand where is the nil being yielded from. Does this mean there are no timers to process? I'm not as familiar with the timers code.

Yes, as explained, nil means no timers.

It seems that the timers group is not triggering the exceptions properly. Here is a bit more logging for you @ioquatix :
# good
...
successfull timeout fetch: 2016-01-29 14:35:35 +0100
timeout: adding timer: 2
timeout: timer added: 2 ; timers: 1

new timer: 1.9999205120000028
reactor inside: 1.9999205120000028
reactor outside: 1.9999205120000028
kaboom!!!!!!!!

successfull timeout fetch: 2016-01-29 14:35:37 +0100
...

# also good
...
successfull timeout fetch: 2016-01-29 14:35:33 +0100
timeout: adding timer: 2
timeout: timer added: 2 ; timers: 1

new timer: 1.9999047220000037
reactor inside: 1.9999047220000037
reactor outside: 1.9999047220000037

new timer: 0.00020313299999941137
reactor inside: 0.00020313299999941137
reactor outside: 0.00020313299999941137
kaboom!!!!!!!!

successfull timeout fetch: 2016-01-29 14:35:35 +0100
...

# bad
...
successfull timeout fetch: 2016-01-29 14:35:37 +0100
timeout: adding timer: 2
timeout: timer added: 2 ; timers: 1

new timer: 1.9999061400000002
reactor inside: 1.9999061400000002
reactor outside: 1.9999061400000002

new timer:
reactor inside:
reactor outside:

new timer:
mailbox: no message
reactor inside:
reactor outside:
.... (add eternum)

to add a bit of context, first block of logs is what happens when the timeout fails and the action is retried; (a new timeout is added to timers); the second block of logs is from the Actor#run method: new timer interval, interval passed to the reactor; "kaboom" is when the timeout timer event triggers.
So, the latter is not happening in our bug scenario. I don't know yet why, though, what I see is that, if the timer event associated with the timeout call doesn't trigger within the 2 second interval, it's all gone. Now, why is it not triggering? Next step.

Can you make a spec in timers for the failing behaviour.

still in the process of figuring out, I'll get to the failing scenario later.

@ioquatix problem is here: https://github.com/celluloid/timers/blob/master/lib/timers/group.rb#L66-L70. I've been following this and, in the normal case, you yield two intervals, fire the events in the end, and the timer get's cancelled after. You can see an example below:
successfull timeout fetch: 2016-01-29 15:42:59 +0100
Group has 1 events
Handle: #<Timers::Events::Handle:0x007fe97ad1cf70>
actor#run: interval 1.9999147540000024
Group has 1 events
Handle: #<Timers::Events::Handle:0x007fe97ad1cf70>
kaboom!!!!!!!! (timer.cancel from Celluloid::Actor)
successfull timeout fetch: 2016-01-29 15:43:01 +0100
...

Main feature is that the handle must be a valid (not yet cancelled one), which means the timer hasn't been triggered yet. In fact, the problem arrive as soon as the timer cancels in between the two calls on wait_interval there in the link:
successfull timeout fetch: 2016-01-29 15:43:03 +0100
Group has 1 events
Handle: #<Timers::Events::Handle:0x007fe97ad146b8>
actor: interval 1.999946142000006
(timer gets cancelled here, but the task is not resumable at this point)
Group has 1 events
Handle: #<Timers::Events::Handle:0x007fe97ad146b8> but it is cancelled!!!
Group has 0 events
actor: interval (nil)
# our problems start here
....

I can't investigate this further, at least for today and the weekend, but my the main issue is right there: you can't fire cancelled handles.

I need a failing test case or it's impossible for me to debug :) Do you have Google Hangouts?

Okay dude so I'm trying to understand the failure case you describe above, but it's not failing for me:
  it "should be okay to cancel during wait" do
    fired = false
    timer = subject.after(0.1) { puts "Fired!"; fired = true }

    first = true

    subject.wait do |interval|
      puts "Interval: #{interval}"

      if first
        timer.cancel
        first = false
      end

      break if interval < 0 or interval == nil
    end

    expect(fired).to be_falsey
  end

Please try to reproduce the issue in that spec.

@ioquatix no worries, I'll take the weekend to cool off, but I'll have time to come back to this issue next week, single-out exactly what it is causing this behaviour (sporadically, apparently) and write a meaningful spec. Sorry for the noise, just sharing info while getting to the bottom in the hope someone can have some input.

@ioquatix false alarm. Problem was that my test wasn't waiting for socket to be readable, and the blocking call stops on EOF. wait_readable returns nil in such a case, but EOFError is only triggered once you actually try to read from the socket... which my small test script wasn't doing, and so therein lies the impression that the timeout wasn't being properly triggered. I think that the main bug has already been fixed in nio4r.

We should probably make that nil is handled correctly by run_once though.

Ok, so let's just document here, so anyone come back to it:

JRuby: falls back to select(), waits indefinitely for events until wakeup is invoked or thread is interrupted.
MRI: uses IO.select, if timeout argument is nil, same thing.
libev: when passed nil for timeout it stops the loop timer and calls ev_loop with EVLOOP_ONESHOT , which according to the documentation,  "... will block your process until at least one new event arrives, and will return after one iteration of the loop."

It seems to me that they all handle nil more or less the same way.

Okay, awesome digging, thanks for providing a summary :+1:


Simplify use and specify link colours

Hey there! I've been using this template to update my own CV and it looks incredible! My Experience section goes on over the right column, and using grids with the left and right side specified separately makes it difficult to work that out.
I've converted this to instead use the columns layout, with a colbreak to indicate that it should move to the next column. This means that content will automatically flow from the left to the right column, even on additional pages!
I also tossed in a small change that colourises and italics links in the content, which has been useful for me since I'm mostly sending my CV out as PDFs.

Thanks for the PR! Using #colbreak() definitely makes more sense.


Extend role to use a google service account for authorization

What
The current implementation of this playbook only supports 2 of the 3 options for accessing storage buckets i.e.:

oauth2,
Interoperability keys - gcs_access_key and gcs_secret_key.

The purpose of this PR is to add support for the third option:

Using a google service account for authorization

Testing
We have tested this playbook with postiive and negative conditions to ensure there are adverse side effects.

Hi @aalda is this PR ok as it is or do you want me to make any changes in order to get it merged?

Sorry for the delay in replying. The pull request seems ok to me. Thanks!

@aalda No probs is it possible to do a 1.0.5 release?

Just pushed it to Galaxy :-)

@aalda Fantastic!


Can't upload files via MediaHubField

Hello,
I'm not sure if this is intended or not. But there is no way to upload files directly from the resource.
I'm using version 1.4.1 and this is what I see (https://imgur.com/a/VSpBP82)
Can anyone confirm if this is intended ?

That's weird. There should be Nova's file upload field on the left there. What Nova version are you using?

I'm using 4.11.0

Ok updated it to 4.19.3 and it shows the upload and dropzone.

Yeah, I think it's a discrepancy between Nova versions. I have to look into making it backwards compatible. Ie extarct the dropzone out of Nova just in case.

Thank you


Search suggestions not working

The search suggestions/autocomplete doesn't work at all.

No suggestions are shown when a user starts typing
Clicking on a "suggestion" does nothing
We probably shouldn't show the suggestions until a user begins typing

duplicate of #34