bug: the term 'appwrite' is not recognized as the name of a cmdlet

After installing the install script from the CLI without problem, it does not recognize appwrite as a command within the console. I am using PowerShell to run the command.
appwrite: The term 'appwrite' is not recognized as the name of a cmdlet, function, script file, or executable program. Check
if you typed the name correctly, or if you included a path, verify that the path is correct and try again.
Online: 1 Character: 1
+ appwrite version
+ ~~~~~~~~
     + CategoryInfo: ObjectNotFound: (appwrite: String) [], CommandNotFoundException
     + FullyQualifiedErrorId: CommandNotFoundException

@christyjacob4 @TorstenDittmann any ideas?

@yeikel16 Did this problem resolve when you restarted powershell?

It's not solved after restarting the console

n

@yeikel16 Did this problem resolve when you restarted powershell?

Sorry for the late reply, restarting the console didn't fix the problem.
To solve this problem you have to install the CLI by npm install -g appwrite-cli and not by iwr -useb https://appwrite.io/cli/install.ps1 | iex, because the latter causes such an error.
I close this issue for now.

perhaps take it out from the docs ? or fix the issue


Incorrect mIoU calculation?

Hi, thank you for the very well written code!
I have found an issue however, or actually two issues.
Firstly, when calculating mIoU, you add the sizes of intersections from all samples, and add the unions from all samples, and then divide the two. Wouldn't the correct way be to calculate IoU for each sample and then add these? These are not equal as the mean of quotients is not equal to the quotient of sums.
Secondly, when calculating the mean IoU over classes you include the classes that did not have any instances in either the prediction or the ground truth, i.e. where both the intersection and the union is zero. These will significantly lower the mIoU in a way it should not, right?
The first issue somewhat neutralises the second, as most (or all) classes will be represented when adding a lot of samples together. The result should still be erroneous though, which could be part of the reason for people not being able to recreate results from papers?
Please let me know if you agree with these issues, or if I have misunderstood something.

Hi, thanks for the question.
For the first issue, computing the mIoU over all examples and each one separately does give the same results, maybe I didn't understand your issue quite right.
For the second issue, you're right, at the beginning of training (or eval) the mIoU will not be correct, however, the final value (or after some seeing batches) will be since we'll have all of the classes.
And i I remember correctly, I did compare the mIoU results with the ones computed using chainer's eval_semantic_segmentation and I got the same results.

Thanks for the quick reply.
The included image illustrates what I mean with the quotients not being equal. After some investigation I have found that depending on the sizes of the individual IoU scores (i.e. the performance of the network) the scores differ in different ways. When individual IoUs on average are larger than 0.5, the right hand side in the inequality (your method) is larger, while when IoUs on average are smaller than 0.5, the left hand side is instead larger. (I tried this for many samples, not just two as in the inequality).
Which method is correct for calculating mIoU in semantic segmentation I am not sure of however. I think that averaging over IoUs feels more intuitive, but I have seen the method you use in several other implementations, so that may very well be the correct way.
For the second issue I agree that it is not important when calculating mIoU the way you do it.

Oh ok, I see what you mean, but I do think the correct way to do it is over the whole images, this way the results are more representative of the real performance of the model, where, if we summed the mIoUs of each image, some might be quite trivial (eg, majority of image is background or one class) and might slightly skew the results (they will be quite similar tho).

If you still have some doubts we can repoen the issue

If we separate (a+b)/(c+d) into (a/(c+d)) + (b/(c+d)), we can see that 'a' is divided also by 'd' to which is has no relation to (as it would come from a different image) and similarly for 'b' with 'c'. Also, as the metric is called 'mean' IoU, so the '0.5 * (a/c + b/d)' seems to the correct method.
Did you find an answer to this @vikolss ?

If we separate (a+b)/(c+d) into (a/(c+d)) + (b/(c+d)), we can see that 'a' is divided also by 'd' to which is has no relation to (as it would come from a different image) and similarly for 'b' with 'c'. Also, as the metric is called 'mean' IoU, so the '0.5 * (a/c + b/d)' seems to the correct method.
Did you find an answer to this @vikolss ?

In my opinion '0.5 * (a/c + b/d)' is a more intuitive way of calculating a mean intersection over union. But the way that it is done in this repository, (a+b)/(c+d), is the same way it is done in several other repositories that I have looked at. So I believe that that is the correct way to do it.

In my opinion '0.5 * (a/c + b/d)' is a more intuitive way of calculating a mean intersection over union. But the way that it is done in this repository, (a+b)/(c+d), is the same way it is done in several other repositories that I have looked at. So I believe that that is the correct way to do it.


Fix Exclude to not call MoveNext past end of sequence

Based on morelinq/morelinq#692, however this PR uses simpler evaluation of the enumeration; benchmarking shows that complex code is not actually faster.

exclude-bench.txt
UserQuery-20230122-204407.log.txt

Codecov Report
Base: 87.19% // Head: 87.19% // Decreases project coverage by -0.01% :warning:

Coverage data is based on head (df6d5c2) compared to base (6ce4733).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #145      +/-   ##
==========================================
- Coverage   87.19%   87.19%   -0.01%
==========================================
  Files         190      190
  Lines        5124     5122       -2
  Branches     1190     1189       -1
==========================================
- Hits         4468     4466       -2
  Misses        359      359
  Partials      297      297

Impacted Files
Coverage Δ

Source/SuperLinq/Exclude.cs
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Reregister

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/50219
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v1.0.2 -m "<description of version>" 25706abebc692fc1514f045568ceacbcecb1f427
git push origin v1.0.2


update cargo

This is all passing locally, there's an odd build artifact here that doesn't make sense.


Stalker Anomaly 1.5.0 update 3 glitched text

Some text gets glitched in following fashion:
https://files.catbox.moe/hnmab4.png
https://files.catbox.moe/ndjt29.png
https://files.catbox.moe/nkc632.png
When using these https://drive.google.com/drive/folders/1-gcaFB0nazU-ucUy0wycxIVmIL72Ogqs dlls inside Anomaly's /bin/ folder the problem was not happening.
Software information
Anomaly 1.5.0 beta update 3
System information

GPU: RX 5700 XT
Driver: DRM 3.35.0, 5.4.0-050400rc5-generic, LLVM 9.0.0, Mesa 20.0.0-devel (git-01cae57 2019-11-12 eoan-oibaf-ppa)
Wine version: 4.19 staging
DXVK version: 1.4.4

Apitrace file(s)
https://files.catbox.moe/or9ker.rdc
Renderdoc file. I was advised by someone to use renderdoc for Vulkan because Apitrace on linux is only for OpenGL. If it's not sufficient I'll try to wrestle with Apitrace. Unable to capture frame after loading or starting new game because I'm getting ring gfx_0.0.0 error each time (driver issue unrelated to DXVK).
Log files

d3d11.log: https://files.catbox.moe/z74qhb.log
dxgi.log: https://files.catbox.moe/fm6rz4.log

Renderdoc file. I was advised by someone to use renderdoc for Vulkan because Apitrace on linux is only for OpenGL. If it's not sufficient I'll try to wrestle with Apitrace. Unable to capture frame after loading or starting new game because I'm getting ring gfx_0.0.0 error each time (driver issue unrelated to DXVK).

You have to use the Windows version with Wine. More info here: https://github.com/Joshua-Ashton/d9vk/wiki/Making-a-Trace (I know it's a guide for D9VK but the steps are mostly the same except that you'll have to use d3d11.dll, dxgi.dll and dxgitrace.dll instead of d3d9.dll)

Renderdoc captures only work on the same GPU with the same driver, so sharing those is pretty much useless. I can't open it on my RX 480.

When using these https://drive.google.com/drive/folders/1-gcaFB0nazU-ucUy0wycxIVmIL72Ogqs dlls inside Anomaly's /bin/ folder the problem was not happening.

And what are those DLLs? If that's an older DXVK build, please just mention the version.

Works fine for me:

The one that has glitchless text is DXVK v1.0-7-gd011102. Author of that guide I linked told that he didn't exactly know what they are or where did he get them from so hell if I know.
Here's proof https://files.catbox.moe/pqs41s.png
Tried running it it a 1920x1080 and it's the same. I'll try getting the apitrace. It seems like Cyrillic script is interfering with English. It oddly looks similar to when icons for items inside inventory are misaligned but it was the same before I added any submods (and I only have HD models anyway)...

When I tried d3d11.dll from apitrace I couldn't run the game with library override but without it (or any library overrides for that matter) it did run, it didn't give me any problems with text but...
https://files.catbox.moe/di9tux.png (A-Life becoming self-aware lol)
Also this "DO NOT use DXVK together with apitrace!" will pose a problem. I don't think game will run in dx11 at all without DXVK. Before I found a way to do it with DXVK I could only get dx9 executable to work.

Just d3d11.dll isn't enough for apitrace. You need dxgi.dll and dxgitrace.dll too.
https://github.com/doitsujin/dxvk/wiki/Common-issues#apitrace

TL;DR: I REALLY DON'T THINK I CAN RUN IT IN DX11 WITHOUT DXVK
And more importantly even if I could the problem is triggered by something inside DXVK 1.4.4 libraries anyway so I don't know how would I trace it with apitrace libraries.
In a prefix that had DXVK on it with these 3 dlls from apitrace in /Anomaly/bin/ it's just the "terror not a morning" situation. With these 3 ddls and library overrides I get an error I didn't see before:
https://files.catbox.moe/xq8glq.png
With clean prefix without dll overrides I get standard "install dx11 you twat" prompt (https://files.catbox.moe/z7jzo4.png), with overrides I get yet another bug I didn't see:
https://files.catbox.moe/qg9t52.jpg
And, for the curious, the log reads:

Detected CPU: AMD Ryzen 5 3600 6-Core Processor               [AuthenticAMD], F15/M1/S0, 4199.00 mhz, 37-clk 'rdtsc'
CPU features: RDTSC, MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, HTT
CPU cores/threads: 6/12

Found commandline file!
Initializing File System...
using fs-ltx fsgame.ltx
File System Ready...
FS: 61333 files cached 47 archives, 992Kb memory used.
Init FileSystem 0.987728 sec
'xrCore' build 7541, Sep 29 2019
EH: 3CAABCFCFF6F3A810019C6A72180F166
-----loading c:/path/to/the/game/anomaly\gamedata\configs\system.ltx
-----loading c:/path/to/the/game/anomaly\gamedata\configs\system.ltx
Initializing Engine...
Starting INPUT device...
Loading DLL: xrRender_R4.dll
stack trace:
[error][       2] : File not found.

Those errors mention d3dx11. That's a helper library that interacts with d3d11. Make sure you're using the Microsoft version of that as the Wine one is broken. You can install that with Winetricks/Protontricks.

Could you point me to an exact d3d11 in Winetricks? Not sure which is which. I think I'm using d3d11_43.dll and as long as DXVK is used with it it works fine (aside from text errors). I get this error when I try to run Anomaly with DXVK but without d3d11 or d3dcompiler.

I am currently playing Anomaly and I don't have this issue; for reference these are what I have installed with winetricks:
d3dcompiler_42
d3dcompiler_43
d3dcompiler_47
d3dx10_43
d3dx10
d3dx11_42
d3dx11_43
I am on an Nvidia GTX860m

Just tried it in new prefix, just to be sure. Nothing changed. Without DXVK I got the "terror not a morning" situation, with DXVK (1.4.5) I had borked text.
I'm assuming it is something Navi 10 related but it's not just Navi 10 issue because with those dlls from the google folder I linked (1.0-7-gd011102) I was able to run it without the glitched text so the newer DXVK does something wrong.

So what exactly did you do to reproduce the problem? What exactly are your in-game settings?

Steps:

Using winetricks I create new 64 bit prefix, I choose "install a Windows  dll or component" and pick: d3dcompiler (tested with just _43 or _47 - no difference)
d3dx11_43 (don't remember if I tried 42 alone but when I installed every dll from post above there was no change)
dxvk
(normally I'd also add quartz for in-game movies but it didn't affect the problem)
Run explorer, choose AnomalyLaucher.exe
Choose DirectX 11 (R4), Shadow Map 2048 (all options same problem), everything else ticked off.

As for in-game settings I assume only Video tab is relevant:
https://files.catbox.moe/yfoni2.png
https://files.catbox.moe/ej7gdy.png
https://files.catbox.moe/if4gy4.png
https://files.catbox.moe/tpfuwg.png
https://files.catbox.moe/1zs0x4.png

So i followed your instructions but I still cannot reproduce the problem here:

Not sure what's going on but DXVK seems to be doing fine.

But you're using RX 480 right? As I said it's highly probable the problem is caused by something in Navi 10 but then again with the other dlls the problem simply wasn't there. I don't get it.
I understand I haven't given you any data you could work with, I just hoped maybe you'd miraculously remember or notice what was the difference between these dlls and current DXVK dlls affecting the text.

I've made a discovery.
Fonts work properly with dxvk 1.3.3/1.3.4 available through lutris but will bork out with anything higher than this.
There is something in 1.4 that causes problems with Navi 10 with oibaf drivers and it isn't only this game. Recently I had Nier: Automata start displaying weird colours in cracking sequences (like the normally gray map being yellow, reddish or green tinted) and missing shaders in Bunker area (lift leading to the command bridge being bright gray in contrast to dark gray all around it, npc clothes being brighter than they should be). It all worked nicely with 1.4.5 couple days ago but one of the recent patches caused every dxvk version past 1.4 to lack some shaders or display weird colours in cracking sequences. Importantly 1.3.3 and 1.3.4 don't have these problems.

Actually I was wrong about Nier Automata - every dxvk version causes some problems, it's just the tint difference in 1.3.3 and 1.3.4 was harder to notice. It's somewhat random too, I managed somehow to get 1.4.4 to work properly once and then it wen't back to heavy red backgrround.
Point being what's happening with Nier Automata is unrelated to what's happening with Anomaly.

Problem has been solved for me. I thought it was DXVK 1.5.1 but it works on older versions now as well. Runner version also didn't change anything in that regard. Something in recent drivers must've fixed it.


Invalid literal for int()

Since the update to the release 0.7.0 i am receiving the following errors from this integration:
Source: runner.py:128
First occurred: 27. Februar 2023 um 01:20:56 (53 occurrences)
Last logged: 28. Februar 2023 um 22:59:24

>> PioneerAVR._connection_listener() exception: invalid literal for int() with base 10: ''
>> PioneerAVR._connection_listener() exception: string index out of range

Despite these errors I did not find any problems with controlling my AVR.
I have installed the integration via HACS and configured my AVR in yaml, to increase the timeout (needed it for the previous release).

I have the same errors.

I have enabled logging as per instruction. I have reduced the log to only include lines referring to "pioneer".
The contents of the log don´t tell me much, I could just find the error will occure between sending ?VST to the AVR and receiving the response.
I hope this can help.
home-assistant_pioneer.log

The Reciever states the following firmware id: 1-151-081-111-031
I have the model VSX-921 (I think with "-k" suffix?).
I did manually update the firmware via USB at some point because of network issues I had. This was way after official update-servers were shutdown, I guess. The firmware file is named: AVR_12_0236.mem

I can't find any docs on the AVRs serial interface so not sure on the decoding of responses for VST and AST commands for AVRs manufactured <=2012. I've sent an email off to Pioneer in hopes they might be able to provide some docs, but I'm quite doubtful.
The immediate "workaround" to get rid of the errors is to disable the extended audio/video functionality for VSX-921 range of amplifiers. I don't think any of the new stuff has been exposed in HA yet anyway so this won't impact the functionality you currently have.

I have the same errors.

@spudje do you have the same model of AVR?

I can't find any docs on the AVRs serial interface so not sure on the decoding of responses for VST and AST commands for AVRs manufactured <=2012. I've sent an email off to Pioneer in hopes they might be able to provide some docs, but I'm quite doubtful.
The immediate "workaround" to get rid of the errors is to disable the extended audio/video functionality for VSX-921 range of amplifiers. I don't think any of the new stuff has been exposed in HA yet anyway so this won't impact the functionality you currently have.

Apparently my model is very similar to the later model version 1021k. They share the same hardware manual, and I think even firmware. Not sure if it helps maybe there is more docs for 1021k.

The closest I've got is the VSX-***2 models (such as VSX-922 or VSX-1022), which all say that the responses for the VST/AST commands are all 40 odd data bits long (with a number "reserved for future use"), whereas your AVR seems to only be returning around 20 data bits.
Thank you for confirming though, I've started some work to at least disable that functionality for now by default (although it can be overridden with the config parameters).

I can't find any docs on the AVRs serial interface so not sure on the decoding of responses for VST and AST commands for AVRs manufactured <=2012. I've sent an email off to Pioneer in hopes they might be able to provide some docs, but I'm quite doubtful.
The immediate "workaround" to get rid of the errors is to disable the extended audio/video functionality for VSX-921 range of amplifiers. I don't think any of the new stuff has been exposed in HA yet anyway so this won't impact the functionality you currently have.

?AST<CR>	Request AUDIO information	AST*<CR+LF>	        See "command list2" sheet.
?VST<CR>	Request VIDEO information	VST*<CR+LF>		See "command list2" sheet.
Command list 2

I can't find any docs on the AVRs serial interface so not sure on the decoding of responses for VST and AST commands for AVRs manufactured <=2012. I've sent an email off to Pioneer in hopes they might be able to provide some docs, but I'm quite doubtful.
The immediate "workaround" to get rid of the errors is to disable the extended audio/video functionality for VSX-921 range of amplifiers. I don't think any of the new stuff has been exposed in HA yet anyway so this won't impact the functionality you currently have.

?AST<CR> Request AUDIO information AST*<CR+LF> See "command list2" sheet. ?VST<CR> Request VIDEO information VST*<CR+LF> See "command list2" sheet.
Data to 53 digits 2014 Command list 2
Older version is data to 43 digits 2011 Command List 2

Thanks, thing is this is still incorrect for the vsx-921 as it doesn't return that many digits at all (around half)

I have the same error except it is on both ?VST and ?AST.  I have a VSX-925 (which does not response to the ?RGD)
home-assistant_2023-03-04T16-23-00.255Z.log
I think if there was a
if len(value) > 43:
at line 2420 for the remainder of the block and
if len(value) > 33:
at line 2573
It might fix it up.
Two questions:

how do I use the CLI for aiopioneer from within HA to try and test some things out and
where is the code for aiopioneer stored, because I can not find it to try editing it (or updating).

I have the same error except it is on both ?VST and ?AST. I have a VSX-925 (which does not response to the ?RGD)
home-assistant_2023-03-04T16-23-00.255Z.log
I think if there was a if len(value) > 43: at line 2420 for the remainder of the block and if len(value) > 33: at line 2573
It might fix it up.

I'll checkout the VSX-925, are you able to provide some logs so I can see a sample response for?

how do I use the CLI for aiopioneer from within HA to try and test some things out and

Not sure you can use the CLI directly from HA, as long as python3 is installed on a machine (Windows, Mac, Linux) you should be able to clone the repo to your machine and run the CLI directly (see the info here for more: )

where is the code for aiopioneer stored, because I can not find it to try editing it (or updating).

Do you mean on HA? If so it installs automatically when you create the integration, another one I'm not sure on, but I've actually forked the repo, and then updated the manifest.json for the custom integration to point directly to the git repository instead.

@D-Jeffrey can you test this on your AVR?
https://github.com/crowbarz/aiopioneer/pull/7
The parameters seem to actually all be the same as newer AVRs which is good.

@MahrWe This should be fixed with 0.7.2

Sorry for the late reply, I have a VSX-922.

I did the update today. Everything seems do work fine and I had no error-messages since the update.


update variant edit comment form to current design

This is actually OK in the current master, I think I was looking at the live version when I saw this.


Rustish lexer returns byte offsets in token positions instead of character offsets

All users of ParserContext in EdgeDB assume character offsets, whereas currently a byte offset is returned.

@tailhook, what's the status on this?  Anything left to do?

I think this is done


CoL+ to AG reference table issues

Just to help keep track of what still needs to be fixed in the references table export, here's the remaining issues:

[x] authors column is in JSON
[x] year column is not populated
[ ] source column is not populated

This has been fixed in https://github.com/Sp2000/colplus-backend/commit/f288f84f688999e1392a7cce55a346677d7509d8

The references.source column is still not populated in the latest export I generated a few minutes ago.


Add ability to match multiple broadcasts

def make_multiple_broadcasts
  ActionCable.server.broadcast "user_channel", message: "Hi"
  ActionCable.server.broadcast "user_channel", message: "Bye"
end

The above function broadcasts multiple times. Is there a way to test this with action-cable-testing, without calling make_multiple_broadcasts twice?

Do you use RSpec or Minitest?
With RSpec you should be able to combine expectations like this:
expect { subject }.to have_broadcasted_to("user_channel").with(message: "Hi").and
  have_broadcated_to("user_channel").with(message: "Bye")

With Minitest you should be able to nest #assert_broadcasts as well.

@palkan Thanks! The and worked for my use case.


Fix users old cards color

fix for users all cards that are red but should be orange. cause they can't cancel it

fixed by #1112

Fixed.
Now new and old transactions have:

red color when finished and can't be cancelled;
green color when withdraw;
orange color when still in progress and can be cancelled by sender (checked only with new transactions)


Always link rmf_traffic against the Threads target

https://github.com/open-rmf/rmf_demos/issues/132 reported that their machine failed to link rmf_traffic against pthread. In theory this PR should fix that problem.

I believe that this PR will solve one of the issues preventing rmf_traffic_examples from successfully building RPM packages for RHEL: https://build.ros2.org/view/Rbin_rhel_el864/job/Rbin_rhel_el864__rmf_traffic_examples__rhel_8_x86_64__binary/


Implement Remote Callbacks

Currently, all communications between the host app and plugins are oneway (host->plugin). Implement remote callbacks to support twoway communication. (Use PluginResponse for callbacks).

Added in commit 2412a3ae8dde65815a9ad4bfc450be473012be55.


[bypass staging-next] Merge pull request #157631 from ivanbrennan/re-add-alsa-lib-patch

Motivation for this change
A lot of people's audio is (partially) broken at the moment, and the fix for this is already in staging-next. This is a cherry-pick of that merge. I think this has high enough priority that a staging bypass is warranted.
See also:

https://github.com/NixOS/nixpkgs/pull/158080#issuecomment-1034101571
#157631

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg test pulseaudio.user

Previous check shows over 17k rebuilds.  We might as well merge staging-next as that's not much higher anymore.

I'm just worried that things are gonna stay broken while staging-next is figured out.

There are security fixes in staging-next that I'd prioritize over partially broken audio.

We might as well merge staging-next as that's not much higher anymore.

Can we throttle darwin builds on staging-next-21.11 to speed up staging-next a bit?

In any case you can also fix the NixOS module that makes use of this misfeature. libs.native is not an ALSA config and we should not patch ALSA to add it.

Do you know more precisely where the problem is, or would it be possible for you to make a PR?

I'm in favor of merging to staging-next, then just merging staging-next

It is already there as 3b139b96c54e56f.  This PR is an attempt to go even faster.

#158080 has beem merged.
All platforms but x86_64-darwin were in double digit rebuild counts.

I really need to figure out why matrix won't let me sign in so I'm not asking semi-unrelated questions on PRs but until then:
Are we missing x86_64 darwin builders? Seems even slower than usual and https://hydra.nixos.org/machines doesn't seem to list as many as I thought we had?

Yes, that's a temporary state.  (though I don't know any estimate)


❇ Added Infinite Scroll and Video API

[x] Fixes #190 #164
[x] #120
[x] tests passed
[x] Ensure all linting tests/pre-commit pass, see here for how to run them.
[x] Added video API
[x] Added infinite scroll
[x] Improved get_top_videos logic

Codecov Report

Merging #197 (679c6ce) into master (105c2bc) will decrease coverage by 0.13%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##           master     #197      +/-   ##
==========================================
- Coverage   86.13%   86.00%   -0.14%
==========================================
  Files          10       10
  Lines         642      650       +8
==========================================
+ Hits          553      559       +6
- Misses         89       91       +2

Impacted Files
Coverage Δ

crud/crud.py
79.76% <75.00%> (-0.36%)
:arrow_down:

main.py
76.08% <87.50%> (+0.19%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 105c2bc...679c6ce. Read the comment docs.


wast: incorrect parse in testsuite comments.wat

Simpler version of the failing test:
(module quote
  "(func (export \"f2\") (result i32)"
  "  (i32.const 1)"
  "  ;; comment\0d"
  "  (return (i32.const 2))"
  "\0a"
  ")"
)

(assert_return (invoke "f2") (i32.const 2))

The QuoteModule for this function is missing one of the i32.consts:
[crates/spec-test/src/spec.rs:240] &module = [
    ...
    10, // code section
    6,  // size = 6
    1,  // number of entries = 1
    4,  // size = 4
    0,  // 0 locals
    65, // i32.const
    1,  // 1
    11, // end
]

In the full test there are three copies of this function with comments in different places. The other two are parsed as expected.

Thanks for the report! I think this should be fixed in https://github.com/bytecodealliance/wasm-tools/pull/1372


first accessibility updates

First updates for accessibility improvements based on the aXe extension: https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd

I think there is a more "react" way to handle this! Very nice tool
https://reactjs.org/docs/accessibility.html

This might help in spotting them directly in code:
https://reactjs.org/docs/accessibility.html#eslint-plugin-jsx-a11y

Was going to add eslint-plugin-jsx-a11y in a later update, but I did it now.
Each tool that I use detects different issues with the UI. jsx-a11y seems happy with the changes recommended by the aXe extension, but raises various other possible design issues with the UI.

Was going to add eslint-plugin-jsx-a11y in a later update, but I did it now.
Each tool that I use detects different issues with the UI. jsx-a11y seems happy with the changes recommended by the aXe extension, but raises various other possible design issues with the UI.

Yeah that's expected.
I just feel like the hints from source code will be more react oriented than analyzing the built bundle that goes through transformations

Was going to add eslint-plugin-jsx-a11y in a later update, but I did it now.
Each tool that I use detects different issues with the UI. jsx-a11y seems happy with the changes recommended by the aXe extension, but raises various other possible design issues with the UI.

Yeah that's expected. I just feel like the hints from source code will be more react oriented than analyzing the built bundle that goes through transformations

Yeah, agreed, But jsx-a11y is not enough. The other tools raise issues with navigation, the minting form, and with visual issues that the plugin can't find.

jsx-a11y is not enough.

I'm no accessibility expert, but maybe we should prfioritize from what is the most needed for users?
I'm asking SurfingDarkness is she has inputs on the most critical issues

Linking here Synk's write up: https://www.notion.so/teia-community/Accessibility-UI-UX-87d2598ed49b49b785e904d0ec56ef61

Thanks. That's why I started with the aXe extension since it verifies issues from a user's perspective.
It does look like even after using various tools, it is going to require feedback from users since it feels like I'm fixing technical issues and not yet all the usability issues.


Remove --allow-unauthenticated from apt-get call.

Adds the public key matching the docker-engine package, and authenticates the package during installation.
Addresses #173

I don't think this fixes the problem with installing Docker package. Have you tested this?

The following worked on a fresh Ubuntu 16.04 VM:
$ sudo add-apt-repository "deb https://apt.dockerproject.org/repo ubuntu-xenial main"
$ sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com F76221572C52609D
$ sudo apt-get update
$ sudo apt-get install docker-engine
< docker-engine installs with no errors >

In the issue you mentioned in #173, someone pointed out that they also got this sequence of commands to work, although other sequences did not. Also, this repository's tests passed.

Also, while we're at it, thoughts on installing apt-transport-https and ca-certificates as well?

The entire aws-ec2-terraform folder appears to have been removed from the repo, so I'm closing this PR and its underlying issue.


allowed to remove jwt auth header by setting it to null

Delete jwt header by setting it to null.
api.jwt('my token')
// this.headers.Authorization = `Bearer ${token}`;
api.jwt(null);
// this.headers.Authorization deleted

See v1.2.0 published to NPM, thank you for your PR @sampsasaarela - npm install --save frisbee@latest


Can't capture screen shot

Hi, when I try to capture screenshot it show an error like this
[1] Error occurred in handler for 'captureScreenshot': TypeError: Error processing argument at index 2, conversion failure from
[1]     at node:electron/js2c/browser_init:177:945
[1]     at new Promise (<anonymous>)
[1]     at Object.t.getSourcesImpl (node:electron/js2c/browser_init:177:697)
[1]     at Object.getSources (node:electron/js2c/browser_init:45:277)
[1]     at C:\Users\LENOVO\Desktop\ImageDetect\TranslateManga\VGT-master\electron\handlers\ipcHandler.js:138:45
[1]     at node:electron/js2c/browser_init:201:579
[1]     at Object.<anonymous> (node:electron/js2c/browser_init:165:10272)
[1]     at Object.emit (node:events:394:28)

can you help me

I'm working on trying to replicate your error
Few questions:
Does the error occur when using the compiled version (.exe) or running the source code?
Is the error consistent? Does it continue to happen even if you restart the software or try to take other screenshots?

I'm working on trying to replicate your error
Few questions:
Does the error occur when using the compiled version (.exe) or running the source code?
Is the error consistent? Does it continue to happen even if you restart the software or try to take other screenshots?

I've started getting this issue consistently since June 1st, when I tried to pull my fork down to a new environment. It's happened both on Linux and Windows, though it was working in both prior to some undetermined point after which I haven't been able to capture another screenshot.
[1] Error occurred in handler for 'captureScreenshot': TypeError: Error processing argument at index 2, conversion failure from
[1]     at node:electron/js2c/browser_init:177:945
[1]     at new Promise (<anonymous>)
[1]     at Object.t.getSourcesImpl (node:electron/js2c/browser_init:177:697)
[1]     at Object.getSources (node:electron/js2c/browser_init:45:277)
[1]     at C:\Users\Me\workspace\ORIGINAL-VGT\electron\handlers\ipcHandler.js:143:45
[1]     at node:electron/js2c/browser_init:201:579
[1]     at Object.<anonymous> (node:electron/js2c/browser_init:165:10272)
[1]     at Object.emit (node:events:394:28)

Here's a link to my package-lock.json: https://gist.github.com/rDarge/42b2a709d37bfd2b3052d72a1ce7237e
I was actually trying to boot up a version of VGT I had working so I could add screenshots for some of the issues I was wanted to make, but wasn't able to actually take a screenshot.

It was a huge pain troubleshooting this, but I think I've actually figured it out. If you supply a number with a decimal point in one of the parameters for thumbnailSize in desktopCapturer.getSources, you will hit this error. I'm able to avoid it by applying a Math.floor operation to both screenWidth and screenHeight in ipcHandler.js in the captureScreenshot handler.
Here are some logs I pulled from testing this, and a fix I've put in place.
(manga-ocr-fast-api-py3.9) PS C:\Users\Ryan\workspace\VGT> npm run electron-dev

><EMAIL_ADDRESS>electron-dev
> cross-env NODE_ENV=development concurrently "webpack serve --config webpack.dev.config.js"  "electron ."

[1]
[1] INFO:     Will watch for changes in these directories: ['C:\\Users\\Ryan\\workspace\\VGT']
[1]
[1] INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
[1] INFO:     Started reloader process [11556] using WatchFiles
[1]
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] Loopback: http://localhost:3000/
[0] <i> [webpack-dev-server] On Your Network (IPv4): http://<IP_ADDRESS>:3000/
[0] <i> [webpack-dev-server] Content not from webpack is served from 'C:\Users\Ryan\workspace\VGT\dist' directory
[1] INFO:     Started server process [11112]
[1] INFO:     Waiting for application startup.
[1]
[1] INFO:     Application startup complete.
[1]
[1] INFO:     <IP_ADDRESS>:58653 - "GET /check HTTP/1.1" 200 OK
[1]
[1] INFO:     <IP_ADDRESS>:58654 - "GET /modelCheck HTTP/1.1" 200 OK
[1]
[1] 2023-06-03 12:03:58.237 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
[1]
[1] C:\Users\Ryan\AppData\Local\pypoetry\Cache\virtualenvs\manga-ocr-fast-api-PMbG7k51-py3.9\lib\site-packages\transformers\models\vit\feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead.
[1]   warnings.warn(
[1]
[0] <i> [webpack-dev-middleware] wait until bundle finished: /index.html
[1] 2023-06-03 12:04:01.017 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
[1]
[1] 2023-06-03 12:04:01.751 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
[1]
[1] loadMangaOCR
[1] INFO:     <IP_ADDRESS>:58655 - "POST /loadMangaOCR HTTP/1.1" 200 OK
[1]
[0] asset bundle.js 9.13 MiB [emitted] (name: main)
[0] asset index.html 233 bytes [emitted]
[0] orphan modules 3.01 MiB [orphan] 2041 modules
[0] runtime modules 28.9 KiB 16 modules
[0] built modules 3.06 MiB [built]
[0]   modules by path ./node_modules/ 2.95 MiB 606 modules
[0]   modules by path ./react/ 114 KiB
[0]     modules by path ./react/components/*.js 110 KiB
[0]       ./react/components/App.js 17.1 KiB [built] [code generated]
[0]       ./react/components/Main.js 4.64 KiB [built] [code generated]
[0]       + 8 modules
[0]     modules by path ./react/*.css 3.65 KiB
[0]       ./react/App.css 2.23 KiB [built] [code generated]
[0]       ./node_modules/css-loader/dist/cjs.js!./react/App.css 1.42 KiB [built] [code generated]
[0]     ./react/index.js 330 bytes [built] [code generated]
[0]   external "events" 42 bytes [built] [code generated]
[0]   external "electron" 42 bytes [built] [code generated]
[0] webpack 5.76.2 compiled successfully in 6251 ms
[1] 2256 1504.5
[1] Error occurred in handler for 'captureScreenshot': TypeError: Error processing argument at index 2, conversion failure from
[1]     at node:electron/js2c/browser_init:177:945
[1]     at new Promise (<anonymous>)
[1]     at Object.t.getSourcesImpl (node:electron/js2c/browser_init:177:697)
[1]     at Object.getSources (node:electron/js2c/browser_init:45:277)
[1]     at C:\Users\Ryan\workspace\VGT\electron\handlers\ipcHandler.js:144:45
[1]     at node:electron/js2c/browser_init:201:579
[1]     at Object.<anonymous> (node:electron/js2c/browser_init:165:10272)
[1]     at Object.emit (node:events:394:28)
Terminate batch job (Y/N)? [0] <i> [webpack-dev-server] Gracefully shutting down. To force exit, press ^C again. Please wait...
[1] C:\Users\Ryan\workspace\VGT\node_modules\electron\dist\electron.exe exited with signal SIGINT
[1] electron . exited with code 1
[0] webpack serve --config webpack.dev.config.js exited with code 0

Note the following line, which is a console log of screenWidth and screenHeight, respectively:

[1] 2256 1504.5

In ipcHandler.js I made the following changes to lines 138-140:
      const screenWidth = Math.floor(screenDetails.size.width * screenDetails.scaleFactor);
      const screenHeight =
        Math.floor(screenDetails.size.height * screenDetails.scaleFactor)

After applying this change I was able to take a screenshot again.
(manga-ocr-fast-api-py3.9) PS C:\Users\Ryan\workspace\VGT> npm run electron-dev

><EMAIL_ADDRESS>electron-dev
> cross-env NODE_ENV=development concurrently "webpack serve --config webpack.dev.config.js"  "electron ."

[1]
[1] INFO:     Will watch for changes in these directories: ['C:\\Users\\Ryan\\workspace\\VGT']
[1]
[1] INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
[1] INFO:     Started reloader process [2148] using WatchFiles
[1]
[0] <i> [webpack-dev-server] Project is running at:
[0] <i> [webpack-dev-server] Loopback: http://localhost:3000/
[0] <i> [webpack-dev-server] On Your Network (IPv4): http://<IP_ADDRESS>:3000/
[0] <i> [webpack-dev-server] Content not from webpack is served from 'C:\Users\Ryan\workspace\VGT\dist' directory
[1] INFO:     Started server process [15656]
[1] INFO:     Waiting for application startup.
[1]
[1] INFO:     Application startup complete.
[1]
[1] INFO:     <IP_ADDRESS>:53409 - "GET /check HTTP/1.1" 200 OK
[1]
[1] INFO:     <IP_ADDRESS>:53410 - "GET /modelCheck HTTP/1.1" 200 OK
[1]
[1] 2023-06-03 12:05:07.255 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
[1]
[1] C:\Users\Ryan\AppData\Local\pypoetry\Cache\virtualenvs\manga-ocr-fast-api-PMbG7k51-py3.9\lib\site-packages\transformers\models\vit\feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead.
[1]   warnings.warn(
[1]
[0] <i> [webpack-dev-middleware] wait until bundle finished: /index.html
[1] 2023-06-03 12:05:10.143 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
[1]
[1] 2023-06-03 12:05:10.797 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
[1]
[1] loadMangaOCR
[1] INFO:     <IP_ADDRESS>:53411 - "POST /loadMangaOCR HTTP/1.1" 200 OK
[1]
[0] asset bundle.js 9.13 MiB [emitted] (name: main)
[0] asset index.html 233 bytes [emitted]
[0] orphan modules 3.01 MiB [orphan] 2041 modules
[0] runtime modules 28.9 KiB 16 modules
[0] built modules 3.06 MiB [built]
[0]   modules by path ./node_modules/ 2.95 MiB 606 modules
[0]   modules by path ./react/ 114 KiB
[0]     modules by path ./react/components/*.js 110 KiB
[0]       ./react/components/App.js 17.1 KiB [built] [code generated]
[0]       ./react/components/Main.js 4.64 KiB [built] [code generated]
[0]       + 8 modules
[0]     modules by path ./react/*.css 3.65 KiB
[0]       ./react/App.css 2.23 KiB [built] [code generated]
[0]       ./node_modules/css-loader/dist/cjs.js!./react/App.css 1.42 KiB [built] [code generated]
[0]     ./react/index.js 330 bytes [built] [code generated]
[0]   external "events" 42 bytes [built] [code generated]
[0]   external "electron" 42 bytes [built] [code generated]
[0] webpack 5.76.2 compiled successfully in 6233 ms
[1] 2256 1504
[1] Guardando nueva imagen
[1] Guardando texto detectado
[1] INFO:     <IP_ADDRESS>:64460 - "POST /translateDataUrlImg HTTP/1.1" 200 OK
[1]
[1] INFO:     <IP_ADDRESS>:64463 - "POST /translateText HTTP/1.1" 200 OK
[1]
[1] Guardando texto traducido

screenWidth and screenHeight are even numbers now.

[1] 2256 1504

I'll dig into this a bit more so I understand the implications of fractional screen width/height calculations, but as a stopgap fix flooring these numbers has worked fine for me, with no other issues.

Well, maybe there's something else going on here... on another test rounding down didn't work.

Good job finding that bug! Yes, I think that must be it. I never tested the system with operating systems that are using additional scaling.
I will check that everything continues to work correctly and I accept your PR.
I have never tested the program on Linux. So, as you mentioned, you were able to run it correctly, right? It would be great to make an installable version.
Thank you very much for your work

No problem. With some tweaks I was able to run it on linux, and OSX as well, but I don't have immediate access to an OSX build environment currently. I'll open up another issue for linux compatibility and prepare a PR for that soon.


In flashgames.DuskDrive-v0 mouse jumps for some time and then environment crashes

I seem to have been able to install all the dependencies and run the following code:
import gym
import universe # register the universe environments

env = gym.make('flashgames.DuskDrive-v0')
env.configure(remotes=1) # automatically creates a local docker container
observation_n = env.reset()

while True:
  action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n] # your agent here
  observation_n, reward_n, done_n, info = env.step(action_n)
  env.render()

however the result that I see is that the mouse pointer jumps in the rendered environment, and after some time the environment crashes:
https://www.youtube.com/watch?v=8WDDzbGVJ3o
the output that I get in console is as follows:
[2016-12-05 13:48:17,797] Making new env: flashgames.DuskDrive-v0
[2016-12-05 13:48:17,800] Writing logs to file: /tmp/universe-4612.log
[2016-12-05 13:48:17,809] [0] Creating container: image=quay.io/openai/universe.flashgames:0.20.1. Run the same thing by hand as: docker run -p 5900:5900 -p 15900:15900 --cap-add SYS_ADMIN --ipc host --privileged quay.io/openai/universe.flashgames:0.20.1
[2016-12-05 13:48:18,060] Remote closed: address=localhost:5900
[2016-12-05 13:48:18,062] At least one sockets was closed by the remote. Sleeping 1s...
universe-psnd10-0 | Setting VNC and rewarder password: openai
universe-psnd10-0 | [Mon Dec  5 12:48:18 UTC 2016] Waiting for /tmp/.X11-unix/X0 to be created (try 1/10)
universe-psnd10-0 | [Mon Dec  5 12:48:18 UTC 2016] [/usr/local/bin/sudoable-env-setup] Disabling outbound network traffic for none
universe-psnd10-0 | [init] [2016-12-05 12:48:18,155] PID 55 launched with command ['sudo', '-H', '-u', 'nobody', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=/dev/null', '/app/universe-envs/controlplane/bin/controlplane.py', '--rewarder-port=15901']
universe-psnd10-0 | [init] [2016-12-05 12:48:18,156] init detected end of child process 23 with exit code 0, not killed by signal
universe-psnd10-0 | WebSocket server settings:
universe-psnd10-0 |   - Listen on :5898
universe-psnd10-0 |   - Flash security policy server
universe-psnd10-0 |   - No SSL/TLS support (no cert file)
universe-psnd10-0 |   - proxying from :5898 to localhost:5900
universe-psnd10-0 | [2016-12-05 12:48:18,710] [INFO:root] Starting play_controlplane.py with the following: command=['/app/universe-envs/controlplane/bin/controlplane.py', '--rewarder-port=15901'] args=Namespace(bot_demonstration=False, demonstration=False, env_id=None, idle_timeout=None, integrator_mode=False, no_env=False, no_rewarder=False, no_scorer=False, no_vexpect=False, remotes='vnc://<IP_ADDRESS>:5900', rewarder_fps=60, rewarder_port=15901, verbosity=0) env=environ({'SUDO_USER': 'root', 'DBUS_SESSION_BUS_ADDRESS': '/dev/null', 'USER': 'nobody', 'USERNAME': 'nobody', 'SUDO_COMMAND': '/app/universe-envs/controlplane/bin/controlplane.py --rewarder-port=15901', 'TERM': 'xterm', 'SHELL': '/usr/sbin/nologin', 'HOME': '/nonexistent', 'LOGNAME': 'nobody', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'SUDO_UID': '0', 'HOSTNAME': '657c3c28301b', 'SUDO_GID': '0', 'MAIL': '/var/mail/nobody', 'DISPLAY': ':0'})
universe-psnd10-0 | [2016-12-05 12:48:18,710] [INFO:root] [EnvStatus] Changing env_state: None (env_id=None) -> None (env_id=None) (episode_id: 0->0, fps=60)
universe-psnd10-0 | [2016-12-05 12:48:18,710] [INFO:universe.rewarder.remote] Starting Rewarder on port=15901
universe-psnd10-0 | [2016-12-05 12:48:18,713] [INFO:universe.extra.universe.wrappers.logger] Running VNC environments with Logger set to print_frequency=5. To change this, pass "print_frequency=k" or "print_frequency=None" to "env.configure".
universe-psnd10-0 | [2016-12-05 12:48:18,715] [INFO:universe.envs.vnc_env] Using the golang VNC implementation
universe-psnd10-0 | [2016-12-05 12:48:18,715] [INFO:universe.envs.vnc_env] Using VNCSession arguments: {'encoding': 'zrle', 'start_timeout': 7, 'fine_quality_level': 50, 'subsample_level': 2, 'compress_level': 9}. (Customize by running "env.configure(vnc_kwargs={...})"
universe-psnd10-0 | [2016-12-05 12:48:18,716] [INFO:universe.envs.vnc_env] Printed stats will ignore clock skew. (This usually makes sense only when the environment and agent are on the same machine.)
universe-psnd10-0 | [2016-12-05 12:48:18,719] [INFO:universe.envs.vnc_env] [0] Connecting to environment: vnc://<IP_ADDRESS>:5900 password=openai. Use the browser-based VNC client: http://None/viewer/?password=openai
universe-psnd10-0 | [2016-12-05 12:48:18,720] [INFO:universe.extra.universe.envs.vnc_env] [0] Connecting to environment details: vnc_address=<IP_ADDRESS>:5900 vnc_password=openai rewarder_address=None rewarder_password=openai
universe-psnd10-0 | 2016/12/05 12:48:18 I1205 12:48:18.720314 56 gymvnc.go:417] [0:<IP_ADDRESS>:5900] opening connection to VNC server
universe-psnd10-0 | [2016-12-05 12:48:18,720] [INFO:root] [EnvStatus] Changing env_state: None (env_id=None) -> resetting (env_id=None) (episode_id: 0->1, fps=60)
universe-psnd10-0 | [2016-12-05 12:48:18,720] [INFO:root] [MainThread] Env state: env_id=None episode_id=1
universe-psnd10-0 | 2016/12/05 12:48:18 I1205 12:48:18.724959 56 gymvnc.go:550] [0:<IP_ADDRESS>:5900] connection established
universe-psnd10-0 | [Mon Dec  5 12:48:18 UTC 2016] [/usr/local/bin/sudoable-env-setup] Disabling outbound network traffic for none
universe-psnd10-0 | [2016-12-05 12:48:18,745] [INFO:gym_flashgames.launcher] [MainThread] Launching new Chrome process (attempt 0/10)
universe-psnd10-0 | [2016-12-05 12:48:18,746] [INFO:root] Replacing selenium_wrapper_server since we currently do it at every episode boundary
universe-psnd10-0 | [2016-12-05 12:48:18,857] [selenium_wrapper_server] Calling webdriver.Chrome()
[2016-12-05 13:48:19,064] Remote closed: address=localhost:5900
[2016-12-05 13:48:19,064] Remote closed: address=localhost:15900
[2016-12-05 13:48:19,064] At least one sockets was closed by the remote. Sleeping 1s...
universe-psnd10-0 | [2016-12-05 12:48:19,067] [INFO:universe.rewarder.remote] Client connecting: peer=tcp4:<IP_ADDRESS>:60428 observer=True
universe-psnd10-0 | [2016-12-05 12:48:19,067] [INFO:universe.rewarder.remote] WebSocket connection established
universe-psnd10-0 | [init] [2016-12-05 12:48:19,129] init detected end of child process 17 with exit code 0, not killed by signal
[2016-12-05 13:48:20,065] Using the golang VNC implementation
[2016-12-05 13:48:20,065] Using VNCSession arguments: {'subsample_level': 2, 'start_timeout': 7, 'fine_quality_level': 50, 'encoding': 'tight'}. (Customize by running "env.configure(vnc_kwargs={...})"
[2016-12-05 13:48:20,068] [0] Connecting to environment: vnc://localhost:5900 password=openai. Use the browser-based VNC client: http://localhost:15900/viewer/?password=openai
2016/12/05 13:48:20 I1205 13:48:20.069146 4612 gymvnc.go:417] [0:localhost:5900] opening connection to VNC server
2016/12/05 13:48:20 I1205 13:48:20.071467 4612 gymvnc.go:550] [0:localhost:5900] connection established
universe-psnd10-0 | [2016-12-05 12:48:20,066] [INFO:universe.rewarder.remote] WebSocket connection closed: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
universe-psnd10-0 | [2016-12-05 12:48:20,080] [INFO:universe.rewarder.remote] Client connecting: peer=tcp4:<IP_ADDRESS>:60468 observer=False
universe-psnd10-0 | [2016-12-05 12:48:20,080] [INFO:universe.rewarder.remote] WebSocket connection established
universe-psnd10-0 | [2016-12-05 12:48:20,131] [selenium_wrapper_server] Call to webdriver.Chrome() completed: 1.27s
universe-psnd10-0 | [2016-12-05 12:48:20,131] [INFO:gym_flashgames.launcher] [MainThread] Navigating browser to url=http://localhost
universe-psnd10-0 | [2016-12-05 12:48:20,375] [INFO:root] [EnvStatus] Changing env_state: resetting (env_id=None) -> running (env_id=None) (episode_id: 1->1, fps=60)
universe-psnd10-0 | Manhole[1480942100.3775]: Patched <built-in function fork> and <built-in function fork>.
universe-psnd10-0 | Manhole[1480942100.3777]: Manhole UDS path: /tmp/manhole-56
universe-psnd10-0 | Manhole[1480942100.3778]: Waiting for new connection (in pid:56) ...
universe-psnd10-0 | [2016-12-05 12:48:23,727] [INFO:universe.wrappers.logger] Stats for the past 5.01s: vnc_updates_ps=2.2 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=359679.6 vnc_pixels_ps[total]=532589.2 reward_lag=None rewarder_message_lag=None fps=40
universe-psnd10-0 | [2016-12-05 12:48:25,377] [INFO:universe.pyprofile] [pyprofile] period=5.00s timers={"rewarder.compute_reward": {"mean": "214.29us", "calls": 301, "std": "284.49us"}, "vnc_env.VNCEnv.vnc_session.step": {"mean": "86.38us", "calls": 301, "std": "82.80us"}, "rewarder.frame": {"mean": "16.77ms", "calls": 300, "std": "35.17us"}, "rewarder.sleep": {"mean": "16.30ms", "calls": 300, "std": "283.67us"}} counters={} gauges={} (export_time=92.27us)
universe-psnd10-0 | [2016-12-05 12:48:25,377] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.00s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.n': None, 'rewarder.vnc.updates.pixels': None, 'rewarder.profile': '<606 bytes>', 'rewarder.vnc.updates.bytes': None}
universe-psnd10-0 | [2016-12-05 12:48:28,744] [INFO:universe.wrappers.logger] Stats for the past 5.02s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=60
universe-psnd10-0 | [2016-12-05 12:48:30,393] [INFO:universe.pyprofile] [pyprofile] period=5.02s timers={"rewarder.compute_reward": {"mean": "195.87us", "calls": 301, "std": "64.43us"}, "vnc_env.VNCEnv.vnc_session.step": {"mean": "81.01us", "calls": 301, "std": "31.80us"}, "rewarder.frame": {"mean": "16.76ms", "calls": 301, "std": "34.13us"}, "rewarder.sleep": {"mean": "16.32ms", "calls": 301, "std": "93.57us"}} counters={"agent_conn.reward": {"std": 0, "calls": 1, "mean": 0.0}} gauges={} (export_time=93.46us)
universe-psnd10-0 | [2016-12-05 12:48:30,393] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.02s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.n': None, 'rewarder.vnc.updates.pixels': None, 'rewarder.profile': '<677 bytes>', 'rewarder.vnc.updates.bytes': None}
universe-psnd10-0 | [2016-12-05 12:48:33,760] [INFO:universe.wrappers.logger] Stats for the past 5.02s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=60
universe-psnd10-0 | [2016-12-05 12:48:35,393] [INFO:universe.pyprofile] [pyprofile] period=5.00s timers={"rewarder.compute_reward": {"mean": "187.30us", "calls": 300, "std": "60.73us"}, "vnc_env.VNCEnv.vnc_session.step": {"mean": "77.51us", "calls": 300, "std": "30.43us"}, "rewarder.frame": {"mean": "16.76ms", "calls": 300, "std": "32.75us"}, "rewarder.sleep": {"mean": "16.34ms", "calls": 300, "std": "85.00us"}} counters={"agent_conn.reward": {"std": 0, "calls": 1, "mean": 0.0}} gauges={} (export_time=87.98us)
universe-psnd10-0 | [2016-12-05 12:48:35,393] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.00s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={'rewarder.vnc.updates.n': None, 'rewarder.vnc.updates.pixels': None, 'rewarder.profile': '<673 bytes>', 'rewarder.vnc.updates.bytes': None}
universe-psnd10-0 | [2016-12-05 12:48:38,777] [INFO:universe.wrappers.logger] Stats for the past 5.02s: vnc_updates_ps=0.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=0.0 vnc_pixels_ps[total]=0.0 reward_lag=None rewarder_message_lag=None fps=60
[2016-12-05 13:48:40,089] [0:localhost:5900] ntpdate -q -p 8 localhost call timed out after 20.0s; killing the subprocess. This is ok, but you could have more accurate timings by enabling UDP port 123 traffic to your env. (Alternatively, you can try increasing the timeout by setting environment variable UNIVERSE_NTPDATE_TIMEOUT=10.)
[2016-12-05 13:48:40,109] [0:localhost:5900] Sending reset for env_id=flashgames.DuskDrive-v0 fps=60 episode_id=0
universe-psnd10-0 | [2016-12-05 12:48:40,098] [INFO:universe.rewarder.remote] CONNECTION STATUS: Marking connection as active: observer=False peer=tcp4:<IP_ADDRESS>:60468 total_conns=1
universe-psnd10-0 | [2016-12-05 12:48:40,109] [INFO:universe.rewarder.remote] Received reset message: {'body': {'seed': None, 'env_id': 'flashgames.DuskDrive-v0', 'fps': 60}, 'method': 'v0.env.reset', 'headers': {'sent_at':<PHONE_NUMBER>.109446, 'message_id': 10, 'episode_id': '0'}}
universe-psnd10-0 | [2016-12-05 12:48:40,117] [INFO:root] [EnvStatus] Changing env_state: running (env_id=None) -> resetting (env_id=flashgames.DuskDrive-v0) (episode_id: 1->2, fps=60)
universe-psnd10-0 | [2016-12-05 12:48:40,117] [ERROR:root] Closing server (via subprocess.close()) and all chromes (via pkill chromedriver || :; pkill chrome || :)
universe-psnd10-0 | [2016-12-05 12:48:40,126] [INFO:root] [Rewarder] Blocking until env finishes resetting
universe-psnd10-0 | [init] [2016-12-05 12:48:40,128] init detected end of child process 96 with exit code 0, killed by SIGTERM: 15
universe-psnd10-0 | [2016-12-05 12:48:40,129] [INFO:root] [EnvController] RESET CAUSE: changing out environments due to v0.env.reset (with episode_id=0): flashgames.DuskDrive-v0 -> flashgames.DuskDrive-v0 (new episode_id=2 fps=60)
universe-psnd10-0 | [2016-12-05 12:48:40,130] [INFO:root] [EnvController] Env state: env_id=flashgames.DuskDrive-v0 episode_id=2
universe-psnd10-0 | [Mon Dec  5 12:48:40 UTC 2016] [/usr/local/bin/sudoable-env-setup] Allowing outbound network traffic to non-private IPs for git-lfs. (Going to fetch files via git lfs.)
universe-psnd10-0 | [init] [2016-12-05 12:48:40,163] init detected end of child process 99 with exit code 0, not killed by signal
universe-psnd10-0 | [init] [2016-12-05 12:48:40,163] init detected end of child process 107 with exit code 0, not killed by signal
universe-psnd10-0 | [init] [2016-12-05 12:48:40,163] init detected end of child process 108 with exit code 0, not killed by signal
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:40,231] Fetching files: git lfs pull -I git-lfs/flashgames.DuskDrive-v0.tar.gz
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,921] Finished running git lfs pull
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,922] git-lfs fetch succeeded
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,922] Unpacking files for flashgames.DuskDrive-v0
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,985] Merged 5 files from /tmp/flashgames.DuskDrive-v0/public -> /app/universe-envs/flashgames/build/public/flashgames.DuskDrive-v0
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,987] Merged 20 files from /tmp/flashgames.DuskDrive-v0/private -> /app/universe-envs/flashgames/build/private/flashgames.DuskDrive-v0
universe-psnd10-0 | [unpack-lfs] [2016-12-05 12:48:46,987] Completed unpack for flashgames.DuskDrive-v0 in 6.756s
universe-psnd10-0 | [Mon Dec  5 12:48:47 UTC 2016] [/usr/local/bin/sudoable-env-setup] [debug] unpack-lfs completed with status code: 0. Created completion file: /usr/local/openai/git-lfs/flashgames.DuskDrive-v0
universe-psnd10-0 | [Mon Dec  5 12:48:47 UTC 2016] [/usr/local/bin/sudoable-env-setup] Disabling outbound network traffic for flashgames.DuskDrive-v0
universe-psnd10-0 | [2016-12-05 12:48:47,085] [INFO:gym_flashgames.launcher] [EnvController] Launching new Chrome process (attempt 0/10)
universe-psnd10-0 | [2016-12-05 12:48:47,085] [INFO:root] Replacing selenium_wrapper_server since we currently do it at every episode boundary
universe-psnd10-0 | [2016-12-05 12:48:47,202] [selenium_wrapper_server] Calling webdriver.Chrome()
universe-psnd10-0 | [2016-12-05 12:48:48,497] [selenium_wrapper_server] Call to webdriver.Chrome() completed: 1.29s
universe-psnd10-0 | [2016-12-05 12:48:48,497] [INFO:gym_flashgames.launcher] [EnvController] Navigating browser to url=http://localhost/flashgames.DuskDrive-v0
universe-psnd10-0 | [2016-12-05 12:48:48,521] [INFO:root] [EnvController] Running command: /app/universe-envs/controlplane/bin/play_vexpect -e flashgames.DuskDrive-v0 -r vnc://<IP_ADDRESS>:5900 -d
universe-psnd10-0 | [2016-12-05 12:48:49,096] [play_vexpect] Using the golang VNC implementation
universe-psnd10-0 | [2016-12-05 12:48:49,096] [play_vexpect] Using VNCSession arguments: {'encoding': 'zrle', 'start_timeout': 7, 'subsample_level': 2, 'compress_level': 0, 'fine_quality_level': 50}. (Customize by running "env.configure(vnc_kwargs={...})"
universe-psnd10-0 | [2016-12-05 12:48:49,096] [play_vexpect] Printed stats will ignore clock skew. (This usually makes sense only when the environment and agent are on the same machine.)
universe-psnd10-0 | [2016-12-05 12:48:49,099] [play_vexpect] [0] Connecting to environment: vnc://<IP_ADDRESS>:5900 password=openai. Use the browser-based VNC client: http://None/viewer/?password=openai
universe-psnd10-0 | [2016-12-05 12:48:49,099] [play_vexpect] [0] Connecting to environment details: vnc_address=<IP_ADDRESS>:5900 vnc_password=openai rewarder_address=None rewarder_password=openai
universe-psnd10-0 | 2016/12/05 12:48:49 I1205 12:48:49.099818 711 gymvnc.go:417] [0:<IP_ADDRESS>:5900] opening connection to VNC server
universe-psnd10-0 | 2016/12/05 12:48:49 I1205 12:48:49.101615 711 gymvnc.go:550] [0:<IP_ADDRESS>:5900] connection established
universe-psnd10-0 | [2016-12-05 12:48:49,555] [play_vexpect] Waiting for any of [MaskState<initializing0>] to activate
universe-psnd10-0 | 2016/12/05 12:48:55 I1205 12:48:55.444911 56 gymvnc.go:374] [0:<IP_ADDRESS>:5900] update queue max of 60 reached; pausing further updates
[2016-12-05 13:49:40,151] [0:localhost:5900] Attached to running environment without reset
universe-psnd10-0 | [2016-12-05 12:49:40,150] [INFO:universe.rewarder.remote] WebSocket connection closed: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
[2016-12-05 13:49:40,163] [0] Closing rewarder connection
Traceback (most recent call last):
  File "utest.py", line 10, in <module>
    observation_n, reward_n, done_n, info = env.step(action_n)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 122, in step
    observation, reward, done, info = self._step(action)
  File "/home/iaroslav/universe/universe/wrappers/timer.py", line 20, in _step
    observation_n, reward_n, done_n, info = self.env.step(action_n)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 122, in step
    observation, reward, done, info = self._step(action)
  File "/home/iaroslav/universe/universe/wrappers/render.py", line 30, in _step
    observation_n, reward_n, done_n, info_n = self.env.step(action_n)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 122, in step
    observation, reward, done, info = self._step(action)
  File "/home/iaroslav/universe/universe/wrappers/throttle.py", line 117, in _step
    observation_n, reward_n, done_n, info = self._substep(action_n)
  File "/home/iaroslav/universe/universe/wrappers/throttle.py", line 132, in _substep
    observation_n, reward_n, done_n, info = self.env.step(action_n)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 122, in step
    observation, reward, done, info = self._step(action)
  File "/home/iaroslav/universe/universe/envs/vnc_env.py", line 447, in _step
    self._handle_crashed_n(info_n)
  File "/home/iaroslav/universe/universe/envs/vnc_env.py", line 518, in _handle_crashed_n
    raise error.Error('{}/{} environments have crashed! Most recent error: {}'.format(len(self.crashed), self.n, errors))
universe.error.Error: 1/1 environments have crashed! Most recent error: {'0': 'Rewarder session failed: Lost connection: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake) (clean=False code=1006)'}
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Xvnc TigerVNC 1.7.0 - built Sep  8 2016 10:39:22
universe-psnd10-0 | [tigervnc] Copyright (C) 1999-2016 TigerVNC Team and many others (see README.txt)
universe-psnd10-0 | [tigervnc] See http://www.tigervnc.org for information on TigerVNC.
universe-psnd10-0 | [tigervnc] Underlying X server release 11400000, The X.Org Foundation
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Initializing built-in extension VNC-EXTENSION
universe-psnd10-0 | [tigervnc] Initializing built-in extension Generic Event Extension
universe-psnd10-0 | [tigervnc] Initializing built-in extension SHAPE
universe-psnd10-0 | [tigervnc] Initializing built-in extension MIT-SHM
universe-psnd10-0 | [tigervnc] Initializing built-in extension XInputExtension
universe-psnd10-0 | [tigervnc] Initializing built-in extension XTEST
universe-psnd10-0 | [tigervnc] Initializing built-in extension BIG-REQUESTS
universe-psnd10-0 | [tigervnc] Initializing built-in extension SYNC
universe-psnd10-0 | [tigervnc] Initializing built-in extension XKEYBOARD
universe-psnd10-0 | [tigervnc] Initializing built-in extension XC-MISC
universe-psnd10-0 | [tigervnc] Initializing built-in extension XINERAMA
universe-psnd10-0 | [tigervnc] Initializing built-in extension XFIXES
universe-psnd10-0 | [tigervnc] Initializing built-in extension RENDER
universe-psnd10-0 | [tigervnc] Initializing built-in extension RANDR
universe-psnd10-0 | [tigervnc] Initializing built-in extension COMPOSITE
universe-psnd10-0 | [tigervnc] Initializing built-in extension DAMAGE
universe-psnd10-0 | [tigervnc] Initializing built-in extension MIT-SCREEN-SAVER
universe-psnd10-0 | [tigervnc] Initializing built-in extension DOUBLE-BUFFER
universe-psnd10-0 | [tigervnc] Initializing built-in extension RECORD
universe-psnd10-0 | [tigervnc] Initializing built-in extension DPMS
universe-psnd10-0 | [tigervnc] Initializing built-in extension X-Resource
universe-psnd10-0 | [tigervnc] Initializing built-in extension XVideo
universe-psnd10-0 | [tigervnc] Initializing built-in extension XVideo-MotionCompensation
universe-psnd10-0 | [tigervnc] Initializing built-in extension GLX
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Mon Dec  5 12:48:18 2016
universe-psnd10-0 | [tigervnc]  vncext:      VNC extension running!
universe-psnd10-0 | [tigervnc]  vncext:      Listening for VNC connections on all interface(s), port 5900
universe-psnd10-0 | [tigervnc]  vncext:      created VNC server for screen 0
universe-psnd10-0 | [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/Type1/, removing from list!
universe-psnd10-0 | [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/75dpi/, removing from list!
universe-psnd10-0 | [tigervnc] [dix] Could not init font path element /usr/share/fonts/X11/100dpi/, removing from list!
universe-psnd10-0 | [tigervnc]  Connections: accepted: <IP_ADDRESS>::48030
universe-psnd10-0 | [tigervnc]  SConnection: Client needs protocol version 3.8
universe-psnd10-0 | [tigervnc]  SConnection: Client requests security type VncAuth(2)
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Server default pixel format depth 24 (32bpp) little-endian rgb888
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Client pixel format depth 24 (32bpp) little-endian bgr888
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Mon Dec  5 12:48:19 2016
universe-psnd10-0 | [tigervnc]  Connections: accepted: <IP_ADDRESS>::36246
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Mon Dec  5 12:48:20 2016
universe-psnd10-0 | [tigervnc]  Connections: closed: <IP_ADDRESS>::36246 (Clean disconnection)
universe-psnd10-0 | [tigervnc]  EncodeManager: Framebuffer updates: 0
universe-psnd10-0 | [tigervnc]  EncodeManager:   Total: 0 rects, 0 pixels
universe-psnd10-0 | [tigervnc]  EncodeManager:          0 B (1:-nan ratio)
universe-psnd10-0 | [tigervnc]  Connections: accepted: <IP_ADDRESS>::36286
universe-psnd10-0 | [tigervnc]  SConnection: Client needs protocol version 3.8
universe-psnd10-0 | [tigervnc]  SConnection: Client requests security type VncAuth(2)
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Server default pixel format depth 24 (32bpp) little-endian rgb888
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Client pixel format depth 24 (32bpp) little-endian bgr888
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Mon Dec  5 12:48:49 2016
universe-psnd10-0 | [tigervnc]  Connections: accepted: <IP_ADDRESS>::48294
universe-psnd10-0 | [tigervnc]  SConnection: Client needs protocol version 3.8
universe-psnd10-0 | [tigervnc]  SConnection: Client requests security type VncAuth(2)
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Server default pixel format depth 24 (32bpp) little-endian rgb888
universe-psnd10-0 | [tigervnc]  VNCSConnST:  Client pixel format depth 24 (32bpp) little-endian bgr888
universe-psnd10-0 | [tigervnc]
universe-psnd10-0 | [tigervnc] Mon Dec  5 12:49:40 2016
universe-psnd10-0 | [tigervnc]  Connections: closed: <IP_ADDRESS>::36286 (Clean disconnection)
universe-psnd10-0 | [tigervnc]  EncodeManager: Framebuffer updates: 516
universe-psnd10-0 | [tigervnc]  EncodeManager:   Tight:
universe-psnd10-0 | [tigervnc]  EncodeManager:     Solid: 61 rects, 3.26606 Mpixels
universe-psnd10-0 | [tigervnc]  EncodeManager:            976 B (1:13386.2 ratio)
universe-psnd10-0 | [tigervnc]  EncodeManager:     Bitmap RLE: 261 rects, 3.17045 Mpixels
universe-psnd10-0 | [tigervnc]  EncodeManager:                 43.0459 KiB (1:287.777 ratio)
[2016-12-05 13:49:40,324] Killing and removing container: id=657c3c28301b3a75e90606a12b29025229d8eed7b516874c2c5a69e655f3ad55.
My system is Ubuntu 16.04, running with 8 core cpu, 16 gb of ram, and Titan GPU.
Do you know what could cause such behavior? Is there something I am missing?

This looks like the same issue as https://github.com/openai/universe/issues/6#issuecomment-264947599. Can you follow the instructions there, and paste me the output? I'll close this one as a dupe.


Add helper script for updating the signal spreadsheet semi-programmatically

Summary:
Add helper script for updating the signal spreadsheet semi-programmatically. Defines descriptive values by data source, with some additional logic to handle certain signals and cases (active/inactive signals) separately.
Saves updated spreadsheet to disk as a CSV. Updated columns must be manually pasted into the online spreadsheet.
Prerequisites:

[x] Unless it is a documentation hotfix it should be merged against the dev branch
[x] Branch is up-to-date with the branch to be merged with, i.e. dev
[ ] Build is successful
[x] Code is cleaned up and formatted

@melange396 this has been manually checked and doesn't need an involved review for correctness. Is the scripts dir an okay place for this to stay?


Trim only top and bottom

Is there a way to use .trim for only top and the bottom and not left and right?
http://sharp.dimens.io/en/stable/api-operation/#trim

Hello, the trim operation will be switching to the libvips-provided version from the next release as part of #914. This work will expose the dimensions calculated by the trim operation so perhaps you could use these for a second call to extract the area you're interested in?

Hi @lovell so is it possible to achieve this at the current version? I have a few .resize operations but was trying to find a way so trim does not keep its original size but resize down to that trimmed version. This way I can minus from original width and put extend with left and right. Is it possible to do this?

This will only be possible after the work for #914 is done, and only then as a two-step operation, so your best bet is to subscribe to it for updates.

It would be great if someone had a workaround solution to this


Sub rows disappear after global filter

First off, thanks for the great lib. I have been impressed at how easy it is to drop this in and use it (with your own UI). Easily in the top 5 best JS libs I have ever used.
Describe the bug (required)
A row can be expanded, showing the sub rows. When a row is filtered using globalFilter the subrow is removed and never seen again, even after the filter has been cleared. The subRows is still in "data", but it's no longer in "rows".
Provide an example via Codesandbox! (required)
https://codesandbox.io/s/serene-chebyshev-80wsi?file=/src/App.js
Steps To Reproduce (required)
Steps to reproduce the behavior:

Use the input to filter a specific row.
The expand hand is now gone.
Clear the filter
The expand hand is still gone.

Expected behavior (Recommended)
When the filter is removed the sub row should reappear.
(I can see the sub rows being filtered out based on the query, but after the query is cleared the sub rows should come back)
Screenshots

Desktop (please complete the following information):

OS: Win 10
Browser:
-- Chrome (v81.0.4044.122 (Official Build) (64-bit));
-- FF (v75.0)

Smartphone (please complete the following information):
Not tested
Additional context
using v7, in case that wasn't obvious.
I did do some light digging and it appears in useGlobalFilter.js:138-140 it filters the rows and sets row.subRow to that result. When the filter is removed the row.subRow is still an empty array; it doesn't get reverted back to what is in data. 🤷♂

Duplicate of https://github.com/tannerlinsley/react-table/issues/2064


Unable to download Virtual Display Driver?

It is repeatedly failing to download the virtual display driver. Any ideas what's going on?

Hello, please download and install VDD manually until i update the code for the AIO because new VDD updates breaks my code (different names used for packages)

The new release resolved the issue.


Switch Gw2Sharp rate-limiting / token bucket implementation to middleware.

Our current self rate-limiting method abuses Gw2Sharp's existing caching methods by wrapping them with a token bucket.  This works fairly well, but with the release of Gw2Sharp v0.11.0 we now have more appropriate ways to do this using middlewares.  This will let us handle cache hit handling and delaying of requests in a less hacky manor.
As it stands, the existing CacheMethod abusing method still appears to be working after a couple of changes were made to account for the breaking changes in v0.11.0.

Fixed in https://github.com/blish-hud/Blish-HUD/pull/348


Add BE analytics event for templates

@kocharrahul7 I've logged and attached application name property when template is forked. What other things we need to do for this? We're logging event for fork template only.

This is enough for now.
I assume we have a user ID as well, so we can see the total number of users who are forking templates?

Yes, that's correct.


http: add composite filter allowing dynamic filter selection

Adds a new filter that makes use of the new matching API to allow specifying a filter configuration to use for specific match
results. Once a configuration is selected, the desired filter is created and all callbacks delegated to the specific filter.
Risk Level: Low, new filter
Testing: IT + UT
Docs Changes: Inline proto comments
Release Notes: wip

@htuch FYI this is the alternative implementation for the "specify a filter config via a match action" filter impl
This can be reviewed now, CI failure is a coverage failure due to NOT_IMPLEMENTED macros

I think there are a few open comment threads. Are you still making changes? LMK when it's ready to review again.
/wait-any

@htuch you have pending changes requested, ptal


How to support chinese text?

The bug like this picture:

Jack

为什么？

Same bug with Japanese

Thanks for the report, this issue should be fixed with release 1.0.5 (also available on CocoaPods).


Fix #64077 - Allow to sort branches alphabetically

@joaomoreno , As discussed I have added a setting called  "sortOrderForBranchList" to fix #64077.
Please review this and let me know if anything needs to be changed , Thanks :)

@joaomoreno , Please review this :)

@skprabhanjan Thanks for your contribution! :beers:
I've changed the setting to git.branchSortOrder.


cloud_bigtable_ycsb failure

I'm having trouble running cloud_bigtable_ycsb.
I got: com.yahoo.ycsb.DBException: java.io.IOException: Failed to getTableDescriptor()

Do you have anything else in the traceback?

Fixed by pull request #932


v5.1.0 bridge cannot work, cause the https request was unknown.

I updated to the latest version and loaded my page, received this warning:
WebViewJavascriptBridge: WARNING: Received unknown WebViewJavascriptBridge
If I change the url to http or back to 5.0.5, It does work!

Did you get it to work?

Yes, I updated to 5.1.2 and it worked, thanks a lot!


Improve JAXey LLMish model compilation through flow/stream.

JAX inputs have a lot of quirks that need to be fixed at the frontend level (like passing in/returning thousands of tensors to each invocation, not having non-tensor operations, etc). Looking at the large models in #13543 and #13459 there are several patterns we could handle better post-HLO, though, and that would be good to have even if frontend improvements are made.
Some things found in an initial skim through:
Detensoring creates many bad load + arith + splat sequences
Should probably disable detensoring as it was never finished - or a retensoring pass in flow should be able to find slices of load -> splat and move them into dispatch regions
    // 476 blocking loads total
    %950 = stream.async.transfer %211 : !stream.resource<transient>{%c4} -> !stream.resource<staging>{%c4}
    %951 = stream.async.load %950[%c0] : !stream.resource<staging>{%c4} -> f32
    %952 = arith.addf %951, %cst_18 : f32
    %953 = arith.bitcast %952 : f32 to i32
    %8167 = stream.async.splat %953 : i32 -> !stream.resource<external>{%c4}

Splats are used to pass scalar values into dispatches that could instead be done via normal dispatch operands
Need a pattern in flow to capture the splat operand and drop the tensor use
    %3766 = stream.async.splat %3765 : i32 -> !stream.resource<transient>{%c4}
    %3767 = stream.async.splat %3756 : i32 -> !stream.resource<transient>{%c4}
    %3768 = stream.async.dispatch @_main_dispatch_2777::@_main_dispatch_2777_generic_1x8192_f32(%3766[%c0 to %c4 for %c4], %3569[%c0 to %c32768 for %c32768], %3767[%c0 to %c4 for %c4], %arg199[%c0 to %c32768 for %c32768]) : (!stream.resource<transient>{%c4}, !stream.resource<transient>{%c32768}, !stream.resource<transient>{%c4}, !stream.resource<external>{%c32768}) -> !stream.resource<transient>{%c32768}

Splats that later become fills don't coalesce due to resource padding
Could special case pack any allocation under the resource config alignment so these could be dense, then pull all into a single dispatch to perform at once (scatter) instead of one transfer operation per 4 byte splat
    %8402 = stream.async.splat %8144 : i32 -> !stream.resource<external>{%c4}
    %8403 = stream.async.splat %8149 : i32 -> !stream.resource<external>{%c4}
    %8404 = stream.async.splat %8154 : i32 -> !stream.resource<external>{%c4}
->
      stream.cmd.fill %3307, %arg922[%c7317683072 for %c4] : i32 -> !stream.resource<external>{%c7317692032}
      stream.cmd.fill %3311, %arg922[%c7317683136 for %c4] : i32 -> !stream.resource<external>{%c7317692032}
      stream.cmd.fill %3315, %arg922[%c7317683200 for %c4] : i32 -> !stream.resource<external>{%c7317692032}
      stream.cmd.fill %3319, %arg922[%c7317683264 for %c4] : i32 -> !stream.resource<external>{%c7317692032}
      stream.cmd.fill %3323, %arg922[%c7317683328 for %c4] : i32 -> !stream.resource<external>{%c7317692032}
      stream.cmd.fill %3327, %arg922[%c7317683392 for %c4] : i32 -> !stream.resource<external>{%c7317692032}

Dispatch region formation puts lots of scalar arithmetic in their own dispatches
Need much more aggressive dispatch region formation, or aggressive detensoring so that retensoring makes its own dispatch regions, etc - these are ~100k to ~1m times more expensive than they should be
    // 200 splat + scalar dispatches
    %1390 = stream.async.splat %1047 : i32 -> !stream.resource<transient>{%c4} attributes {util.ordinal = 2133 : index}
    %1391 = stream.async.dispatch @_main_dispatch_468::@_main_dispatch_468[%1387](%1390[%c0 to %c4 for %c4], %764[%c0 to %c4 for %c4], %1387) {util.ordinal = 2134 : index} : (!stream.resource<transient>{%c4}, !stream.resource<transient>{%c4}, index) -> %764{%c4}
    %1392 = stream.async.splat %1058 : i32 -> !stream.resource<transient>{%c4} attributes {util.ordinal = 2135 : index}
    %1393 = stream.async.dispatch @_main_dispatch_468::@_main_dispatch_468[%1387](%1392[%c0 to %c4 for %c4], %765[%c0 to %c4 for %c4], %1387) {util.ordinal = 2136 : index} : (!stream.resource<transient>{%c4}, !stream.resource<transient>{%c4}, index) -> %765{%c4}
    %1394 = stream.async.splat %1066 : i32 -> !stream.resource<transient>{%c4} attributes {util.ordinal = 2137 : index}
    %1395 = stream.async.dispatch @_main_dispatch_468::@_main_dispatch_468[%1387](%1394[%c0 to %c4 for %c4], %766[%c0 to %c4 for %c4], %1387) {util.ordinal = 2138 : index} : (!stream.resource<transient>{%c4}, !stream.resource<transient>{%c4}, index) -> %766{%c4}

Dispatch region formation isn't aggressive enough around reductions so 100's of elementwise + reduction dispatches on 2048 elements
Likely because of multiple users of %5564, but all users are reductions of the same input that could be fused together
// elementwise add of 2048 elements
%5564 = stream.async.dispatch @_main_dispatch_4395::@_main_dispatch_4395_generic_2048_f32(%arg121[%c0 to %c8192 for %c8192], %4631[%c0 to %c8192 for %c8192]) : (!stream.resource<external>{%c8192}, !stream.resource<transient>{%c8192}) -> !stream.resource<external>{%c8192}
// sum reduction of 2048 elements -> scalar
%6712 = stream.async.dispatch @_main_dispatch_4578::@_main_dispatch_4578_generic_2048_f32(%5564[%c0 to %c8192 for %c8192], %6711[%c0 to %c4 for %c4]) : (!stream.resource<external>{%c8192}, !stream.resource<transient>{%c4}) -> !stream.resource<transient>{%c4}

Some transient allocations end up host-ordered (stream.resource.alloc vs stream.async.alloca)
May have to do with control flow?
%632:36 = stream.resource.alloc uninitialized : !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c8192}, !stream.resource<transient>{%c4096}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c8388608}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}, !stream.resource<transient>{%c16777216}

There's a mysterious tf.aliasing_output attribute on function args that could be useful
// I think this means that output 0 aliases arg0
%arg0: !hal.buffer_view {jax.arg_info = "state.step", mhlo.sharding = "{replicated}", tf.aliasing_output = 0 : i32}

Hey Ben, this is great. Right now I'm not seeing this on the profiles though. FYI we're currently 10x off on performance on this workload due to first-order codegen issues mainly. I think the most actionable thing in this vein that I see is a very large amount of time spent in memory allocation (~200-300ms out of ~1.1s, with a strong baseline for this workload being ~100ms).
I filed https://github.com/openxla/iree/issues/13566 for that, but it sounds like we need more ownership and input from higher up in the stack on that one -- I'm wondering if you can think of anything we can do to move that one forward on our end here in IREE, or are we completely blocked without further from higher in the stack? Even something like a concrete decision tree or forward-looking set of options that seem feasible to implement on our end here, that higher up can consider, could be useful.

Right now I'm not seeing this on the profiles though. FYI we're currently 10x off on performance on this workload due to first-order codegen issues mainly.

Some of these issues are structural, so simplifying could unlock more of our existing compiler optimizations, or at least make it easier to read the IR to see what to focus on. Directly going after the profiled hotspots might just find a local minimum (e.g. we might not need to optimize a convolution with striding if we can instead convert it into a matmul or skip running it altogether).

Yep! This is why instrumented tracing is so useful - looking at the top 5 hotspots is useful for driving codegen work but not useful for non-codegen work that is much more about the entire workload instead of any particular work item.

Good example from immediately before --pass-pipeline=builtin.module(func.func(iree-flow-form-dispatch-regions)):
https://gist.github.com/benvanik/a6846aa7ad6999ce167705fe3f6f0853
As an example here's one where we are going device->host->device but shouldn't be; there's several things we can do but I'll file an issue for the real solution to this particular issue:
func.func @folding() -> tensor<f32> {
  %cst_13 = arith.constant 0.000000e+00 : f32
  %cst_14 = arith.constant 0xFF800000 : f32
  %cst_17 = arith.constant 1.536000e+03 : f32
  %cst_63 = arith.constant 1.000000e+00 : f32
  %33 = tensor.empty() : tensor<f32>
  %34 = linalg.fill ins(%cst_17 : f32) outs(%33 : tensor<f32>) -> tensor<f32>
  %847 = tensor.empty() : tensor<4194304xf32>
  %848 = linalg.fill ins(%cst_14 : f32) outs(%847 : tensor<4194304xf32>) -> tensor<4194304xf32>

  %876 = linalg.fill ins(%cst_14 : f32) outs(%33 : tensor<f32>) -> tensor<f32>
  %883 = flow.dispatch.region -> (tensor<f32>) {
    %989 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> ()>], iterator_types = ["reduction"]} ins(%848 : tensor<4194304xf32>) outs(%876 : tensor<f32>) {
    ^bb0(%in: f32, %out: f32):
      %990 = math.absf %in : f32
      %991 = arith.maxf %out, %990 : f32
      linalg.yield %991 : f32
    } -> tensor<f32>
    flow.return %989 : tensor<f32>
  }

  // this extract + maxf should be folded into either the producer or the consumer
  // in this case this max should be able to fold with the producer above as
  // max(max(reduction), cst) == max(reduction, cst)
  %extracted_852 = tensor.extract %883[] : tensor<f32>
  %884 = arith.maxf %extracted_852, %cst_63 : f32

  // this splat should be eliminated if the maxf is folded above
  // if not, the scalar %884 should be captured by the dispatch region instead
  %from_elements_853 = tensor.from_elements %884 : tensor<f32>

  %885 = flow.dispatch.region -> (tensor<f32>) {
    %989 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> ()>, affine_map<(d0) -> ()>], iterator_types = ["reduction"]} ins(%848, %from_elements_853 : tensor<4194304xf32>, tensor<f32>) outs(%34 : tensor<f32>) {
    ^bb0(%in: f32, %in_910: f32, %out: f32):
      %990 = arith.divf %in, %in_910 : f32
      %991 = arith.mulf %990, %990 : f32
      %992 = arith.addf %out, %991 : f32
      linalg.yield %992 : f32
    } -> tensor<f32>

    flow.return %989 : tensor<f32>
  }

  return %885 : tensor<f32>
}

@MaheshRavishankar if there are any parcelable/concrete sub tasks, please let me and @Young768 know.

There is a lot of content here and in other bugs that is a bit hard to parse and categorize... https://github.com/openxla/iree/issues/13545 is a place where we have some tasks... I have to look at everything listed on this and related bugs and organize them a bit. So far I am going through this list here https://github.com/openxla/iree/issues/13545#issuecomment-1560063164  . Once I land that I can look at what is needed after that to get a better picture...
For a more deep dive list of tasks waiting for an inference model to help better prioritize.


Workflow triggered by Dependabot fails

Upgrading from 1.5.0 to 1.6.0 causes my script to error out with this line.

Thank you for the report! Are you 100% sure that if you revert to v1.5.0 everything works? There are very little changes between the two version. It is effectively only one changed line. So it would really surprise me if this change from Node 12 to Node 16 could cause this issue...

I had a similar problem with a Dependabot-created merge commit. The error message was the same, and the failure also happened when upgrading from 1.5.0 to 1.6.0:
Run<EMAIL_ADDRESS>  with:
    gistID: 6762bee806477c52e079f21d2f252688
    filename: timeeval__heads_main.json
    label: Test Coverage
    message: 0.8946
    color: green
    namedLogo: pytest
    forceUpdate: false
  env:
    CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir
    COVERAGE: 0.8946
    BRANCH: heads_main
Failed to get gist, response status code: 401, status message: Unauthorized
/home/runner/work/_actions/schneegans/dynamic-badges-action/v1.6.0/index.js:209
        if (oldGist.body.files[filename]) {
                              ^

TypeError: Cannot read properties of undefined (reading 'timeeval__heads_main.json')
    at /home/runner/work/_actions/schneegans/dynamic-badges-action/v1.6.0/index.js:209:[3](https://github.com/HPI-Information-Systems/TimeEval/actions/runs/3393366151/jobs/5640618821#step:8:3)1
    at processTicksAndRejections (node:internal/process/task_queues:96:[5](https://github.com/HPI-Information-Systems/TimeEval/actions/runs/3393366151/jobs/5640618821#step:8:5))

However, the cause was that the commit was authored by Dependabot and the Action missed the correct access rights to the gist. The default secrets for GitHub Actions do not apply for Dependabot and you have to provide them separately in Repository Settings > Secrets > Dependabot.
I hope this information can help.

Ohh, I did not knew this. Thank you very much! I'll add a corresponding note to the README.

Thank you @CodeLionX!
I was running into this issue as well.


CLI improvements 3/n

from @gadkins:
CLI improvements needed:
1. More user-friendly login error - currently it just says Not Found
2. When uffizzi login is run, replace the current message to STDOUT
Login to Uffizzi to view and manage your previews.
with something more useful like:
Logging in to Uffizzi server
...
Login successful
or similar
3.  More user-friendly Resource Not Found error - tell me which resource is not found and how to fix it
4.  Add single quotes around the service name to improve readability. E.g.: Service 'worker' has …

:point_right: 1. @gadkins Hello! How did you get an error Not found after uffizzi login command? Please, could you describe with more details.
2. TESTED - OK
bash-5.1# bundle exec uffizzi login
Login to Uffizzi server.
Login successfull
bash-5.1#

:point_right: 3. Needs to remove substring UffizziCore:: from the output.
bash-5.1# bundle exec uffizzi project compose describe
UffizziCore::ComposeFile Not Found
bash-5.1# bundle exec uffizzi project set-default test
UffizziCore::Project Not Found
bash-5.1# bundle exec uffizzi preview list
The project has no active deployments
bash-5.1# bundle exec uffizzi preview describe deployment-1
UffizziCore::Deployment Not Found

:point_right: 4.  Needs to add single quotes around the service name in the error whew image does not defined in the compose file:
bash-5.1# bundle exec uffizzi preview create test-compose/uffizzi-compose-webhook-docker.yml
Service webhooks-test-app has neither an image nor a build context specified. At least one must be provided.

E.g. file:
services:
  webhooks-test-app:
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres

x-uffizzi-ingress:
  service: webhooks-test-app
  port: 80

x-uffizzi-continuous-preview:
  share_to_github: true
  delete_preview_after: 1h
  tag_pattern: uffizzi_request_*
  delete_preview_when_image_tag_is_updated: true
  deploy_preview_when_image_tag_is_created: true
  delete_preview_when_pull_request_is_closed: true
  deploy_preview_when_pull_request_is_opened: true

Skipped

TESTED - OK

TESTED - OK

bash-5.1# bundle exec uffizzi project compose describe
ComposeFile Not Found
bash-5.1# bundle exec uffizzi project set-default test
Project Not Found
bash-5.1# bundle exec uffizzi preview describe deployment-1
Deployment Not Found
bash-5.1#

TESTED - OK

bash-5.1# bundle exec uffizzi project compose set -f test-compose/uffizzi-compose-webhook-docker.yml
Service 'webhooks-test-app' has neither an image nor a build context specified. At least one must be provided.
bash-5.1#


change readme test

Adding a language

[ ] The code displays "Hello World" (tio.run may help for testing)
[ ] I have no association with the language
[ ] There are no copyright issues with this code
[ ] The language has not been added prior to this pull request
[ ] I have updated the README

Link to programming language:

ignore


Filters are not functioning

My server is running HuskChat | v3.0.4
I tested with this advertising filter config:
  ADVERTISING:
    type: filter
    enabled: true
    channels:
    - global
    - local
    private_messages: true
    broadcast_messages: false

and this channel:
- id: local
  format: '&8[&r&#62dbfc&%server% &8&l| &r&eLocal&8]&f %fullname%&r&f:&e '
  broadcast_scope: LOCAL
  log_to_console: true
  restricted_servers: []
  filtered: true
  permissions: {}
  shortcut_commands:
  - /local
  - /l

As far as I could tell, it seems like spam, repeat, and the emoji replacer are also not functioning.

Up this issue please @WiIIiam278 ! Tested for me too on Velocity side. HuskChat | v3.0.4
I confirm this one. All filters didn't work.

HuskChat's been a little lower on my priority list at them moments but yeah I will get to it as I do appreciative that this is a significant issue

Closed in #261


Add fees_paid API

The amount of fees paid in different coins

SELECT
	height,
	JSONExtractString(arrayJoin(JSONExtractArrayRaw(JSONExtractString(fee, 'coins'))), 'denom')	as denom,
	sum(toUInt256OrZero(JSONExtractString(arrayJoin(JSONExtractArrayRaw(JSONExtractString(fee, 'coins'))), 'amount'))) as amount
FROM spacebox.`transaction` FINAL
GROUP BY height, denom

Here is a nice example
Now we can get all currencies as fee

return structure
['y': date, 'x': [{'denom': denom, 'amount': amount}, ...]

Historical and static endpoints.

@bro-n-bro-0 @bro-n-bro-2 please check apis, if everything is ok, please close issue

Thanks !