[Docker Desktop] Add GPU support with WSL2

Tell us about your request
Microsoft just announced GPU virtualization support in WSL2, demonstrating a Tensorflow workload running on the GPU. I want my containers to leverage that for Cuda, TF and other GPGPU worloads.
Which service(s) is this request for?
Docker Desktop w/ wsl2
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Deep Learning, ML, AI and other projects leveraging GPUs can't get addressed today with Docker Desktop. We want to be able to work on those on a local machine.
Are you currently working around the issue?
No, the only viable solution today is working on a remote GPU-enabled Linux server.

I'll add in that native GPU support would be very helpful in terms of rendering realtime GUI and physics environments like Gazebo and custom Unreal/Unity environment for robotics.

@mrkoz note that the currently announced driver is just for GPU compute, not GUI; there may later be GUI support, but with a different driver and unknown timetable.

@justincormack interesting, i honestly don't know of gazebo's physics engine runs in gui space or gpu compute. either way it's cool to see things moving foward.

This is extremely important for Machine Learning development on Windows PCs.

Are you currently working around the issue?
No, the only viable solution today is working on a remote GPU-enabled Linux server.

There is now a workaround where you can use the latest Windows 10 Insider Preview to utilise CUDA on WSL that lets you run GPU Linux containers natively on Windows, however it does involve many extra steps and requires installing/configuring a Linux distro. It would be much more seamless if this were to be directly integrated into Docker Desktop! :)

This is extremely important for Machine Learning development on Windows PCs.

Are you currently working around the issue?
No, the only viable solution today is working on a remote GPU-enabled Linux server.

There is now a workaround where you can use the latest Windows 10 Insider Preview to utilise CUDA on WSL that lets you run GPU Linux containers natively on Windows, however it does involve many extra steps and requires installing/configuring a Linux distro. It would be much more seamless if this were to be directly integrated into Docker Desktop! :)

In addition to this, it seems that Docker Desktop is the only supported method for VSCode's remote containers extension; see https://github.com/microsoft/vscode-remote-release/issues/3534

Anyone knows when this will be implemented?

Vey excited for this too, currently stopped by https://github.com/docker/for-win/issues/7581

So this means that I have to wait for this feature to be added to a Windows 10 Update (stable) for being able to use GPU in Docker in WSL 2?

when will be release?

We released a preview of this to the users on our development preview program yesterday. If you're interested in joining the program, please see https://www.docker.com/blog/expanding-dockers-developer-preview-program/.

Thank you; I've signed up for developer builds and will validate this when approved.
FWIW there is a work-around for VSCode (see discussion in https://github.com/microsoft/vscode-remote-release/issues/3534) using a Windows Docker CLI with a WSL docker context connected over tcp.

I just signed up. I am excited to have GPU support in WSL2 so that I can run MLPerf.

I just signed up. I am excited to have GPU support in WSL2 so that I can run MLPerf.

@stephen-turner Thanks. I'm using it for the past few days and it looks good.
Just for curiosity and because maybe others that search for it will land on this thread: Is there alread an implementation for the "--gpus=all" flag in docker compose?
I did not find anything in the docs or discussions yet.

@stephen-turner Thanks. I'm using it for the past few days and it looks good.
Just for curiosity and because maybe others that search for it will land on this thread: Is there alread an implementation for the "--gpus=all" flag in docker compose?
I did not find anything in the docs or discussions yet.

We just release Docker Desktop 3.1.0 with WSL2 GPU para-virtualization support for NVidia GPUs. A last minute issue prevented us to update compose though and we are still on 1.27.4. We'll do an update soon with compose 1.28 so that you can leverage GPU with docker-compose (note that you can also download docker-compose 1.28 rc3 separatly from https://github.com/docker/compose/releases/tag/1.28.0-rc3 and use it with docker desktop)

We just release Docker Desktop 3.1.0 with WSL2 GPU para-virtualization support for NVidia GPUs. A last minute issue prevented us to update compose though and we are still on 1.27.4. We'll do an update soon with compose 1.28 so that you can leverage GPU with docker-compose (note that you can also download docker-compose 1.28 rc3 separatly from https://github.com/docker/compose/releases/tag/1.28.0-rc3 and use it with docker desktop)

We've now shipped 3.2.0 with compose 1.28.
I'm going to close this ticket now. GPU support in Docker Desktop is still officially experimental until Microsoft ship it in a non-Insiders build of Windows, but our work is done.

Thank you docker team for the great work, much appreciated! 👏 👏 🎉


Set up scenario testing CI

As pointed out here, it's useful if we can run scenario testing on CI.
We can use TIER IV's Autoware Evaluator for this use case.
In that case, we need to add .webauto-ci.yml to this repository.
https://github.com/tier4/AutowareArchitectureProposal.proj/blob/474da436880f93bd59d87107bd9d83254b0cd9b3/.webauto-ci.yml
https://gitlab.com/autowarefoundation/autoware.auto/AutowareAuto/-/blob/635b36e0b461f66f25fd4a9c30231ca5f129e5db/.webauto-ci.yml

this is a great feature.

Closed by #397.


Security

src/pypadel/app/client/package-lock.json seems to breach some security issues.
Should this file be a secret?

My suggestion would be to remove the web app directory completely for now until it is more developed. I added this to PR #30.

Removed the web app development to a different branch for now.


[feature request] Laminas Framework support

Zend Framework has been replaced by the Laminas MVC framework (https://framework.zend.com/ and https://getlaminas.org/).
We use custom installers and have the type "zend-module" in use. It would be nice to have a "laminas-module" type available with identical functionality.

Why do you use it/need it? AFAIK this was done initially for BC with old ZF setups. I can't imagine Laminas still requires things to be installed in some specific dir rather than being in vendor like they should be?

As per the Laminas Skeleton app (https://github.com/laminas/laminas-mvc-skeleton), Laminas modules are still a thing, residing in /module. When having a unique repo for each module, the ZF2 way to install the module was used: "zend-module".
At this time, I don't use that method any longer.


Refactor general project structure

Refactor general project structure/directories. You can check out [INSERT LINK TO OUR SAMPLE REPO] as an example.

I believe the project structure is quite set now (not saying it can't be refactored). This is not so much an issue but an ongoing process so I am closing this now.


Simplify s2n_record_write code for TLS 1.3

Issue # (if available):
Description of changes:
remove logic for dealing with record type as it can be handled by its caller in handshake_io
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

:exclamation: No coverage uploaded for pull request head (tls13_simplify_record_encryption@85f5d01). Click here to learn what that means.
The diff coverage is n/a.

On further consideration, I'm reversing this decision as it would be more work for downstream users of s2n_send() to make this wire format change.


Transfer simple features from AMIII repo

Update the generic version with the simple developments as identified in #10.
For each feature, include a test serving as example in both script and pipeline mode.

already done


Highlighting for custom types in html script tag

I am developing roll20 Character sheets in vscode, and they use a custom type for their script tag, "sheet/worker".
It would look something like this:
<script type="text/worker">
on('sheet:opened', function() {
    console.log("Hey!");
});
</script>

It uses javascript, so it would be nice if there could be a setting for custom types and corresponding languages in the html Language Features server, and I imagine there are other unsupported or custom types other people would have uses for.

You mean like a setting where you could list additional script types. That would be nice, but not syntax highlighting is based onTextMate grammars which are static files and don't have the concept of variables.
TextMate has a mechanism to inject additional rules: Grammar injection. Extension authors can add such rules.
https://github.com/mjbvz/vscode-fenced-code-block-grammar-injection-example is an example for this.


Remove dependency on SharpZipLib (GPL) for if_warden

We've removed the dependency on SharpZipLib, due its GPL license causing unhappiness with our legal team.  Brannon should have context on this.

Never mind!  This includes waaay too much.


Remove old libs & add a test

I've removed old libraries usage and improved the readme with a php example

Any idea why I committed the entire php-parser and php-unparser repos? They must have been unstable or I couldn't get the versions to work together or something.
This is awesome. I tested it and it works much better than it did before! I'm going to go ahead and merge this into master because it's much better than master currently.
Running it on my pile o' files, I did find one tiny bug:
-                url: '<?php echo BASE;?>single_car.php?vin='+vin,
+            url: '<?php
+echo BASE;single_car.php?vin=' + vin,

Here's another example:
-                <?php } ?>
-              $(".qq_car_list").html('loading...');
+                <?php
+
+} $('.qq_car_list').html('loading...');

Is this an OK place to report a bug or should I post it in php-parser or php-unparser?

I also have one feature request... Could you check for one-liners and make them inline? That appears to be the one case where the beautifier makes code less readable. Here's an example:
-                            <?php if ($qq_id ==1) { ?>
-                              $('#previewSalesperson').val('<?php echo $salesperson; ?>');
-                              $("#popupName").val('<?php echo $guest; ?>');
-
-                              $("#popupPhone").val('<?php echo $phone; ?>');
-                              $("#popupEmail").val('<?php echo $popup_email; ?>');
-                              $("#popupAddress").val('<?php echo $address; ?>');
-                              $("#popupCity").val('<?php echo $city; ?>');
-                              $("#popupState").val('<?php echo $state; ?>');
-                              $("#popupZip").val('<?php echo $zip; ?>');
-
-                                <?php } else { ?>
-                                $('#previewSalesperson').val('<?php echo urldecode($info->salesperson); ?>');
-                                $("#popupName").val('<?php echo urldecode($info->name); ?>');
-                                $("#popupEmail").val('<?php echo $info->email; ?>');
-                                $("#popupPhone").val('<?php echo $info->phone; ?>');
-
-                                $("#popupAddress").val('<?php echo $info->address; ?>');
-                                $("#popupCity").val('<?php echo $info->city; ?>');
-                                $("#popupState").val('<?php echo $info->state; ?>');
-                                $("#popupZip").val('<?php echo $info->zip; ?>');
-                                <?php } ?>
+                            <?php
+    if ($qq_id == 1) {
+        ?>                              $('#previewSalesperson').val('<?php
+
+        echo $salesperson;
+        ?>');
+                              $("#popupName").val('<?php
+
+        echo $guest;
+        ?>');
+                              $("#popupPhone").val('<?php
+
+        echo $phone;
+        ?>');
+                              $("#popupEmail").val('<?php

+        echo $popup_email;
+        ?>');
+                              $("#popupAddress").val('<?php
+
+        echo $address;
+        ?>');
+                              $("#popupCity").val('<?php
+        echo $city;
+        ?>');
+                              $("#popupState").val('<?php
+        echo $state;
+        ?>');
+                              $("#popupZip").val('<?php
+        echo $zip;
+        ?>');
+
+                                <?php
+
+    } else {
+        ?>                                $('#previewSalesperson').val('<?php
+
+        echo urldecode($info->salesperson);
+        ?>');
+                                $("#popupName").val('<?php
+
+        echo urldecode($info->name);
+        ?>');
+                                $("#popupEmail").val('<?php
+
+        echo $info->email;
+        ?>');

Hi @wmhilton, I would be better to report these kind of bugs on php-unparser as it could be fixed there.
For the inline PHP mixed with HTML, I'm not sure that it could be fixed from php-unparser.
The best approach could be to treat each PHP file as HTML (as you have done before), make it indent, and next, extract PHP, make it indent, and next merge both indentations. We can discuss on https://github.com/wmhilton/beautify/issues/4


bug: torch installed without cuda support

Description
def build():
    base(os="ubuntu20.04", language="python3.8")
    install.cuda(version="11.2.2", cudnn="8")
    install.conda_packages(
        name=[
            "pytorch==1.12.1",
            "cudatoolkit=10.2",
        ],
        channel=["pytorch"],
    )

https://pytorch.org/get-started/previous-versions/

/cc @VoVAllen @kemingy

What's the error message?

I failed to reproduce failed with initial frozen solve

It may be caused by cache. The error is that I didn't install a torch with cuda support.

Why do you need install.cuda(version="11.2.2", cudnn="8") and "cudatoolkit=10.2" at the same time?

I don't know wether we can install.cuda version 10.2


Add require_implicit style to RSpec/ImplicitSubject

Although the explicit style is recommended by default, some users may prefer the implicit style. For such users, it would be better if there is an option to make the style consistent with the explicit style, rather than asking them to disable this cop.

Before submitting the PR make sure the following are checked:

[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Updated documentation.
[x] Added an entry to the CHANGELOG.md if the new code introduces user-observable changes.
[x] The build (bundle exec rake) passes (be sure to run this locally, since it may produce updated documentation that you will need to commit).

If you have created a new cop:

[ ] Added the new cop to config/default.yml.
[ ] The cop is configured as Enabled: pending in config/default.yml.
[ ] The cop is configured as Enabled: true in .rubocop.yml.
[ ] The cop documents examples of good and bad code.
[ ] The tests assert both that bad code is reported and that good code is not reported.
[ ] Set VersionAdded in default/config.yml to the next minor version.

If you have modified an existing cop's configuration options:

[x] Set VersionChanged in config/default.yml to the next major version.

I think the name of the style "always" is debatable. Since this cop is called ImplicitSubject, I named it "always" as in "always use implicit subject."

I added a multi-line example and then an example about named subject.
I figured that if named subject is used, it would be better to ignore it. Or should we also flag named subject as an offense and prompt to convert to unnamed subject?
This is my opinion, but I personally think it would be better to keep this cop only to decide whether is_expected should be used or expect(subject) should be used. Rather than having this cop decide what the named subject should be, it would be better to have a separate cop that decides whether the named subject should be recommended or not, and have it decide there.

I tend to agree that if there is named subject in use, it's better to be left explicit, as you can't use the name with the implicit syntax, and there might be a reason you have chosen a name.

if there is named subject in use, it's better to be left explicit, as you can't use the name with the implicit syntax, and there might be a reason you have chosen a name.

Furthermore, there can be more than one:
subject(:foo) { ... }
context '' do
  subject(:bar) { ... }
  it do
    expect(foo).to ...
  end

and we'll end up with a task to properly detect if the used explicit subject is also the right implicit one.

I have renamed some of the required CI checks, which unfortunately means that you will need to rebase this branch to make CI pass.


[tests] Fix "the input device is not a TTY" error in CI tests

[tests] Add tests for all TENANT_ID/TENANT_TOKEN variables
All these variables are actually empty at the moment. We'll fix it, but
these tests will catch future regressions.

[tests] Production installation: add -T flag to docker-compose exec
Fixes "the input device is not a TTY" error in CI tests.

This should fix the issue in master.


Controller Input mappings inconsistancy in Prefabs

While working on another project, I noticed an inconsistency in the Input names that are used in different components of the project.
MixedRealityCameraParent uses:

CONTROLLER_LEFT_STICK_HORIZONTAL
CONTROLLER_LEFT_STICK_VERTICAL
CONTROLLER_RIGHT_STICK_HORIZONTAL
CONTROLLER_RIGHT_STICK_VERTICAL

However the Input system prefab included with the Manager Prefab uses:

ControllerLeftStickX
ControllerLeftStickY

I did note some of the Examples / tests have been updated to use the Fist set but these have not been replicated to the Prefab

Thanks for reporting.  I've got plans to look this over as soon as we get the initial input updates completed.

@DDReaper can you post a screen shot of this?
I can't seem to find the component you're talking about.

Ok, sorry.  So if you start a new scene and drag the MixedCameraRealityParent and Managers prefabs to the scene.  You'll see one set of mappings in the MCRP and one in the Gestures control.
Although to be honest, we need to go thorough all the prefabs and check.
One big issue with having so many different Input settings, is that you can't include them in an asset package. So the user either has to set them up manually or import the project Input assets file manually.
It might also be worth looking at integration with input assets like InControl (later)

I anyway have the opportunity to directly download from GitHub. Check out the project setup wizard.

Ahh found it.  Thanks!


[Accessibility]A11y_VSCode_GoToSymbolInFile_Keyboard:Tree view closes on pressing the "Tab " key .

Environment Details:
VS Code Version:1.24
OS Version:Windows10
Additional Details:
NVDA 2018.2,MAS3.2.1
Steps to Reproduce:
1.Launch VS Code.
2Navigate   to Go(Alt+G) menu option and select "Go To Symbol in File" from   dropdown or use Ctrl+shift+o.
3.Use the "Tab key" to navigate through the tree view.
Actual:
Tree view closes on pressing the "Tab " key
Expected:
Tree view closes on pressing the "Tab " key
"Recommendations:
Refer below link which is repository of bug fixes code snippets:
https://microsoft.sharepoint.com/teams/msenable/mas/pages/browse-fixes.aspx
[User] Impact:
User will not be able to use tree view.
MAS Reference:
https://microsoft.sharepoint.com/:w:/r/teams/msenable/_layouts/15/WopiFrame.aspx?sourcedoc={a4bbd37d-ee11-485e-b5c8-0539f0ac94b2}
Does this issue occur when all extensions are disabled?: Yes
Attachment for Reference:
[Accessibility]A11y_VSCode_GoToSymbolInFile_Keyboard.pptx

The same control is opened in many different scenarios (whenever user input is requested) and has the same tab behavior. I'm going to close all but one instance of this problem since the root cause is one control.
(https://github.com/Microsoft/vscode/issues/51854)


access_loggers/http_grpc: add request method and support additional headers config

Description:
Fills the in the request_method field of the HTTPRequestProperties contained by HTTPAccessLogEntry. Similarly, if the HttpGrpcAccessLogConfig specifies additional_request_headers_to_add or additional_response_headers_to_add, the matching headers are added the HTTPRequestProperties.request_headers and HTTPResponseProperties.response_headers fields.
Risk Level: Low
Testing: unit and integration tests
Docs Changes: n/a
Release Notes: n/a
Related: #743
Signed-off-by: Stephan Zuercher<EMAIL_ADDRESS>

@rodaine PTAL since this was on your list. :)
@zuercher maybe you already did this but please sync up with @rodaine so you don't end up doing duplicate work on the remaining gRPC logging TODOs.

@rodaine Sorry, didn't realize this was actively being worked on. I needed these parts to be implemented for something I'm working on outside envoy itself. I don't have any plans to do more of this, but I can try to make time and do more if that's helpful.


added logcat feature

Added logcat feature. Now user can view logcat for every single android device. closes #106

Thanks for contributing @gokul1099!


Changelog not appearing

VSCode Version: 1. She 7.2
OS Version: OSX

I'm not sure if this a mistake on my part but for some reason my change log isn't appearing in the extensions marketplace. Nor, incidentally, can I see anybody else change log despite them being present for other users.
my extension is https://github.com/mat-mcloughlin/vscode-elixir and as far as I can tell I've defined the CONTRIBUTIONS.md file correctly

Make sure to update vsce: npm install -g vsce.

yeah already did that I'm running 1.17.0

Hm ok, let me check.

Thanks man, just done some experimenting. Seems the changelog only appears if you have the extension installed. If you don't it says non is available

The asset is definitely not in the store:

But when I clone your repo, run vsce package and inspect the VSIXManifest, I do see it. It needs to work even from the store.
Did you perhaps add the CHANGELOG.md and republished without bumping up the version? You need to bump the version up before publishing.

OK something is definitely wrong here, there's some issue in the Marketplace.

@joaomoreno you're typing quicker than me. I checked the install/not installed with the beautify extension too and its the same

@hamenon Some asset types seem to not be properly indexed any more.
vsce adds a Microsoft.VisualStudio.Services.Content.Changelog asset to the VSIX manifest of this extension:
<?xml version="1.0" encoding="utf-8"?>
<PackageManifest Version="2.0.0" xmlns="http://schemas.microsoft.com/developer/vsx-schema/2011" xmlns:d="http://schemas.microsoft.com/developer/vsx-schema-design/2011">
  <Metadata>
    <Identity Language="en-US" Id="vscode-elixir" Version="0.1.1" Publisher="mjmcloug"/>
    <DisplayName>vscode-elixir</DisplayName>
    <Description xml:space="preserve">Elixir support for VSCode</Description>
    <Tags>Elixir,syntax,snippet,elixir,__ext_.ex,__ext_.exs,Eex,eex,__ext_.eex,HTML (Eex),__ext_.html.eex</Tags>
    <Categories>Languages</Categories>
    <GalleryFlags>Public</GalleryFlags>
    <Badges></Badges>
    <Properties>
      <Property Id="Microsoft.VisualStudio.Code.Engine" Value="^0.10.1" />
      <Property Id="Microsoft.VisualStudio.Code.ExtensionDependencies" Value="" />

        <Property Id="Microsoft.VisualStudio.Services.Links.Source" Value="https://github.com/mat-mcloughlin/vscode-elixir.git" />
        <Property Id="Microsoft.VisualStudio.Services.Links.Getstarted" Value="https://github.com/mat-mcloughlin/vscode-elixir.git" />

          <Property Id="Microsoft.VisualStudio.Services.Links.GitHub" Value="https://github.com/mat-mcloughlin/vscode-elixir.git" />

      <Property Id="Microsoft.VisualStudio.Services.Links.Support" Value="https://github.com/mat-mcloughlin/vscode-elixir/issues" />
      <Property Id="Microsoft.VisualStudio.Services.Links.Learn" Value="https://github.com/mat-mcloughlin/vscode-elixir" />

      <Property Id="Microsoft.VisualStudio.Services.GitHubFlavoredMarkdown" Value="true" />
    </Properties>
    <License>extension/LICENSE.md</License>
    <Icon>extension/images/logo.png</Icon>
  </Metadata>
  <Installation>
    <InstallationTarget Id="Microsoft.VisualStudio.Code"/>
  </Installation>
  <Dependencies/>
  <Assets>
    <Asset Type="Microsoft.VisualStudio.Code.Manifest" Path="extension/package.json" Addressable="true" />
    <Asset Type="Microsoft.VisualStudio.Services.Content.Details" Path="extension/README.md" Addressable="true" /><Asset Type="Microsoft.VisualStudio.Services.Content.Changelog" Path="extension/CHANGELOG.md" Addressable="true" /><Asset Type="Microsoft.VisualStudio.Services.Content.License" Path="extension/LICENSE.md" Addressable="true" /><Asset Type="Microsoft.VisualStudio.Services.Icons.Default" Path="extension/images/logo.png" Addressable="true" />
  </Assets>
</PackageManifest>

But when VS Code queries for it, it only gets these back:

I'm also having a similar issue (vscode commit 7499a9f3c565f4fbe5205e30c8664e5a5a11ae94). After waiting a couple minutes, the changelog does sometimes appear. In contrast to @mat-mcloughlin's experience, the extension did not have to be installed for it to sometimes, arbitrary appear after a while.
Also, if an extension has a changelog and it is loading asynchronously, you should replace the default message, "No Changelog available.", with "Loading Changelog..." (or equivalent).

We have someone from the team looking into it. Will update the thread once we identify the issue.

@joaomoreno I checked the call made by VS Code and don't see the asset type Microsoft.VisualStudio.Services.Content.Changelog specified in the query.

The code below seems to have it though..
https://github.com/Microsoft/vscode/blob/master/src/vs/platform/extensionManagement/node/extensionGalleryService.ts#L104

Oh I forgot we now list them explicitly over here! OK fix is on our side then. Thanks for the help!

@joaomoreno when can see this released?

Soon, either this or next week. It's already fixed in the Insiders.


Gaussian Sampling

According to the paper, the polynomials sampled must be from an error distribution, like the Gaussian Distribution. But in this implementation, the random polynomial seems to be sampled from a uniform distribution. Can someone explain why it was done so, or am I missing something here?

Dear Kalyan,
In the implementation our normal setting was {%25, %50, %25} for {-1, 0,
+1} and our implementation was like that. Later for a more generic B we
used the NTL's random number generator which is uniform. We should have
made a note in the code sorry for the confusion.
The main reason was the practicality. At the time of the implementation the
ciphertext required sampling size in the order of log{q}. There are some
algorithms such as ziggurat and I find some implementation of the
algorithm. However, it was causing the encryption to take too much time. In
order to test our codes in a timely manner we switch it to uniform. Now
there might be better Gaussian sampling algorithms which can be implemented
or ziggurat implementations that might faster than before.
An important note !! Another reason we did not pursue it is that it does
not effect security, actually it should be even more secure since you are
using more noise space. Furthermore, when you multiply 2 ciphertext with a
gaussian distribution, its noise distribution starts turning to uniform
distribution (basically there is a path from gaussian to uniform). In a few
multiplicative levels, the noise in a ciphertext actually becomes a uniform
distribution. The noise analysis still holds as well.
Since the above paragraph is true, for practical reasons we just used the
NTL's function.
I hope it is clear. Do you still need a gaussian? If so I might try and
find ziggurat.
Best,
Yarkin
On Thu, Feb 9, 2017 at 5:43 AM, Kalyan Kumar<EMAIL_ADDRESS>wrote:

According to the paper, the polynomials sampled must be from an error
distribution, like the Gaussian Distribution. But in this implementation,
the random polynomial seems to be sampled from a uniform distribution. Can
someone explain why it was done so, or am I missing something here?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vernamlab/DHS-LTV/issues/1, or mute the thread
https://github.com/notifications/unsubscribe-auth/AQ5A6aFDQQiV7qjibwfLGLdnE3jS-CoFks5rau3agaJpZM4L79-l
.

--
Yarkın Doröz
Master Student, Computer Science and Engineering Program
Sabanci University
Faculty of Engineering and Natural Sciences
Tuzla 34956, Istanbul, Turkey

Thank You so much Yarkin for the detailed answer, explaining everything so clearly :).
Actually, I am trying to implement modules that are trivially used in all Lattice-based cryptography primitives. I was unable to find a Discrete Gaussian sampler's implementation anywhere, finally ending up here. I would be really thankful if you know of any library or simple implementations of the primitives, so that any public key encryption schemes, functional encryption schemes could be conveniently implemented in software. Thank you again :)

There are other FHE implementations. One of them is Helib. Here is the link:
https://github.com/shaih/HElib
It seems there is a guassian sampling under NumTh.cpp. Not sure how
efficient, but I believe it should be since it is a decent library.
For Lattice operations, I think NTL is good since it has vector, matrix and
polynomial types with multiplication and addition operations.
On Fri, Feb 10, 2017 at 5:00 AM, Kalyan Kumar<EMAIL_ADDRESS>wrote:

Thank You so much Yarkin for the detailed answer, explaining everything so
clearly :).
Actually, I am trying to implement modules that are trivially used in all
Lattice-based cryptography primitives. I was unable to find a Discrete
Gaussian sampler's implementation anywhere, finally ending up here. I would
be really thankful if you know of any library or simple implementations of
the primitives, so that any public key encryption schemes, functional
encryption schemes could be conveniently implemented in software. Thank you
again :)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/vernamlab/DHS-LTV/issues/1#issuecomment-278903611,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQ5A6X3wWvMFDM5tiVFI97l1rBIfhDYlks5rbDUmgaJpZM4L79-l
.

--
Yarkın Doröz
Master Student, Computer Science and Engineering Program
Sabanci University
Faculty of Engineering and Natural Sciences
Tuzla 34956, Istanbul, Turkey

Thank You Yarkin. I am not aware of the Box-Muller method that is mentioned in that implementation. I will look into it.


Nightly 224 tests

To be tested for example:

Test playback of various contents offered in the demo.
Playlist management.
Control center integration.
AirPlay.
Error reporting.
Behavior with poor network conditions.

Not implemented yet:

End of playback behavior (when the last item has been played entirely).
User interface (e.g. automatic control toggling).

Remark:

Only URN-based medias have associated artworks.

Tests performed using Pillarbox Ntly 204 on Apple TV – some observations :

Play/Pause icon in shrink format (middle of the screen).
no dismiss of the control button.
no timeline visible, no scrubber (not possible to seek).
play/pause button on remote works only for pausing.
Showcase stories : player format is zoomed.
Playlist : impossible to choose manually.
at the end of a playlist, the arrow "->" is transformed in a white button.

Tests performed using Pillarbox Ntly 222 on iPhone/iPad - some observations :

dismiss controls (not hiding automatically).
once played (at the end) play button visible but not active.
Airplay icon in blue (should be blue when active...).
random button (playlist) only mix the list (already told to Sam).
no controls available (forward/backward, full screen, ...)

Tests performed using Pillarbox Ntly 224 on iPhone, casting via Airplay to an Apple TV 4K : some observations :

C3 Live DVR / SRF DVR : position of the scrubber on the seekbar is not fully at the right side, when playing live.
RSI Live : scrubber position and behavior when I go live on the iPhone.
spinning wheel very little on iPhone (while casting).
Stories : bottom bullets on Apple TV : small but OK, not a big difference between the current and the others, jiggling/virbrate.
Playlist : controls dismiss after 5 seconds; too long comparing to content ?
error messages (Token) don't show on TV when casting (only on iPhone).
when paused, seeking do not show (seek bar & scrubber) on TV.

Thanks for the tests. Some answers:

Tests performed using Pillarbox Ntly 204 on Apple TV – some observations :

Play/Pause icon in shrink format (middle of the screen).
no dismiss of the control button.
no timeline visible, no scrubber (not possible to seek).
play/pause button on remote works only for pausing.

No tvOS view layout has been implemented. We will use the default tvOS in the end anyway.

Showcase stories : player format is zoomed.
Playlist : impossible to choose manually.
at the end of a playlist, the arrow "->" is transformed in a white button.

The tvOS demos have received no dedicated attention yet.

Tests performed using Pillarbox Ntly 222 on iPhone/iPad - some observations :
dismiss controls (not hiding automatically).

We have only implemented a basic user interface. Gestures or timers to automatically hide controls are still missing.

once played (at the end) play button visible but not active.

That is something we need to discuss and implement, see #173.

Airplay icon in blue (should be blue when active...).

Improved in later nightlies.

random button (playlist) only mix the list (already told to Sam).

Yes, this is not a shuffle as in audio apps, rather a randomize button to reorder the current playlist randomly.

no controls available (forward/backward, full screen, ...)

Not implemented yet.

C3 Live DVR / SRF DVR : position of the scrubber on the seekbar is not fully at the right side, when playing live.

Should be the case.

RSI Live : scrubber position and behavior when I go live on the iPhone.

The stream is incorrectly seen as supporting DVR because of a packaging bug. Hopefully RSI will follow RTS and SRF and deliver only DVR livestreams in the future so that no ugly mitigation work is required from us, see #147.

spinning wheel very little on iPhone (while casting).

No work has been done on UI usability yet.

Stories : bottom bullets on Apple TV : small but OK, not a big difference between the current and the others, jiggling/virbrate.
when paused, seeking do not show (seek bar & scrubber) on TV.
Playlist : controls dismiss after 5 seconds; too long comparing to content ?

No special work has been made for tvOS demo layouts.

error messages (Token) don't show on TV when casting (only on iPhone).

@svenduvoisin I provided answers to what you found. I also linked to existing issues where appropriate. Thanks for your tests.


Operator should allow modification of tolerations

Right now it looks like the operator ignores taints for the controller nodes with no way to remove them.  I have isolated controllers that don't run workloads so I don't want this behavior and need a way to override the operator tolerations.

Issue was with the daemonset.  I will try next adding tolerations to the operator deployment next time I add the operator to see if that changes anything.


Incorrect owner symbol set to method symbol created in the TypeParamAnalyzer

The following line causes the enclosing function's symbol to be set as the owner for a method symbol. The owner should actually be the object type symbol for which we are creating this method symbol.
https://github.com/ballerina-platform/ballerina-lang/blob/dc7b1b1a4255e180c48c41c76ef817512c6e852f/compiler/ballerina-lang/src/main/java/org/wso2/ballerinalang/compiler/semantics/analyzer/TypeParamAnalyzer.java#L952

Fixed with #30580


Wrong github organization?

It doesn't seem like this has anything to do with PANDA, unless I'm missing something?
If it's unrelated, can we move it out of this org?

It's functionality for the guest agent.

Oh, that makes sense, thanks. I didn't see any panda references in this repo so I was confused.


NBSerializer: refactor, add tests

[x] Add notebook.metadata.id an UUID  that will be useful for identifying the same notebook  after a rename
[x] Add sparkNotebook, customSparkConf, customVars as notebook metadata keys
[x] Add tests for NBSerializer
[x] Simplify NBSerializer code

@maasg @andypetrella

@maasg
I also refactored copyNotebook.
Let's 🚀  as is without any API-braking changes (deserialize to return Try[Notebook]) - these will come as a separate PR.

okay let's :shipit:
Try will be indeed done in a separate PR, but I'm already happy since the expections aren't sucked!


CAMPD 508 Testing PI-8

Epic Benefit
Issues that require 508 testing in PI 8   @aprematta
Context
508 Compliance Checklist

Testing complete for 2698, 2696. Tested entire bulk data download page, found multiple issues:

There is no visual focus indicator (box) on the select all files checkbox (others have them)... it is getting keyboard focus.
Files Selected, Size text and values need to be in an aria live region so they are announced when updated
When clear all is activated, focus should be put back on the data type dropdown
"There are no records to display" needs to be in an aria live region so it is announced when filters are updated
There should be text or a tooltip saying that certain filter selections will display new filter options
Couldn't test the file selection limit of 50GB message.... make sure that is also in an aria live region
Filters disappear at 150% zoom

Testing complete for 2697, the code of the hidden label and actual search input matches that of the search in the popup menu. To remedy, change the for and id attributes of the new search label/input so they are different from the one in the menu. I would also suggest changing the hidden label to be different, perhaps 'Search Bulk Data Files Table'

No issues found with 2809

Testing complete for 2835....... none of the new functionality is keyboard accessible. This will need to be added then reviewed again fully for 508 compliance. Unable to do any screen reader testing until keyboard accessibility is implemented.

Testing complete for 2796..... 2 issues found:

The icons next to Data and Maps & Graphs in the main content can be considered decorative. Currently they have repetitive alt text alt="Data Icon" for example. Change both to the empty alt tag alt=""
The Data and Maps & Graphs content in the main section are now each contained in a layout table but their is data table markup. Either remove the layout tables and position in some other method, or add a role of "Presentation" to the table tag and change all TH cells to TD.

No issues found for 3364

Testing again for 2796.... only issue is the 3 icons next to the headings are decorative so they should have the empty alt tag "" instead of current: img src="/images/icons/cdd-icon.svg" alt="Custom Data Download Tool Icon"

Testing complete for 2758, couple of issues found:

Tab order is incorrect, the tooltip for select all files comes before the Keyword search text box and search button. Need to adjust the order in the dom to put the search functionality above the tooltip.
When the filters button is selected, there is no focus shift. It needs to be put on the first focusable item in the filters window.
When the filters window is active, there is no keyboard trap, I should not be able to get to the other items on the web page until the window is dismissed.
When I hit the back or preview data buttons, focus is not shifted, it should be placed back on the filters button

No issues found on 3405

Testing complete for 3355, ANDI is indicating that there is an ARIA Hidden = True when the modal is active, hiding it from a screen reader. Looking at the code, I think it's this from the HTML:
div id="root" aria-hidden="true" data-modal-hidden=""

Testing complete for 3356, no issues found

Testing complete for 3357, no issues found

Testing complete for 3420, same issue as the desktop view https://github.com/us-epa-camd/easey-ui/issues/3553

Closing epic. Another 508 epic will be opened by @MaeganWood64 for PI 9


Multi Master possible ?

Hello, I would like to create a cluster K3S with multi-master to have more avaibility. For exemple, if my first master goes down, my cluster still working with the same option/posibility. Do you know if it's possible ? I have already install K3S on 3 raspberry and all works well but with only 1 master.
I have already done with https://github.com/kelseyhightower/kubernetes-the-hard-way on my raspberry but I want to try with K3S to compare the performance.
Thank you in advance

HA / etcd is not currently implemented; see #42.

Closing in favor of #42


/task command to  show assignee task details

Create a command /task to show the task details of the assignee
Overview
For tracking the assigned task of the user, we are using going to use /task username . It will help to check the status of the task of an assignee and get complete details of it.
description

This command should be run by SuperUser
It should take username as an argument
It will return
Title: Create /task command
Status: ASSIGNED
Task Link: More Info    (https://status.realdevsquad.com/tasks/:taskId)
End Date: 21-08-2023
Task Link will allow us to see completed detail of the task
pagination should be added if there is data more than 10

Sub tasks

the status option should be added to filter the assigned tasks status, assigned,inprogress, completed, etc.
limit and skip options could also be considered (optional )

Acceptance creteria

80% test case covered

The ETA to submit this is 25 aug 2023


when delegatebw<PHONE_NUMBER> Error 3100001 occurred

[root@server3 hh]# /disk/gpsio/clgps -u http://localhost:8020 push action gpsio delegatebw '{"from":"'gps1'", "receiver":"'gps1'", "stake_net_quantity":"5000000000.0000 CPS", "stake_cpu_quantity":"5000000000.0000 CPS", "transfer": false}' -p gps1
Error 3100001: Internal state is no longer consistent
Error Details:
Casting a higher bit integer value 0xe6feff7f8bd28c20eb01000000000000 to a lower bit integer value which cannot contain the value, valid range is [0x00000000000000000000000000000000, 0xffffffffffffff7f0000000000000000]

Cleaning up old issues so we can focus on more recent open items. Please reopen this ticket if needed. Thank you.


Dynamic parameter list

I see that the stored procedure name can be dynamic, and the stored procedure parameter values can be dynamic. But I don't know if the number of parameters, or parameter names can be dynamic. It seems to me, if you allow the sproc name to be dynamic, then the parameter names and number of parameters should also be able to be dynamic.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 45e0996a-5a15-b03a-f204-b8725579109d
Version Independent ID: 6c8ed85a-8e40-a7b2-c13a-81123aa499d4
Content: Transform data by using the Stored Procedure activity in Azure Data Factory
Content Source: articles/data-factory/transform-data-using-stored-procedure.md
Service: data-factory
GitHub Login: @douglaslMS
Microsoft Alias: douglasl

@jeffhebrank Yes, you provide whatever parameters your stored procedure requires. Can you let us know what you found misleading, or what made you question whether this was the case? We'd like to  make the page easier to understand, if possible.
@linda33wj

Lets say my dynamic stored procedure is either going to be sprocA or sprocB.  aprocA takes 1 parameter, while sprocB takes 2 parameters. I don't see a way to accomplish this via the data factory UI. The number of parameters and the names of the parameters to not appear to be dynamic.

@jeffhebrank OK, excuse me if I missed your point on first reading. I get it now, and I'll have to ask someone who's more of a subject matter expert. I suspect that this is indeed not possible in the UI, but it may be possible in another way.

@jeffhebrank what's your logic to determine which sproc to call? Seems you are trying to create a single Stored Proc activity serving as template, Instead, will it work if you do e.g. IfCondition activity (https://docs.microsoft.com/en-us/azure/data-factory/control-flow-if-condition-activity) upfront, then fork to execute SprocA activity or execute SprocB activity - two separate Stored Proc activities?

@jeffhebrank thanks for the confirmation. A related note, though the UI doesn't support it natively, I think you can make the entire SP parameters object as an expression, by editing the JSON in the Code directly. The UI may interpret such expression wrongly for now but runtime should work.


Use SAP HANA as the Database for a CAP Java Application

Tutorials: https://developers.sap.com/tutorials/cp-cap-java-hana-db.html
I have completed all the previous steps for tutorial, but am unable to complete this stage.
I am able to set my CF endpoint, and login in using my SAP credentials (using trial account).
I have installed @sap/hdi-deploy and updated my .cdsrc.json
but when I run CDS deploy:
user: bookstore $ cds deploy --to hana:bookstore-hana
[deploy] - Starting deploy to SAP HANA ...
[deploy] -
[deploy] - Creating build tasks
[cds] - determining build tasks for project [/home/user/projects/bookstore].
[cds] - the following build tasks will be executed
[cds] -    {
     "build": {
       "target": ".",
       "tasks": [
         {"for":"java-cf", "src":"srv", "options":{"model":["db","srv"]}}
       ]
     }
   }

[deploy] - Running build
[cds] - building project [/home/user/projects/bookstore], clean [true]
[cds] - cds [5.8.3], compiler [2.15.4], home [/extbin/npm/globals/lib/node_modules/@sap/cds]

[cds] - done > wrote output to:
   db/src/gen/.hdiconfig
<truncated>
   db/src/gen/sap.common.Currencies_texts.hdbtable

[cds] - build completed in 456 ms

[deploy] -
[deploy] - Using container bookstore-hana
[deploy] - Getting service bookstore-hana
[ERROR] CF-ResourceNotFound: Creation of service key failed (10010)

I have tried creating the database manually, and sharing the database into my dev space with no change.
Is there a specific set of instructions I could follow to resolve this?

Hi @manjuX
Thanks for the suggestion!  I don't have any any DB instances at the moment and the same problem occurs.  I've tried the following to resolve:

follow the tutorial as written, believing it will create the database and connect it for me (via CF).  I was able to inspect that the DB was there, and was configured as expected but still fails at service key creation.
created the database manually, and having multiple databases, including the one created by CF.
creating a database, deleting all other instances.

Looks to be the same result in all cases.
Is there a set of explicit manual steps I can follow to replicate what CF is doing that might help with diagnosis?

Hi @jcarter-sap ,
For clarity, In the third case did you create a Hana Instance in Hana Cloud Central and had only one database there?
Because when you run the command it creates a "HDI container" in an existing Hana instance.
Here is an example of the two types when I run cf services:

Here cpapp-db is the hdi-container that got automatically created for me in my cpapp database instance.
Best regards,
Manju

Hi,
Thanks for your help -- I've unblocked myself.
The link you posted previously allowed me to move forward.  I created the database manually, and then created an cdi-shared instance linked via the database UUID.
I think this step is missing from the tutorial, or in most cases does it happen automatically?


Docker compose shutdown before startup

This addresses bob run tasks failing if there are still some running containers on the user's environment.
Originally the user needed to stop+rm all running containers that had port conflicts manually. This PR automates this.

I'd rather have less magic involved as this could potentially interfere with other parts of the user system. Closing it for now.. but we might reconsider later.


How to delete file from S3 bucket

does the library support or we need to use boto3 client delete_ object.

This sounds like a reasonable feature. Now it's possible with:
model.filefield.delete()

on S3 storage will be storage.delete() or something else

Yeah it exists on the storage too. storage.delete(name)

Any guide how to upload  to a specific folder on S3 bucket.

I think you are looking for https://github.com/aminalaee/fastapi-storages/issues/23 which is not implemented yet. PRs are welcome so feel free to pick it up.


random_numbers: 0.3.1-0 in 'melodic/distribution.yaml'

upstream repository: https://github.com/ros-planning/random_numbers
release repository: https://github.com/ros-gbp/random_numbers-release.git
distro file: melodic/distribution.yaml
bloom version: 0.6.2
previous version for package: null

random_numbers

no changes

I think you'll need to cut a new release so that your email for the buildfarm email rather than the previous maintainer.
Looking at the release repo it looks like the release you made for melodic is 0.3.1-0 and not 0.3.1-1. You can use bloom to generate the PRs automatically and it will pick up the exact version number (X.Y.Z-Inc)

both good points. Closing this one.


Switch error when disabling attachments on ntfy-sh service

Describe the bug
services.ntfy-sh.settings.attachment-cache-dir cannot be made blank or Null without error, and thus attachments cannot be disabled on the instance (without activation errors)
when option is empty this error is encountered whilst setting up tmpfiles
/etc/tmpfiles.d/00-nixos.conf:25: Path '0700' not absolute.
due to this line:
https://github.com/NixOS/nixpkgs/blob/56528ee42526794d413d6f244648aaee4a7b56c0/nixos/modules/services/misc/ntfy-sh.nix#L89
Steps To Reproduce
Steps to reproduce the behavior:
set services.ntfy-sh.settings.attachment-cache-dir = "";
Expected behavior
switch can be performed without error
Additional context
this value should be blank or null (equivalent an yaml) to disable attachments
https://docs.ntfy.sh/config/#config-options
Notify maintainers
@happysalada @McSinyx
Metadata
 - host os: `Linux 6.1.80, NixOS, 23.11 (Tapir), 23.11.5541.56528ee42526`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `"nixos-23.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

It looks like this should be made null or string.
Im a little short on time to make a PR. If you do have time to make a PR, i would be happy to help you get it merged.
Thank you for taking the time to report !

ill have to familiarize myself with development on nix, and i should do that, ill say something when/if i get to it


Leveling tracker error message

I finished the campaign and went to Oriath Shores. The tracker signaled it can't go further, so I clicked its button to close it. After it closed, this error message started appearing repeatedly, every second or so. It would keep appearing as long as the tracker was enabled and closed.

Ugh, that's an oversight from implementing the new hint structure from exile leveling. The "target location" that triggers page-switching doesn't get cleared on the final panel, so the script registers reaching that location over and over again.


Add Windows Server 2022 image templates

Description
Windows Server 2022 has recently become available in Azure so we are preparing virtual-environments source code to support it.
https://docs.microsoft.com/en-us/windows-server/get-started/windows-server-release-info
Related issues: https://github.com/actions/virtual-environments/issues/3138 and https://github.com/actions/virtual-environments/issues/3609

/azp run windows2016, windows2019, windows2022

/azp run windows2016, windows2019, windows2022

/azp run windows2016, windows2019, windows2022


Made correction to DialogList documentation

A typographic error was made in the documentation for the DialogList component under the 'Prepare your model' section.
IDialog was written as IDealog. This pull request corrects that error.

Hello,
considering that the library is not maintained and I actively use it, I have decided to fork it for the time being under the same name. Changes to naming and other practices will come in the future.
In the mean time, I have merged your PR and appreciate your contribution.
I hope we can revive this project together.
https://github.com/mario/chatkit
Cheers,
Mario


Arabic text does not align right correctly

When rendering Arabic text (aligned to the right), each row of text is not aligned. Each row is offset.
ArabicExample.zip
Attached is an Arabic SVG file which demonstrates the issue, and a test render made with resvg. It's also possible I'm just doing something wrong, so I apologize if that is the case.
This was rendered with the Qt back-end, on Ubuntu 16.04, and uses a version of resvg compiled from source. Please let me know if I can provide any more details.

I also tried multiple fonts, and all misaligned with Arabic.

On second glance, this might also be related to a filter on the text elements, which appear to affect the bounding box around the text. Just a guess.

It works fine on my machine. (c)
I have Qt 5.11.3. I will try to reproduce it on Ubuntu. It might me Qt/fontconfig/freetype bug.

Was able to reproduce on Ubuntu 16.04.
By the way, 16.04 has Qt 5.5.1 and resvg only support Qt >= 5.6.

As expected, QFontMetrics::width() returns different numbers. Not sure why, but this is the source of the bug.

The cairo backend works, so it's Qt/Qt-backend bug.

I've build Qt 5.6.3 on Ubuntu 16.04 and it fixed rendering. So it's a Qt bug and I'm closing this as out of scope.

By the way, there is also a BIDI reordering bug (QTBUG-72890) in Qt, which will affect an Arabic text rendering too. It can't be fixed on the resvg side, but the next release will feature it's own text rendering engine, so it will be fixed semiautomatically.

Thanks for looking into this! I'll try rebuilding with a more recent build of Qt.


docs(oas-config) document oas-config Admin API endpoint

Add documentation for the /oas-config Admin API endpoint. Include an example usecase.

My recommendation for the introduction would be:
"This Admin API endpoint allows you to read an OpenAPI Spec into Kong to create the appropriate Services and Routes in an automated way."
In my opinion a softer and better way to introduce this :).


Re-implement CSV record reader to skip unparseable lines

Removed skipUnParseableLines flag and always skip un-parseable lines
Add stopOnError flag to stop reading records on error
Support all formats in commons-csv
Also fix a bug in RecordReaderFileConfig to not access record reader when it is already closed

The new record reader is able to handle multi-line values as well as un-parseable lines. When a line cannot be parsed, it resumes from the next line.
More importantly, it doesn't throw exception in hasNext(), and the exception is cached and thrown when calling next(). This aligns with the high level abstraction of ingestion flow.

cc @rajagopr @suvodeep-pyne who recently updated the record reader

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.00%. Comparing base (59551e4) to head (2e09422).
Report is 1293 commits behind head on master.

:exclamation:  There is a different number of reports uploaded between BASE (59551e4) and HEAD (2e09422). Click for more details.
HEAD has 53 uploads less than BASE

Flag
BASE (59551e4)
HEAD (2e09422)

integration
7
1

integration2
3
1

temurin
12
1

java-21
7
1

skip-bytebuffers-true
3
0

skip-bytebuffers-false
7
1

unittests
5
0

unittests1
2
0

java-11
5
0

unittests2
3
0

integration1
2
0

custom-integration1
2
0

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #14396       +/-   ##
=============================================
- Coverage     61.75%    0.00%   -61.76%
=============================================
  Files          2436        3     -2433
  Lines        133233        6   -133227
  Branches      20636        0    -20636
=============================================
- Hits          82274        0    -82274
+ Misses        44911        6    -44905
+ Partials       6048        0     -6048

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <ø> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <ø> (ø)

java-11
?

java-21
0.00% <ø> (-61.63%)
:arrow_down:

skip-bytebuffers-false
0.00% <ø> (-61.75%)
:arrow_down:

skip-bytebuffers-true
?

temurin
0.00% <ø> (-61.76%)
:arrow_down:

unittests
?

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

When should someone set stopOnError  to true?
if set to false, will we still get exception.

@KKcorps when stopOnError is set to true, the record reader will stop parsing when encountering error, and return false in hasNext(). When it is not set, hasNext() will return true, and getNext() will throw the exception, which can be handled by the index creator to either abort, or fill default value and continue when continueOnError is set.


Add (OPTIONAL!) support for ECS

See RFC 7871.
This potentially leaks enough personal info that not everyone will want it.

Looked into the code:
https://github.com/hildjj/dohdec/blob/96942650d152a2817ffad48fa1000968a2990751/lib/dnsUtils.js#L71-L76
Am I right it would only requires additional opts.ip to be included to support ECS?

Looked into the code:
https://github.com/hildjj/dohdec/blob/96942650d152a2817ffad48fa1000968a2990751/lib/dnsUtils.js#L71-L76
Am I right it would only requires additional opts.ip to be included to support ECS?


Fix: Crashing Pokemon

Certain teams/Pokemon crash either the matchup screen or the game screen (investigate Galarian Stunfisk). Investigate crash logs and fix the issue

Crashing in game should be gone. idk if crashing is gone in the matchup screen


<EMAIL_ADDRESS>update version and fix checkver logic

The amazon-workspaces version check was validating the version by pinging the winsparkle appcast config file for the x86 version. The 32-bit version is no longer updated by Amazon, so to make scoop work properly, the url had to be updated to point to the x64 version.
The documentation references the x86 url only so I had to decompile the workspaces.dll file to find the correct filename.
public string GetAppcastFileName()
{
	if (Environment.Is64BitOperatingSystem)
	{
		return "WorkSpacesAppCastx64.xml";
	}
	return "WorkSpacesAppCast.xml";
}

The download url is composed out of:
https://s3.amazonaws.com/workspaces-client-updates/<environment>/<region>/<platform>/WorkSpacesAppCastx64.xml
Combining this information makes the new url for 64-bit only:
https://s3.amazonaws.com/workspaces-client-updates/prod/global/windows/WorkSpacesAppCastx64.xml
For reference, the download url for the outdated 32-bit version of amazon workspaces client is:
https://d2td7dqidlhjx7.cloudfront.net/prod/pdt/windows/x86/Amazon+WorkSpaces.msi

[x] I have read the Contributing Guide.

/verify

Move the url and hash inside architecture block too

/verify


BUG: Previous processed commitment's progress is erroneously displayed during create new commitment flow

After processing a successful commitment (for 19 miles), I started through the flow to create a new commitment (for 30 miles). I made the new commitment (see the tx here, and I was taken to the track progress view. However, in that view I am being shown my progress from the 19-mile commitment that I already completed.
Expected behavior: view progress (which should be 0 at this point after having just created the commitment) for my current commitment
Actual behavior: viewing progress for my previous, already-completed commitment. See screenshot below.

Related: clicking Back takes me back to the create commitment view, and triggers a metamask tx popup

Refreshing and reconnecting to metamask brings me to my profile page. Clicking Continue then shows me the correct commitment progress.
However, clicking Back from that screen (to return to the profile page) and then Continue brings me to the create commitment view with these toasts


Faster to import std.regex, by lazy eval of wordMatcher

After:
▶ time dmd test_case.d
dmd test_case.d  0.24s user 0.03s system 98% cpu 0.269 total
▶ gco master
Switched to branch 'master'
Before:
▶ time dmd test_case.d
dmd test_case.d  1.10s user 0.12s system 99% cpu 1.228 total

Woohoo!

Credits go to @andralex actually

Before:

After:

Awesome!

Looks like std.uni is the next candidate for improvement. :-P

@CyberShadow would be so nice to have that kind of information nicely visible on the website! "The upfront time cost of importing std modules"


attributes are empty after .save() call on a model from belongsTo relation

basically I have a code like this:
let model = something.related('some'); /* belongsTo */
await model.save(newData, options);

the create/update request is then performed successfully but when I check any of the attributes on that model - they are empty, including id
currently my workaround solution for this is
let model = something.related('some');
let tempModel = await new model.constructor(newData).save(null, options);
model.attributes = tempModel.attributes;
model.relations = tempModel.relations;

The project leadership of Bookshelf recently changed. In an effort to advance the project we close all issues older than one year.
If you think this issue needs to be re-evaluated please post a comment on why this is still important and we will re-open it.
We also started an open discussion about the future of Bookshelf.js here https://github.com/bookshelf/bookshelf/issues/1600. Feel free to drop by and give us your opinion.
Let's make Bookshelf great again


CSCETSIN-302: Display PAS datasets

For description see https://wiki.eduuni.fi/display/cscetsin/PAS+datasets+in+Etsin
This feature branch was branched from qvain-light, which explains the messy commit history on this page.
Actual commits that differ from test start from c2669bb. There seems to be 29 commits in total, plus one merge from test.
Related to: https://github.com/CSCfi/etsin-finder-search/pull/49

Here are some suggestions that could be changed before the merge. (Based on best practices and stuff that I try to take into consideration when I refactor old code in Etsin and Qvain-light.)

Use compoentDidMount instead of componentWillMount. componentWillMount is deprecated and will be removed in next reactjs version.
For example, < PasContainer /> component and possibly other components like that one should if possible use one of the ready-made container components or extend one of them. This is just so that if we have made one container component responsive for different screen sizes, it would be good to reuse it.
If a component does not need to be a stateful class component, it should be a stateless functional component. So if there is no need to use lifecycle methods like componentDidMount ore use state, then the component . should be a plain function. Also, if only state is needed Hooks could be used.

btw. good that you fix typos when you notice them 👍🏻

Thank you. All very good suggestions for better development practices. I will fix the identified problems.

Use compoentDidMount instead of componentWillMount. componentWillMount is deprecated and will be removed in next reactjs version.

1fcc63d: The one existing reference to componentDidMount is now removed.

For example, < PasContainer /> component and possibly other components like that one should if possible use one of the ready-made container components or extend one of them. This is just so that if we have made one container component responsive for different screen sizes, it would be good to reuse it.

6f5fce4: Static width removed from the . The search bar, found above the PAS checkbox, does not use a ready-made component or extend one of them, but is still responsive. So, the functionality previously found within the  now uses the same syntax as the search bar, to make it responsive in the same way as the search bar.

If a component does not need to be a stateful class component, it should be a stateless functional component. So if there is no need to use lifecycle methods like componentDidMount ore use state, then the component . should be a plain function. Also, if only state is needed Hooks could be used.

bc18fdf: The new FairDataPasDatasetIcon component was created unnecessarily as a stateful component, but is now a stateless component.


Adding forecasting backtest/evaluate function

Is your feature request related to a problem? Please describe.
I would like to add a backtesting class. We should ideally then also implement an expanding window splitter functionality as discussed in #552 . But the Backtest would just receive a splitter as an argument.
Describe the solution you'd like
A class like Backtest (naming to be discussed) with a signature like this:
class Backtest(object):
    """Backtest forecasters

    Parameters
    ----------
    forecaster: estimator object
    splitter: splitter object
        e.g. SlidingWindowSplitter
    score: function
        A function like smape_loss that has y_pred, y_test as arguments and returns a float

    Returns
    ----------
    error: float
        Average error of fitted estimators calculated by means of the score function
    """

The class should have a fit() function like a regular estimator object. I guess a predict() function does not make sense here. The class will be able to also take a forecaster of type ForecastingGridSearchCV, as in production people might refit and tune the model by progress in time.
Optionally, the class can print the number of fits, like in ForecastingGridSearchCV when fitting.
I am not sure on how to structure the return and what else should be returned, I guess just return self and have the rest as class variables? We could do sth like in ForecastingGridSearchCV with the cv_results_ variable, so we could also store there the errors, predictions  and also the params of each iteration (because params might differ when using a grid search as forecaster).

Where should a class Backtest be placed in the sktime class structure? I propose sktime/sktime/forecasting/model_backtest/_backtest.py.

we could actually also have a predict() function that just takes the last fitted estimator and calls its predict() then. This might be useful in productive scenarios, so when you deliver a productive model you can just use a Backtest forecaster and deliver the backtested error together with the next prediction. So you dont have to do both separately.

we could actually also have a predict() function that just takes the last fitted estimator and calls its predict() then. This might be useful in productive scenarios, so when you deliver a productive model you can just use a Backtest forecaster and deliver the backtested error together with the next prediction. So you dont have to do both separately.

@aiwalter I'm not familiar with this type of class, could you describe its purpose a bit more, what's the use case, what's the user journey/workflow you have in mind?

@aiwalter I'm not familiar with this type of class, could you describe its purpose a bit more, what's the use case, what's the user journey/workflow you have in mind?

@mloning Here is one example, but I am not sure how this is different from performing validation using SlidingWindowSplitter in sktime.
I think one difference may be that in backtesting, you would refit the model in at every split which is currently not implemented in sktime per https://github.com/alan-turing-institute/sktime/issues/621#issuecomment-760228618. I would also be interested in hearing more thoughts from @aiwalter on the backtesting topic.

@mloning Here is one example, but I am not sure how this is different from performing validation using SlidingWindowSplitter in sktime.
I think one difference may be that in backtesting, you would refit the model in at every split which is currently not implemented in sktime per https://github.com/alan-turing-institute/sktime/issues/621#issuecomment-760228618. I would also be interested in hearing more thoughts from @aiwalter on the backtesting topic.

@mloning @ngupta23
Actually the Backtest class would be similar to GridSearchCV. However I tried to use grid search to do a backtest and didnt get the expected result. Lets start with what I want to do and the worklflow of Backtest, then below I will give the example I tried with GridSearchCV.

Assumptions
Lets do some assumptions for an example to make it cleear:

We take the airline data from the notebook example (load_airline()) with freq=M
I want to build a model with fh=[1,2,3,4,5,6,7,8,9,10,11,12], so basically forecast all months of the next year on the end of each year.

Workflow:
With this assumptions I want to know how well my trained model is performing, so I can do a train-test-split and take the last 12 months as test data. However, this can tell me only how the model would have performed in the last year. If I take now a bigger test data with last 24 months, I violate my assumption of only forecast 12 month (and the additional 12 months will have also a worse error, so this will be a bias). Here comes the need for a Backtest class (and also an expanding window splitter), because e.g. I want to backtest my model on last 5 years. So here a sketch on this example, with an expanding window splitter:

Within this 5 model fits, I also want also to be able to do a grid search. This is because in production I also would most likely do it after each year because I have new data and might find better hyperparameters.

Example using GridSearchCV to backtest:
Now what I tried so far.

cv = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=12)

gscv = ForecastingGridSearchCV(
    forecaster=TBATS(), param_grid={"sp": [12]}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)

In case I dont want to do grid search but just use the SlidingWindowSplitter, I have to anyway provide param_grid={"sp": [12]} which looks a bit strange, because I would like to just do TBATS(sp=12) without providing param_grid values. I figured out I can just do param_grid={} to get it running. The main point is however rather the cv_results_, where I would like to see 5 rows now because I have fitted 5 models and I need to exactly know how each of them has performed. Further I would like to get the average error for the 5 years, this is what is currently returned in this 1 row I think. The Backtest class could then also have plot function to make a nice plot of the backtest.
Btw can s.o. explain me the initial_window argument of SlidingWindowSplitter in context of my example? 😃  I am not sure if I have set this value correctly here?
In case I want to do grid search, I would have to nest two grid searches:
cv_1 = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=1)

cv_2 = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=12)

gscv_1 = ForecastingGridSearchCV(
    forecaster=TBATS(), param_grid={"sp": [6, 12], "use_box_cox": [True, False]}, cv=cv, scoring=sMAPE()
)

gscv_2 = ForecastingGridSearchCV(
    forecaster=gscv_1, param_grid={}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)

Btw I am wondering why there is no use_box_cox in the params column?
So long story short: From UX and what I would like to have as return it would be nice to have a Backtest class. Backtesting with ForecastingGridSearchCV is not the best idea I think. Also I think what @fkiraly said in #552 makes sense to me:

we should be careful and not mix evaluation and tuning. Tuning are wrappers/compositors like GridSearchCV in sklearn; evaluation should be external to any estimator.

Happy to get your feedback.

@mloning @ngupta23
Actually the Backtest class would be similar to GridSearchCV. However I tried to use grid search to do a backtest and didnt get the expected result. Lets start with what I want to do and the worklflow of Backtest, then below I will give the example I tried with GridSearchCV.

Assumptions
Lets do some assumptions for an example to make it cleear:

We take the airline data from the notebook example (load_airline()) with freq=M
I want to build a model with fh=[1,2,3,4,5,6,7,8,9,10,11,12], so basically forecast all months of the next year on the end of each year.

Workflow:
With this assumptions I want to know how well my trained model is performing, so I can do a train-test-split and take the last 12 months as test data. However, this can tell me only how the model would have performed in the last year. If I take now a bigger test data with last 24 months, I violate my assumption of only forecast 12 month (and the additional 12 months will have also a worse error, so this will be a bias). Here comes the need for a Backtest class (and also an expanding window splitter), because e.g. I want to backtest my model on last 5 years. So here a sketch on this example, with an expanding window splitter:

Within this 5 model fits, I also want also to be able to do a grid search. This is because in production I also would most likely do it after each year because I have new data and might find better hyperparameters.

Example using GridSearchCV to backtest:
Now what I tried so far.

cv = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=12)

gscv = ForecastingGridSearchCV(
    forecaster=TBATS(), param_grid={"sp": [12]}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)

In case I dont want to do grid search but just use the SlidingWindowSplitter, I have to anyway provide param_grid={"sp": [12]} which looks a bit strange, because I would like to just do TBATS(sp=12) without providing param_grid values. I figured out I can just do param_grid={} to get it running. The main point is however rather the cv_results_, where I would like to see 5 rows now because I have fitted 5 models and I need to exactly know how each of them has performed. Further I would like to get the average error for the 5 years, this is what is currently returned in this 1 row I think. The Backtest class could then also have plot function to make a nice plot of the backtest.
Btw can s.o. explain me the initial_window argument of SlidingWindowSplitter in context of my example? 😃  I am not sure if I have set this value correctly here?
In case I want to do grid search, I would have to nest two grid searches:
cv_1 = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=1)

cv_2 = SlidingWindowSplitter(
    initial_window=84,
    window_length=12,
    step_length=12)

gscv_1 = ForecastingGridSearchCV(
    forecaster=TBATS(), param_grid={"sp": [6, 12], "use_box_cox": [True, False]}, cv=cv, scoring=sMAPE()
)

gscv_2 = ForecastingGridSearchCV(
    forecaster=gscv_1, param_grid={}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)

Btw I am wondering why there is no use_box_cox in the params column?
So long story short: From UX and what I would like to have as return it would be nice to have a Backtest class. Backtesting with ForecastingGridSearchCV is not the best idea I think. Also I think what @fkiraly said in #552 makes sense to me:

we should be careful and not mix evaluation and tuning. Tuning are wrappers/compositors like GridSearchCV in sklearn; evaluation should be external to any estimator.

Happy to get your feedback.

@aiwalter , Here are my initial thoughts and understanding on this.
I think your use of initial_window is more or less Ok (you may want to use 72 instead of 84 though because of the following) -

For each of the candidate parameters in the param_grid, the model is trained on the initial_window first.
Then the window_length of data is appended to the initial_window data (essentially expanding window, if you use start_with_window=True) and the prediction is made which is used for evaluation. So in your example, data will be expanded from 72 to 84 and then future prediction will be made beyond that and those will be used for evaluation.
This process is repeated until you reach the end of the data (keep appending window length of data and making prediction and evaluation.

There are a few caveats though based on my analysis.

I found that in some (maybe all?) cases, it is only doing 1 step ahead prediction in the expanding window phase. When I tried adding a forecast horizon of 12 points, it gave an error that this is not supported yet.
When the window is expanded, in some cases (maybe all?), the model parameters are not refit. It retains the same model parameters that were learned in the initial_window. See discussion on this topic here (https://github.com/alan-turing-institute/sktime/issues/621). I think for backtesting, you want to refit it at each expanding window.

Also, regarding the following...

Btw I am wondering why there is no use_box_cox in the params column?

The params column will only show the params that you have provided in the grid search. Default/implicit parameters are not shown in cv_results_.

@aiwalter , Here are my initial thoughts and understanding on this.
I think your use of initial_window is more or less Ok (you may want to use 72 instead of 84 though because of the following) -

For each of the candidate parameters in the param_grid, the model is trained on the initial_window first.
Then the window_length of data is appended to the initial_window data (essentially expanding window, if you use start_with_window=True) and the prediction is made which is used for evaluation. So in your example, data will be expanded from 72 to 84 and then future prediction will be made beyond that and those will be used for evaluation.
This process is repeated until you reach the end of the data (keep appending window length of data and making prediction and evaluation.

There are a few caveats though based on my analysis.

I found that in some (maybe all?) cases, it is only doing 1 step ahead prediction in the expanding window phase. When I tried adding a forecast horizon of 12 points, it gave an error that this is not supported yet.
When the window is expanded, in some cases (maybe all?), the model parameters are not refit. It retains the same model parameters that were learned in the initial_window. See discussion on this topic here (https://github.com/alan-turing-institute/sktime/issues/621). I think for backtesting, you want to refit it at each expanding window.

Also, regarding the following...

Btw I am wondering why there is no use_box_cox in the params column?

The params column will only show the params that you have provided in the grid search. Default/implicit parameters are not shown in cv_results_.

@aiwalter, thanks for your suggestion - I think having robust functionality for quick evaluation and backtesting is a great idea.
However, in summary of my answer below:

I do not think backtesting should be a class like GridSearchCV, butpart of  an "external" benchmarking module
it would be great if you would like to contribute to building that!

So, why should backtesting not be a class like GridSearchCV?
Because backtesting, or more generally performance evaluation workflows, are not the same as tuning strategies.
A tuning strategy still is a forecaster/estimator, but a performance evaluation workflow isn't. A forecaster is subjected to a performance evaluation workflow. Both may use re-sampling (e.g., cross-validation or temporal re-sampling), but still they aren't the same kinds of formal objects.
For the same reason, sklearn's GridSearchCV should not be used for benchmarking or performance evaluation, because instances of it are representations of estimators, not evaluation workflows.
Some more detail on this, in my opinion, crucial distinction between tuning and evaluation workflows can be found in Section 5.1 of this paper on AI toolbox software design: https://arxiv.org/abs/2101.04938 (compare: anti-pattern 4; we seem to have been pre-scient here :-) )
Somewhat confusingly, the same re-sampling technique can be used in a tuning workflow as well as in an evaluation workflow.
For example, cross-validation is used inside GridSearchCV - to obtain performance estimates which are then used in tuning - and in evaluation workflows - to obtain performance estimates.
In terms of software design, the former should be a class (with a higher-order scitype), but hte latter shouldn't be, even if both use the same re-sampling technique.
Evaluation should be on the workflow layer, and calls subject to something like
benchmark(estimator, Backtesting(config), metrics)
or
Benchmarking.run() (after construction with config setting)
Does this distinction make sense?
Btw, @ViktorKaz has been working on the design of benchmarking workflows for a while.
The newest for sktime is in this extension proposal here: https://github.com/sktime/enhancement-proposals/pull/5
I would suggest you think of contributing and/or designing the evaluation workflow(s)?
There is an also earlier toolbox we've written for benchmarking of sklearn vanilla estimators, but it's still quite immature:
https://arxiv.org/abs/1901.03678
The idea was to re-design, and also introduce an abstract benchmarking interface for sktime at the same time.

@aiwalter, thanks for your suggestion - I think having robust functionality for quick evaluation and backtesting is a great idea.
However, in summary of my answer below:

I do not think backtesting should be a class like GridSearchCV, butpart of  an "external" benchmarking module
it would be great if you would like to contribute to building that!

So, why should backtesting not be a class like GridSearchCV?
Because backtesting, or more generally performance evaluation workflows, are not the same as tuning strategies.
A tuning strategy still is a forecaster/estimator, but a performance evaluation workflow isn't. A forecaster is subjected to a performance evaluation workflow. Both may use re-sampling (e.g., cross-validation or temporal re-sampling), but still they aren't the same kinds of formal objects.
For the same reason, sklearn's GridSearchCV should not be used for benchmarking or performance evaluation, because instances of it are representations of estimators, not evaluation workflows.
Some more detail on this, in my opinion, crucial distinction between tuning and evaluation workflows can be found in Section 5.1 of this paper on AI toolbox software design: https://arxiv.org/abs/2101.04938 (compare: anti-pattern 4; we seem to have been pre-scient here :-) )
Somewhat confusingly, the same re-sampling technique can be used in a tuning workflow as well as in an evaluation workflow.
For example, cross-validation is used inside GridSearchCV - to obtain performance estimates which are then used in tuning - and in evaluation workflows - to obtain performance estimates.
In terms of software design, the former should be a class (with a higher-order scitype), but hte latter shouldn't be, even if both use the same re-sampling technique.
Evaluation should be on the workflow layer, and calls subject to something like
benchmark(estimator, Backtesting(config), metrics)
or
Benchmarking.run() (after construction with config setting)
Does this distinction make sense?
Btw, @ViktorKaz has been working on the design of benchmarking workflows for a while.
The newest for sktime is in this extension proposal here: https://github.com/sktime/enhancement-proposals/pull/5
I would suggest you think of contributing and/or designing the evaluation workflow(s)?
There is an also earlier toolbox we've written for benchmarking of sklearn vanilla estimators, but it's still quite immature:
https://arxiv.org/abs/1901.03678
The idea was to re-design, and also introduce an abstract benchmarking interface for sktime at the same time.

@aiwalter
I believe what you would like to achieve can be simply implemented by extending adding some features to ForecastingGridSearchCV class and by creating an ExpandingWindowSplitter class.
`cv = ExpandingWindowSplitter(
initial_window=84,
window_length=12)
gscv = ForecastingGridSearchCV(
forecaster=TBATS(), param_grid={"sp": [12]}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)`
Ideally this instantiation will have a growing training set performing fitting on a dynamic training set and logging errors for each iteration. Then we may store the results and errors of each iteration in "cv_results_" as well as the mean which would provide an extensive view to the user of how each model behaves at different stages. Satisfying the proposed workflow that you have suggested. Additionally, we may extend how logging takes place in sliding window as well to capture also the behaviors as well. This would enable us to figure out whether there are any anomalies when the training set is changed and a new forecast has been made which maybe hidden from the user in the current setting just because the mean value is satisfactory.

@aiwalter
I believe what you would like to achieve can be simply implemented by extending adding some features to ForecastingGridSearchCV class and by creating an ExpandingWindowSplitter class.
`cv = ExpandingWindowSplitter(
initial_window=84,
window_length=12)
gscv = ForecastingGridSearchCV(
forecaster=TBATS(), param_grid={"sp": [12]}, cv=cv, scoring=sMAPE()
)
gscv.fit(y_train)
pd.DataFrame(gscv.cv_results_)`
Ideally this instantiation will have a growing training set performing fitting on a dynamic training set and logging errors for each iteration. Then we may store the results and errors of each iteration in "cv_results_" as well as the mean which would provide an extensive view to the user of how each model behaves at different stages. Satisfying the proposed workflow that you have suggested. Additionally, we may extend how logging takes place in sliding window as well to capture also the behaviors as well. This would enable us to figure out whether there are any anomalies when the training set is changed and a new forecast has been made which maybe hidden from the user in the current setting just because the mean value is satisfactory.

@fkiraly thanks for your comment here. Before I start coding, one more question: What is the intention of an additional  Benchmarking class? I think what we need as input for a backtest is a splitter (e.g. SlidingWindowSplitter), a forecaster, a score function (e.g. smape_loss) and the data. So would it be ok to design the interface as I wrote initially?

class Backtest(object):
    """Backtest forecasters

    Parameters
    ----------
    forecaster: estimator object
    splitter: splitter object
        e.g. SlidingWindowSplitter
    score: function
        A function like smape_loss that has y_pred, y_test as arguments and returns a float

The data could be given then in backtest.run(y)? Or how would this look like with an additional Benchmarking class?

@fkiraly thanks for your comment here. Before I start coding, one more question: What is the intention of an additional  Benchmarking class? I think what we need as input for a backtest is a splitter (e.g. SlidingWindowSplitter), a forecaster, a score function (e.g. smape_loss) and the data. So would it be ok to design the interface as I wrote initially?

class Backtest(object):
    """Backtest forecasters

    Parameters
    ----------
    forecaster: estimator object
    splitter: splitter object
        e.g. SlidingWindowSplitter
    score: function
        A function like smape_loss that has y_pred, y_test as arguments and returns a float

The data could be given then in backtest.run(y)? Or how would this look like with an additional Benchmarking class?

@aiwalter There is also the benchmarking module and an example notebook. We separate orchestration (fitting and predicting) from evaluation (metrics, uncertainty estimates, etc). But it currently only works for classification, not forecasting.

@aiwalter There is also the benchmarking module and an example notebook. We separate orchestration (fitting and predicting) from evaluation (metrics, uncertainty estimates, etc). But it currently only works for classification, not forecasting.

@mloning ok so in my idea, I planned to also fit/predict on estimators (or grid searches) inside the Backtest class to simulate (backtest) the regular workflow. If this is not the intention of your evaluation approach, I might better keep this class private?

@mloning ok so in my idea, I planned to also fit/predict on estimators (or grid searches) inside the Backtest class to simulate (backtest) the regular workflow. If this is not the intention of your evaluation approach, I might better keep this class private?

@aiwalter, @mloning, since the discussion on evaluation/benchmarking vs tuning is quite subtle - how about a phone call plus anyone involved in implementation of tuning compositors, i.e., @ngupta and @pabworks?
Source of the subtleties is that evaluation is done as part of many tuning meta-algorithms.
One possible point of abstraction here is an "evaluation spec" and a "benchmarker"; the "benchmarker" can be called with an "evaluation spec" to compare algorithms; and an abstract tuning algorithm would take an "evaluation spec" as a hyper-parameter, and internally call the "benchmarker".
The "evaluation spec" may consist of a re-sampling descriptor, a performance metric (or multiple), and some instructions on aggregation over folds.

@aiwalter, @mloning, since the discussion on evaluation/benchmarking vs tuning is quite subtle - how about a phone call plus anyone involved in implementation of tuning compositors, i.e., @ngupta and @pabworks?
Source of the subtleties is that evaluation is done as part of many tuning meta-algorithms.
One possible point of abstraction here is an "evaluation spec" and a "benchmarker"; the "benchmarker" can be called with an "evaluation spec" to compare algorithms; and an abstract tuning algorithm would take an "evaluation spec" as a hyper-parameter, and internally call the "benchmarker".
The "evaluation spec" may consist of a re-sampling descriptor, a performance metric (or multiple), and some instructions on aggregation over folds.

@aiwalter That's the same idea, the orchestrator takes in estimators (be it primitive algorithms or composites) and some CV specification and writes out prediction for each CV fold. The evaluator reads in the predictions and actual test data and computes scores, uncertainty estimates, statistical tests, etc.
@fkiraly Happy to have a call

@aiwalter That's the same idea, the orchestrator takes in estimators (be it primitive algorithms or composites) and some CV specification and writes out prediction for each CV fold. The evaluator reads in the predictions and actual test data and computes scores, uncertainty estimates, statistical tests, etc.
@fkiraly Happy to have a call

aim of the call would be to agree on interface designs for benchmarking and tuning, as well as to discuss how to "pin down" the mathematical issues here, e.g., formal content documentation of "what".

aim of the call would be to agree on interface designs for benchmarking and tuning, as well as to discuss how to "pin down" the mathematical issues here, e.g., formal content documentation of "what".

@fkiraly  @mloning just let me know about the call and I will try to join

@fkiraly  @mloning just let me know about the call and I will try to join

happy to have a call to discuss this.

happy to have a call to discuss this.

Also happy to jump on a call for this discussion.

Also happy to jump on a call for this discussion.

@fkiraly , I will be happy to get on a call to discuss this as well.

@fkiraly , I will be happy to get on a call to discuss this as well.

find a time on slack? @mloning, @pabworks, @ngupta23, @aiwalter, @viktorkaz

find a time on slack? @mloning, @pabworks, @ngupta23, @aiwalter, @viktorkaz

I do not believe I am part of this Slack channel - @mloning, I have sent you my personal email on Gitter. If you could please add me to the Slack channel, that would be much appreciated!

I do not believe I am part of this Slack channel - @mloning, I have sent you my personal email on Gitter. If you could please add me to the Slack channel, that would be much appreciated!

Created a discussion on slack to find a time that works for everyone - let me know if you're not on slack or send me a private message on Gitter.

Created a discussion on slack to find a time that works for everyone - let me know if you're not on slack or send me a private message on Gitter.

So, to recapitulate what I've understood how we've resolved this in https://github.com/sktime/enhancement-proposals/pull/8:

there should be a standalone function "evaluate" or similar for "quick and dirty" evaluation - this includes backtesting as a special case. Exact signature tbd, @aiwalter can make a proposal.
separately from that, there will be advanced benchmarking/inspection functionality in line with the orchestration module, see #64
tuning wrapper compositors - which are not the topic of this issue - will use this evaluation function internally.

From STEP01, evaluation could look like this as per @mloning's proposal:
def evaluate(forecaster, y, fh, X=None, cv=None, strategy="refit", scoring=None):

strategy should have at least "refit" and "update" available.

@fkiraly this looks fine to me. Regarding backtesting, this would work with evaluate(). Important would be what this function is returning. As it is not a class, it would probably have to return a pd.DataFrame directly instead of gscv.cv_results_ as in ForecastingGridSearchCV.
I would propose at least the following columns of the DataFrame:

All columns of  gscv.cv_results_ (maybe not the rank one)
cutoff (of the window)
start of the window (equivalent to cutoff but the first index instead of last)
y_train
y_pred

To save the y_train and y_pred is important to create plots of the evaluation (backtesting).
Any other ideas? I could work on this then @mloning ? Where do we want to locate evalute, in sktime.performance_metrics.forecasting.evaluate ?

Hm, @aiwalter, that doesn´t quite seem to fit when I think through this, since you may nave more predictions y_train than cutoffs, if you make multi-step-ahead predictions? So the different columns would have different lengths?
It also sounds like you actually want a luxury version since you want to create plots.
How about a compromise - class but with default behaviour that gives you the "shorthand" behaviour?
class BenchmarkForecast(object):
    """Backtest forecasters

    Parameters
    ----------
    forecaster: single forecaster, or list of forecasters
    splitter: splitter object to specify re-sampling
        e.g. SlidingWindowSplitter
    fh: forecasting horizon, relative; used in prediction
    benchmark_schema: string, "refit" or "update"; specifies walk-forward behaviour
    scoring: sklearn scorer object
        A function like smape_loss that has y_pred, y_test as arguments and returns a float
    averaging_fh: string or function, default=mean; specifies how to average over the forecasting horizon
    averaging_cutoff: string or function, default=mean; specifies how to average over the different cutoffs

    Methods:

    run(y, X=None)
        runs the experiment and stores intermediate results to self, e.g., sequence of losses or y_train, etc

    score(y, X=None)
        internally calls run and returns one number per forecaster (aggregate error)

    luxury_evaluate(y,X) returns cv_results, y_train etc; alternatively, get_predictions() or similar (?)
        (need to think about the name, behaviour and what is returned)
    """

@fkiraly at Mercedes we say "The best or nothing", and its all about luxury 😄 . And sktime fits to this slogan as well.
I would appreciate going for a class. We are more flexible in what we return, as we can store stuff just in self  and the user can pick what he wants to.

I would start with a stand-alone function and simply return multiple output objects.
Once this is working we can integrate into a more extensive benchmarking module. For this, I'd suggest we go with the current design in the benchmarking module, separating concerns of (i) model fitting and prediction (orchestration) and (ii) analysing/evaluating/comparing results (evaluation).

Alright, @mloning, after some thinking I agree with you:
let's go with the standalone function first, no bells and whistles, no luxury functionality.
In a second step we work on the benchmarking module, which has nice plotting, hypothesis testing, etc.
Otherwise we risk creating a halfway monstrosity, falling into a "feature proliferation" trap too early.
What do you think, @aiwalter?
After all, Rome wasn't built on one day, and even Daimler built the 35 PS before designing the Maybach...

Ok, lets go then for now for the function as I am also a fan of keeping things simple. The plot is actually a nice to have, and might be just added to a notebook and not be part of the  sktime packe. Anyway, I might try to add an optional return argument to return the data that is required for the plot.

Anyway, I might try to add an optional return argument to return the data that is required for the plot.

I've thought about that, but the problem that poses is that it is not intuitive that the return is a tuple if the user doesn't need the optional return. A compromise might be sth like return_predictions and only if this is True the return is multiple.

yes, default would not return a tuple. Same as with predict() and predict(return_pred_int=True)

makes sense to me.
It would be great if you could focus on implementing the core functionality though (e.g., make partial PR without the extra functionality), since we decided to make implementation of grid & random CV a conditionality.


Error while saving and typing

I get this error while saving with this plugin enabled:

Error: Process exited with non-zero code: 1
Error: Process exited with non-zero code: 1
at ChildProcess. (C:\Users\Luca.atom\packages\linter-pep8\node_modules\sb-exec\lib\index.js:51:22)
at emitTwo (events.js:100:13)
at ChildProcess.emit (events.js:185:7)
at maybeClose (internal/child_process.js:850:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)

I'm on Windows 10 64 bit, pep8 and python35 installed and in path

duplicate of https://github.com/AtomLinter/linter-pep8/issues/84

Duplicate of #82 actually 😉.
It's fixed in v1.3.2, sorry about that!

It's ok! It's my fault, I just pushed report issue without checking if there are similar problem...