 It picked up most of the items from it, but it didn't get the stuff in weird slots like my backpack, so i looked into restoring it from a backup. Upon testing, using /yigd grave and then restoring from backup worked, but now many of the items were duped so i decided to use /clear to wipe my inventory and restore fresh from the grave backup.
However, after running the /clear command, the grave backup is now empty. The XP levels are still present but it now says 0 items. I checked with another grave, and this is happened after running the /clear command with items from that grave in my inventory, NOT when running the restore command.
Any idea of what could have happened / ideas to fix?

I think I know what happened, and I'm sorry this has inconvenienced you. Pretty sure this is because the items are shared between the backup, and when they are retrieved. It's pretty easy to fix though.
I can't help you get your items back unfortunately, but I can try to prevent this from happening in future updates of the mod. I'll let you know when I update.


slim_handler. No module named 'django'

I got the issue during deploying: application zip package is too big and it is recommended to use slim_handler: true.
I updated zappa settings but got another issue: toml==0.9.3 was not found. I fixed the listed issue with the setting toml==0.9.4.
The project was successfully deployed, but when I try to run Django migrations(zappa manage staging migrate) I got an error No module named 'django'.
But Django is in requirements and successfully installed in venv. So during deployment with "slim_handler": true Django package is missed. Without "slim_handler": true` everything works fine.
Requirements:
awscli>=1.14.1
toml==0.9.4
zappa>=0.45.1
anyjson>=0.3.3
boto==2.48.0
boto3==1.5
botocore>=1.8.15
cached-property==1.3.1
celery==4.1.0
Django==1.11
django-appconf==1.0.2
django-braces==1.12.0
django-celery==3.2.2
django-cors-headers==2.1.0
django-countries==5.0
django-extensions==1.9.8
django-filter==1.1.0
https://github.com/philippbosch/django-geoposition/archive/django-1.11.zip #django-geoposition==0.3.1
django-import-export==0.6.1
django-location-field==2.0.3
django-material==1.1.1
django-oauth-toolkit==1.0.0
django-registration==2.3
django-rest-auth==0.9.2
django-revproxy==0.9.13
django-s3direct==1.0.4
django-smart-selects==1.5.2
django-storages==1.6.5
djangorestframework==3.6.3
djangorestframework-expiring-authtoken==0.1.4
djangorestframework-filters==0.10.2
Fabric3==1.13.1.post1
future==0.16.0
graphene>=2.0
graphene-django>=2.0
numpy==1.14
pandas==0.22
psycopg2==<IP_ADDRESS>
pyjwkest==1.4
PyJWT==1.5.3
python-social-auth==0.3.6
requests==2.18.4
social-auth-core==1.5.0
social-auth-app-django==2.0
urllib3>=1.22

Your Environment
python = 3.6
zappa = 0.45.1

There were collisions with zappa environment settings. The solution is to rename env variables S3_REGION and others to the unique one. Example is below
{
    "staging": {
       ...
        "environment_variables": {
            ...
            "DJANGO_S3_REGION": "xxx",
            "DJANGO_S3_BUCKET_NAME": "xxx",
            "DJANGO_AWS_STORAGE_BUCKET_NAME": "xxx",
            "DJANGO_S3_ASSETS_BUCKET_NAME": "xxx",
            "DJANGO_AWS_ACCESS_KEY_ID":"xxx",
            "DJANGO_AWS_SECRET_ACCESS_KEY": "xxx",
            "DJANGO_AWS_ACCESS_KEY":"xxx",
            "DJANGO_AWS_SECRET_KEY": "xxx"
        },
        "slim_handler": true,
       ...
    }
}


IDVA6-1953: Added new logging and updated existing logging

Short description outlining key changes/additions
Added new logging and updated existing logging
JIRA Ticket Number
IDVA6-1953
Type of change

[ ] Bug fix
[x] New feature
[ ] Breaking change
[ ] Infrastructure change

Pull request checklist

[x] I have added unit tests for new code that I have added
[x] I have added/updated functional tests where appropriate
[x] The code follows our coding standards

An exhaustive list of peer review checks can be read here

CI: No security warnings found

CI: No security warnings found


⚠ Bofrost.lu has degraded performance

In daff40d, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1502 ms

Resolved: Bofrost.lu performance has improved in 4816162 after 24 minutes.


Http: Connection closed while receiving data

Steps to Reproduce
I've manually written a basic server in Rust that returns some random json data. Web browser reads it fine, but when I try to get the data from Flutter, got this:
E/flutter (24806): [ERROR:flutter/lib/ui/ui_dart_state.cc(199)] Unhandled Exception: Connection closed while receiving data
The code it's just a basic http request using the dart http package.

Keep-Alive headers are included on the request, but it makes no sense have them there.

The another three SO posts haven't helped.

What I am missing?
Regards!
PD: Postman can read the server response perfectly fine, so I am assuming that it's a Flutter issue.

Can you provide a minimal reproducible code sample and the output of flutter doctor -v ?

import 'dart:convert';
import 'dart:io';
import 'package:http/http.dart';

class Server {

  static final localEndpoint = Uri.parse('http://<IP_ADDRESS>:7878/json');

  static final Map<String, String> httpHeaders = {
      HttpHeaders.contentTypeHeader: "application/json",
      "Connection": "Keep-Alive",
      "Keep-Alive": "timeout=5, max=1000"
  };

  static Future retrieveAllProducts() async {

     print("HEADERS => $httpHeaders");

    Response response = await get(
      Uri.parse('http://<IP_ADDRESS>:7878/json'), headers: Server.httpHeaders);

    var products = jsonDecode(response.body) as List;

    print("BODY => $products");

    return products;
  }
}

** Flutter doctor **
[√] Flutter (Channel stable, 2.2.3, on Microsoft Windows [VersiÃ³n 10.0.19042.1083], locale es-ES)
    • Flutter version 2.2.3 at C:\Users\Alex\AppData\Roaming\Flutter\flutter
    • Framework revision f4abaa0735 (3 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at C:\Users\Alex\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[√] Android Studio (version 4.1.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] VS Code (version 1.58.2)
    • VS Code at C:\Users\Alex\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.24.0

[√] Connected device (2 available)
    • sdk gphone x86 arm (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
    • Edge (web)                  • edge          • web-javascript • Microsoft Edge 91.0.864.70

! Doctor found issues in 2 categories.

Hi @Pyzyryab
Is this only API 30 aka Android Q?
See  this AND https://github.com/flutter/flutter/issues/32587
Can you please reproduce the issue on API 28 or 27?

Hi @TahaTesser Thanks for your response.
It's in any of the API's.

Hi @Pyzyryab
Can you please try this on latest master?
Execute the following commands to upgrade
flutter channel master
flutter upgrade --force
flutter doctor -v
Also worth trying alternate local server to be sure, for example python3 -m http.server

Error persists.
The flutter doctor -v output after upgrade, @TahaTesser
[√] Flutter (Channel master, 2.4.0-5.0.pre.146, on Microsoft Windows [VersiÃ³n
10.0.19042.1083], locale es-ES)
• Flutter version 2.4.0-5.0.pre.146 at
C:\Users\Alex\AppData\Roaming\Flutter\flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision c2f09e0638 (2 hours ago), 2021-07-22 04:06:03 -0400
• Engine revision 0507a37934
• Dart version 2.14.0 (build 2.14.0-341.0.dev)
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at C:\Users\Alex\AppData\Local\Android\sdk
• Platform android-30, build-tools 30.0.2
• Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)    • All Android licenses accepted.
[X] Chrome - develop for the web (Cannot find Chrome executable at
.\Google\Chrome\Application\chrome.exe)
! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.
[√] Android Studio (version 4.1)
• Android Studio at C:\Program Files\Android\Android Studio
• Flutter plugin can be installed from:
https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
[√] VS Code (version 1.58.2)
• VS Code at C:\Users\Alex\AppData\Local\Programs\Microsoft VS Code
• Flutter extension version 3.24.0
[√] Connected device (1 available)
• Edge (web) • edge • web-javascript • Microsoft Edge 91.0.864.71
! Doctor found issues in 1 category.

This is a very common issue, please try this https://stackoverflow.com/a/63068022

Keep-Alive was already added as a try to make it work. No success.

@Pyzyryab Do you have a server end-point (or a server we can run locally) that we can test this with? Lots of people have used these APIs without this problem so it's probably something specific about the setup but without being able to reproduce the problem there's not much we can do to debug it.

@Pyzyryab Do you have a server end-point (or a server we can run locally) that we can test this with? Lots of people have used these APIs without this problem so it's probably something specific about the setup but without being able to reproduce the problem there's not much we can do to debug it.

The local server was built on low - level Rust code manually. I solved it by just use a framework (Rocket 0.5).
I assume that something weird issue has to be on how manually are the connections dispatched...
If somebody it's interested, I can upload that code to a GitHub repo and try to catch the error.
PD: Sorry for the late response. Mail notifications was pointing to an old mail account.


[Feature] Spotted for everyone only in exploration

Would it be possible to add an option for the "Make spotted tokens visible" and "Make allied tokens visible" settings to only apply in exploration, but not combat? That way, the Point Out action actually becomes useful in an encounter. :)

I will look into it


Injecting $controller into ng-mocks specs

Hello,
I have my app up and running in the browser, (https://github.com/paynoattn/angular-paynoattn) but I can't get the spec to work.
I see in the home.spec.js you're just creating a new instance of your class to test your controller, but I'm using a function instead of a class. I can use this for my service though, which is a class, perfectly.
Based on the ng-mocks documentation (https://docs.angularjs.org/guide/unit-testing) you can inject the controller using $controller. Have you gotten this to work with that?
I don't know if this is because I'm using Typescript (it shouldn't because it's just compiling to ES6 for Babel to transpile), or something configured incorrectly but I am seeing the following error:
Error: [ng:areq] Argument 'AppController' is not a function, got undefined
      http://errors.angularjs.org/1.5.0/ng/areq?p0=AppController&p1=not%20a%20function%2C%20got%20undefined

I can see the controller properly in the spec bundle, so it should be loaded.
Here's my spec:
/// <reference path="../typings/tsd.d.ts" />

const expect = chai.expect;

import AppModule from './app.ts';
import AppService from './app-service.ts';

describe('App', function() {

  let makeController : Function;
  let makeService : Function;
  let scope : any;
  let $rootScope : any;

  beforeEach(angular.mock.module(AppModule));

  it('App Module should be properly configured with UI router', function() {
    let module = AppModule
    expect(module.requires[0]).to.equal('ui.router');
  });

  describe('Controller', () => {

    beforeEach(angular.mock.inject(function ($rootScope, $controller) {
      scope = $rootScope.$new();
      makeController = function() {
        return $controller('AppController',{
            '$scope': scope
        });
      };
    }));

    it('AppController should have social media accounts', function() {
      let controller = makeController();
      console.log(controller);
      expect(controller).to.have.property('socialMediaAccounts');
    });
  })

  describe('Service', () => {

   beforeEach(angular.mock.inject((_$rootScope_) => {
    $rootScope = _$rootScope_;
    makeService = () => {
      return new AppService();
    };
  }));

    it('AppService should have social media accounts', function() {
      let service = makeService();
      expect(service).to.have.property('socialMediaAccounts');
    })
  })
});

As I said, you can see the rest of my project here: https://github.com/paynoattn/angular-paynoattn

Okay, wow.
Easy fix, apparently.
beforeEach(angular.mock.module(AppModule));

should be
beforeEach(angular.mock.module('paynoattnApp'));

I have no idea how this is working though, since you also apparently need
let module = AppModule

For everything to work correctly. ???

I think that AppModule is instance of module, not it's name:
import angular from 'angular';

export default angular.module('app', []);

so you should use:
beforeEach(angular.mock.module(AppModule.name));

Or you could export just module name (This is what I do in my projects):
export default angular.module('app', []).name;

I see
beforeEach(angular.mock.module(AppModule.name));

works as well. Good tip. :+1:


Amendments to tutorial1 (almost final version)

Modifications to 1st tutorial for being ready to publish (hopefully minor changes needed after this stage)

overall it looks pretty good! especially I like the final visualization of monthly climatology in polar coordinates.
Just out of curiosity why are deling variables that you don't need any more? I thought it to be something like a last resort when performing memory critical operations. Is it considered clean coding?

Hi Niclas, no particular reason for deleting, rather than keeping the environment a bit cleaner and minimizing possible mistakes of using former variables for new operations (I tend to use quite similar names, so there is a risk of using a former variable before defining it again :P ) I guess it's not the best practice though, so we can modify it when we are about to publish the final version of the script.


Applied Brittany and hlint to the source code.

We now have a consistent style.

Maybe add hlint to the travis config: https://github.com/ndmitchell/hlint#running-with-continuous-integration

Needs imports of fromMaybe from Data.Maybe iirc.

@adamse That is indeed true. Sometimes I commit and push without building, this is one of these cases.

No, I didn't fix that, I just added comments so brittany would ignore the parts using Template Haskell.

@Jassob Done! I made one big commit instead of many small.


[react-core] Add hook: useBalanceForAddress

Problem solved
Changes made

[x] Public API changes: list the public API changes made if any
[ ] Internal API changes: explain the internal logic changes

How to test

[ ] Automated tests: link to unit test file
[x] Manual tests: step by step instructions on how to test

/release-pr

LGTM - thanks @kien-ngo !


Working example of a DataTable for the dashboard

Added a working example of a DataTable component in src/components/dashboard/DataTable.js.
Fixed DataView to not render more rows that gets passed through.
Also removed unused icon declarations from public/manifest.json

Added some reviewers. Check the comments and lemme know what you think. I'll pull this in and check it out soon.

If this looks good to @aorelhas and @priyajitm then we can merge ;)

Removing axios will increase performance, maybe for miliseconds but nonetheless it will.

Removing axios will increase performance, maybe for miliseconds but nonetheless it will.

Axios is just super powerful. It's great when you use it's features though. In this case a fetch was more than enough. We can even build custom hooks with fetch to extend it a bit.


Allow arrays/tables as arguments

Writing something somewhere I had an idea.
I thought we could replicate the MAPDL behaviour accepting arrays as arguments.
For examples, the command D can accept tables for the argument values:
D,Node,TEMP,%tabname%

I think we could wrap every APDL command with something like:
    def _allow_array_tables_on_args(self):
        def _upload_arg_as_array(arg):
            # to be defined
            # we might need to sanitize the name of the new array from the variable
            # name.
            return arg_name # Array name in MAPDL database

        def _upload_arg_as_table(arg):
            # to be defined
            # we might need to sanitize the name of the new array from the variable
            # name.
            return arg_name # table name in MAPDL database

        def _process_arg(arg):
            if isinstance(each_arg, np.array):
                arg =_upload_arg_as_array(arg)

            elif isinstance(each_arg, pd.DataFrame):
                arg = _upload_arg_as_table(arg)

            return arg

        def wrap_bc_listing_function(func):
            @wraps(func)
            def inner_wrapper(*args, **kwargs):

                args_ = []
                for each_arg in args:
                    arg = _process_arg(each_arg)
                    args_.append(arg)

                kwargs_ = {}
                for each_key, each_value in kwargs:
                    arg = _process_arg(each_value)
                    args_.append(arg)

                return func(*args_, **kwargs)
            return inner_wrapper

        for name in dir(self):
            if name[0:4].upper() in CMD_LISTING and name in dir(
                Commands
            ):  # avoid matching Mapdl properties which starts with same letters as MAPDL commands.
                func = self.__getattribute__(name)
                setattr(self, name, wrap_listing_function(func))

I think it could be useful because then you can do something like:
mytable = pd.Dataframe(index=[1,2,3], data = [[35],[36],[37]])
# apply temperatures 35,36 and 37 to nodes 1,2 and 3 respectively
mapdl.d("all",TEMP, mytable)

Pinging @mikerife, @mcMunich @koubaa and @pmaroneh for feedback. This an idea, I cannot work on it yet, but I'm happy to heard about this.
References:

Example for implementation: https://github.com/ansys/pymapdl/blob/c551d20e597bc67bc32bafbdb3327974166f35e4/src/ansys/mapdl/core/mapdl_core.py#L1215-L1253

Corollary
Same approach could be done for components. If a list/array is used as an argument, create the component with the parameter name and use it.
It might be not super obvious though to the user.

That is a great suggestion. Being able to use lists and/or pandas dataframes to define tabular load value would be so much easier than having to create MAPDL tables.


CacheBeta task fails with "Method Not Allowed" error during restore

Required Information
Type: Bug
Task Name: CacheBeta
Environment

Azure Pipelines, multiple builds, provided below
Hosted Agent multiple platforms:

It is a public project, you can access them from the URL.
Issue Description
VssServiceResponseException when restoring cache and the job failed
Build URL: https://dev.azure.com/cosmiafu/gugugu/_build/results?buildId=36
Job: Windows
Probable step causing the problem: Cache stack binary
Method Not Allowed when restoring cache and the job failed
Build URL: https://dev.azure.com/cosmiafu/gugugu/_build/results?buildId=36
Job: macOS
Probable step causing the problem: Cache stack binary

Also noticing the Method Not Allowed, was working ~24 hrs ago.

I'm also experiencing ##[error]Method Not Allowed on the CacheBeta@0 task when attempting to restore a cache. I tried to fix it by changing the cache key but I was unsuccessful.

I am no longer experiencing this error, however I haven't had a successful build so I cannot confirm the cache is working as it was before.

seems to be working again for me with a successful build! 🥇

We sadly still see the Method Not Allowed error on our CI, example: https://dev.azure.com/digitalasset/daml/_build/results?buildId=12863&view=logs

We are experiencing "Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown."  error thrown from CacheBeta task as well.

We're seeing the second issue since this morning too. The task version reports as 0.1.0 and the current agent version is 2.154.3.
Error message ##[error]Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown.

@Cosmius  - FYI ... I moved the throttling problem to a separate issue (#11060) to make it easier to track.

Quick update: this issue is under investigation. There is currently no known workaround. Best option right now is to temporarily disable (or remove) the task. We are working to identify the best fix and will report back ASAP.

Getting the same error, here is the log with diagnostics enabled if it can help.

##[debug]Evaluating condition for step: 'Cache Npm' |
-- | --
  | ##[debug]Evaluating: SucceededNode() |
  | ##[debug]Evaluating SucceededNode: |
  | ##[debug]=> True |
  | ##[debug]Result: True |
  | Starting: Cache Npm |
  | ============================================================================== |
  | Task         : Cache (Beta) |
  | Description  : Cache files between runs |
  | Version      : 0.1.0 |
  | Author       : Microsoft Corporation |
  | Help         : https://aka.ms/pipeline-caching-docs |
  | ============================================================================== |
  | ##[debug]Dedup parallelism: 192 |
  | Information, Using the following fingerprint argument (Windows_NT) as a string value. |
  | Information, Using the following fingerprint argument (D:\a\1\s/package-lock.json) as a file. |
  | Information, Generated fingerprint: E97BC7208F7CAE6D627D148FA65CC8A7FBFA78FBBE43D0F154056CA219B3C0DB00 |
  | Information, Getting a pipeline cache artifact with the following fingerprint: E97BC7208F7CAE6D627D148FA65CC8A7FBFA78FBBE43D0F154056CA219B3C0DB00 |
  | ##[error]Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown. |
  | ##[debug]Processed: ##vso[task.logissue type=error;]Exception of type 'Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException' was thrown. |
  | ##[debug]Processed: ##vso[task.complete result=Failed;] |
  | ##[debug]   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken) |
  | at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) |
  | at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) |
  | at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken) |
  | at Microsoft.VisualStudio.Services.PipelineCache.WebApi.PipelineCacheClient.GetPipelineCacheArtifactAsync(GetPipelineCacheArtifactOptions getOptions, CancellationToken cancellationToken) |
  | at Microsoft.VisualStudio.Services.PipelineCache.WebApi.PipelineCacheClient.<>c__DisplayClass9_0.<<GetPipelineCacheArtifactAsync>b__1>d.MoveNext() |
  | --- End of stack trace from previous location where exception was thrown --- |
  | at Microsoft.VisualStudio.Services.BlobStore.Common.Telemetry.BlobStoreClientTelemetry.MeasureActionAsync[TResult](BlobStoreTelemetryRecord record, Func`1 actionAsync) |
  | at Microsoft.VisualStudio.Services.PipelineCache.WebApi.PipelineCacheClient.GetPipelineCacheArtifactAsync(GetPipelineCacheArtifactOptions getOptions, CancellationToken cancellationToken, PipelineCacheTelemetryRecord cacheRecordOptional) |
  | at Agent.Plugins.PipelineCache.PipelineCacheServer.DownloadAsync(AgentTaskPluginExecutionContext context, IEnumerable`1 key, String path, String salt, String cacheHitVariable, CancellationToken cancellationToken) |
  | at Agent.Plugins.PipelineCache.RestorePipelineCacheV0.ProcessCommandInternalAsync(AgentTaskPluginExecutionContext context, String path, String keyStr, String salt, CancellationToken token) |
  | at Agent.Plugins.PipelineCache.PipelineCacheTaskPluginBase.RunAsync(AgentTaskPluginExecutionContext context, CancellationToken token) |
  | at Agent.PluginHost.Program.Main(String[] args)

We have implemented a workaround that allows our builds to keep running while the cache is broken, whilst at the same time keeping our build able to exploit the cache as soon as it gets fixed.
Here is how it works.

Any job that uses caching is marked as continueOnError: true (example).
Immediately following the cache step, we add a new bash step that checks if the cache step has failed; if it has, we reset the state of the current job to succeeded and create a new output variable that flags caching has failed (example).
All of the subsequent cache-related operations (unpacking, packing) are conditional on the cacheFailed variable.
At the very end of the job, we emit an output variable that says the job reached that point, i.e. was actually successful (example). This is required because the "post-job" cache action will run and fail, and because that one is post-job, we can't "catch" it with an additional script the way we did in step 2.
We added an additional job that dependsOn all the jobs that use the cache (example) and checks for that magic "job actually succeeded" output variable produced in step 4. This is required because we still want the entire job to fail if one of the jobs that had a cache failed for another reason (i.e. this partially undoes the "continueOnError" flag).

We've found an issue in the REST version negotiation logic that appears between the AT and DT stages of deployment. DT is now proceeding so, if our understanding is right, this should be working again within the hour.

This problem should be fixed now. If not, let us know. Sorry again for the inconvenience.

Confirmed to be all working here. Thanks!


Update outdated links in the readme

Closes #365
Checks to be done before submitting your PR

[ ] python3 -m pytest tests/ runs successfully.
[ ] black . runs successfully.
[ ] make html runs successfully (in the docs/ directory).
[ ] Updated docstrings related to the changes (as applicable).
[x] Added an entry to the CHANGELOG.rst.

Many thanks for spotting this Jeremy! Can you please change the links of the readme to the user guide pages, instead of the API?

It should be good now!


[BEAM-10723] ElasticsearchIO SSL authentication key fix

With current ElasticsearchIO is not possible to authenticate to ES with key. Also ElasticIO.withKeystorePath doesnt have precise name, which capture the true usage of the keystorePath. It should be trustStorePath.
Details in https://issues.apache.org/jira/browse/BEAM-10723
I also added option to trust only specific self-signed certificates.
This PR changes usage of the keystore which won't be backward compatible. Should I deprecate the method and create different one?

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @echauchot

Run Java PreCommit

Hi @dmvk, I'm just back from vacation, thanks for taking the review !

@mareksimunek can you please add the tests ?


Define a strictly typed data structure for form data

https://github.com/techequitycollaborative/tbhi-evidence/pull/15


Unable to Transcribe Files from External Storage

I was trying to transcribe .wav audio files from external storage of the android device. The app isn't returning any sort of error but the transcription being returned is not correct. It is also able to detect the file i.e. there is no "Input File Not Found" error.
Please advise on how to proceed further.

Did you able to transcribe internal files? If yes, may be your app doesn't have permission to access external files. May your file is not in 16k, mono, pcm_16/pcm_float format. Please check these.


Azure Function Error: Result: Failure Exception: TypeError: '<' not supported

I was running the API for several months. I tried to update the API today and pushed the new code to Azure and ended up with http 500 error codes. Seems to be an error in the http_asgi.py if i am not mistaken. This is the error i got from the Application Insights.
The API is working locally in the virtual environment. But when i deploy it to Azure it stopped working.
Do you know what the problem might be?
Exception while executing function: Functions.my-appname <--- Result: Failure Exception: TypeError: '<' not supported between instances of 'NoneType' and 'int' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 402, in _handle__invocation_request call_result = await self._loop.run_in_executor( File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 611, in _run_sync_func return ExtensionManager.get_sync_invocation_wrapper(context, File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/extension.py", line 215, in _raw_invocation_wrapper result = function(**args) File "/home/site/wwwroot/my-appname/__init__.py", line 230, in main return AsgiMiddleware(app).handle(req, context) File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure/functions/_http_asgi.py", line 146, in handle return self._handle(req, context) File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure/functions/_http_asgi.py", line 152, in _handle asgi_response = self._loop.run_until_complete( File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete return future.result() File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure/functions/_http_asgi.py", line 65, in from_app await app(scope, res._receive, res._send) File "/home/site/wwwroot/.python_packages/lib/site-packages/fastapi/applications.py", line 212, in __call__ await super().__call__(scope, receive, send) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/applications.py", line 119, in __call__ await self.middleware_stack(scope, receive, send) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/middleware/errors.py", line 181, in __call__ raise exc File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/middleware/errors.py", line 159, in __call__ await self.app(scope, receive, _send) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/exceptions.py", line 87, in __call__ raise exc File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/exceptions.py", line 76, in __call__ await self.app(scope, receive, sender) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/routing.py", line 659, in __call__ await route.handle(scope, receive, send) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/routing.py", line 259, in handle await self.app(scope, receive, send) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/routing.py", line 61, in app response = await func(request) File "/home/site/wwwroot/.python_packages/lib/site-packages/fastapi/routing.py", line 216, in app solved_result = await solve_dependencies( File "/home/site/wwwroot/.python_packages/lib/site-packages/fastapi/dependencies/utils.py", line 465, in solve_dependencies response = response or Response( File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/responses.py", line 50, in __init__ self.init_headers(headers) File "/home/site/wwwroot/.python_packages/lib/site-packages/starlette/responses.py", line 77, in init_headers and not (self.status_code < 200 or self.status_code in (204, 304))
Migration of https://github.com/tonybaloney/ants-azure-demos/issues/7
Originally reported by @TOAD-Joey

@TOAD-Joey Hi, thanks for reporting, is this issue still existing today with latest runtime version?


Button Component: Remove warning "Invalid prop color of value" if its value is false

Bug Report
Invalid prop color of value on Button Component
Steps
Lets say we have this component:
const myButton = (props) => {
  const buttonColor = props.state.hasErrors ? 'red' : false
  return (
    <Button
        size="tiny"
        color={buttonColor}
        content="This is a content"
        labelPosition="left"
        icon="file"
        onClick={() => {
          // do something
       }}
    />
  )
}

Rendering it, produces the  "Invalid prop color of value " warning.
Expected Result
React omits the attribute if the value you pass to it is not truthy, so i would expect the same here. Warning should be removed if color=false.
Version
2.0.3

You could use the negative prop here instead.
const myButton = (props) => {
  return (
    <Button
        size="tiny"
        negative={props.state.hasErrors}
        content="This is a content"
        labelPosition="left"
        icon="file"
        onClick={() => {
          // do something
       }}
    />
  )
}


How to get Player Clantag?

Hey, i got an idea with clantags, but i need to get a player clantag, and i dont know how to do that.
The idea: when i click 'Check' button, change my clantag to 'user_check' to a second, if another user using my version of osiris, always checking players clantag, if anyone clantag changed to 'user_check', change his clantag 'user_imchecked', and my cheat do checking always clantags, and when anyone clantag is 'user_imchecked', set a int, to +1, or get player name. (to show users in menu)
I know everything how to do, except, to every get players clantag on the server.
Can you help me? :D
Sorry for bad eng :c

You can get clan tag from CCSPlayerResource->m_szClan.

i tried to get in this way, but didnt work. i got a random number on map change or idk.
'Misc.cpp'
//Testing self clantag detect first
const auto locId = localPlayer->getUserId();

auto clant = interfaces->entityList->getEntity(locId)->GetClanTag();

std::string cmd = "echo clantag: " + std::to_string(clant);

interfaces->engine->clientCmdUnrestricted(cmd.c_str());

'Entity.h'
NETVAR(GetClanTag, "CCSPlayerResource", "m_szClan", char)

From Console:
"clantag: 0"
"clantag: 0"
...
"i not edited anything, but random change"
"clantag: 58"
"clantag: 58"
...
What is the problem?

Try defining it as char[16].

Also CCSPlayerResource is a separate entity, not a player subclass.

I changed 'Misc.cpp'
const auto locIndex = localPlayer->index();

char clant[16];

sprintf_s(clant, interfaces->entityList->getEntity(locIndex)->GetClanTag());

std::string clanstring = clant;
std::string cmd = "echo clantag: " + clanstring;

interfaces->engine->clientCmdUnrestricted(cmd.c_str());

and 'Entity.h'
NETVAR(GetClanTag, "CCSPlayerResource", "m_szClan", char[16])

I tried with real clantag, and fake clantag, but 'clant' is always nothing, and console only show up:
'clantag: '
'clantag: '
...

i tried many to make that, but didn't work :C
Please can you give me an example how to get a Player Clantag? I have many ideas with it :(

NETVAR(clanTag, "CCSPlayerResource", "m_szClan", char[65][16])

and
for (int i = interfaces->engine->getMaxClients() + 1; i <= interfaces->entityList->getHighestEntityIndex(); ++i) {
    const auto playerResource = interfaces->entityList->getEntity(i);
    if (!playerResource || playerResource->getClientClass()->classId != ClassId::CSPlayerResource)
        continue;

    if (localPlayer)
        memory->debugMsg("LOCAL CLAN: %s\n", playerResource->clanTag()[localPlayer->index()]);

    break;
}

Hey daniel, what is the CSPlayerResource ClassId?

Hey, can you help me what is the problem? I tried to get player clantag in your way, and ClassId::CSPlayerResource is didn't definied for me, i changed it to ClassId::CSPlayer, but i got nothing from console, when i delete '|| playerResource->getClientClass()->classId != ClassId::CSPlayerResource' from the if, i got random characters/game like this: ��.
Can you help me? >.<


Renaming the repo

Given that the project has somewhat expanded beyond "nyt-article-recreation", I think it might be a good time to rename the repo.  I'm open to suggestions.  Some ideas off the top of my head:

covid-growth-viz
covid-spread-viz
wi-covid-growth
pandemic-growth-visualizations

We could also create a nifty abbreviation based on what we do.

I like either a variation of wi-covid-growth or generating an spiffy acronym or portmanteau.

Renamed with the port over to https://github.com/UW-Madison-DataScience/wi-covid-growth.


ThemeProvider and React Redux actions to update/replace current theme

It would be nice if we could add features to the ThemeProvider, namely, to replace the currently used theme or to update/merge it with a new theme object using the provider itself, so that would remove the need for an additional context to achieve the aforementioned operations which would add more unnecessary complexity and affect performances.

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary 💡
We could use React redux together with the ThemeProvider to dispatch update and replace actions, that would allow easy implementation of theme toggles across the entire rendering tree since usually the ThemeProvider wraps up the entire app. This would also remove the need for developers to implement their own solutions for setting the ThemeProvider theme prop globally.

Examples 🌈

Motivation 🔦
Easier implementation of theme toggles.

I don't think that we should this problem. It would be better left to the users.

Well, the foundations are already there with the Material-UI ThemeProvider, omitting to implement these very useful features basically just forces devs to implement their own Provider on top of the Material-UI ThemeProvider in order to extend it. As a comparison, React Native Elements, is offering these features in their Theme Provider: https://react-native-elements.github.io/react-native-elements/docs/customization.html


test/k8sT: disable microscope test temporarily

There is an issue on the microscope side that is causing CI to fail. This is
tracked by #4879, and is actively being fixed. This is purely related to
microscope, and not Cilium, so it does not make sense for Cilium CI to be
blocked by the microscope test failing. Temporarily disable the test.
Signed-off by: Ian Vernon<EMAIL_ADDRESS>Fixes: #4965

This change is

test-me-please

@nebril Do we have another way of fixing it? All my CI runs have failed because of this.

After offline discussion we have decided to merge to unblock the pipeline. @nebril is still working on  fixing the issue.


Changed keyword initial to default value

In my case
.modal-body {
  overflow-y: initial; }

is changed to
.modal-body {
  overflow-y: scroll; }

That is OK, because initial stands for the default value and scroll is the default value of overflow-y.
Nevertheless, I would like to keep  initial. Is that possible?

When minifying:
.modal-body {
  overflow-y: initial; }

The result is:
.modal-body{overflow-y:initial}

It looks like your issue is not because of uglifycss.

Hm, it was only created when minimizing. But I'll take another look tomorrow. Thanks for the answer.


[BotBuilder-AI] Remove 'html-entities' from QnA Maker class

Description

Remove html-entities library and its decoding from the answer formatting.

Specific Changes
At this moment when the QnA Maker class in BotBuilder AI receives an answer that contains enconded HTML entities, said entities are decoded before returning the answer.
Remove this behaviour from the QnA Maker answer formatting, as this responsibility should be delegated to the user consuming the answer.
Testing
We did a couple of tests using both the NodeJS and .NET version of the class.
A knowledge base was created with a single answer containing several encoded HTML entities.

We tested the answer from the knowledge base by adding it to a console.log() and the sendActivity() using the Echo Browser bot sample.

As seen in the image above, the answer returned by the QnA Maker class does not match the answer stored in the knowledge base.
We also compared the behaviour of the JavaScript class against the .NET class replicating the same test.

This test confirms that the behaviour of the QnA Maker class in .NET does not decode the HTML entities before returning the answer.

Pull Request Test Coverage Report for Build #2242

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.05%) to 85.737%

Totals

Change from base Build #2236:
0.05%

Covered Lines:
3257

Relevant Lines:
3663

💛  - Coveralls

This looks reasonable to me, as that delegation seems odd.
@zerryth?

Seems good to me


Revert including ASGI root_path in url to validate

The issue I attempted to fix in #46 it turns out was due to Starlette not implementing that part of the ASGI spec correctly. path is supposed to already include root_path. It looks like this was fixed in a combination of https://github.com/encode/starlette/pull/2352 and https://github.com/encode/starlette/pull/2400 with the latter one being the change that made the incorrect fix I made start to break with Starlette >= 0.35.0.
I have updated FastAPI used in the tests to the latest version and fixed a few unrelated tests that broke because of that. I also reverted the code change I made in #46 but left the test.

oh.. fun stuff! looks good, i guess we need a tiny version bump and changelog entry?

Yes, I forgot to add those. Will do so in a few.


Incorrect scan button hitbox

Description: the hitbox for touching the scan button during the exploding star mission is too short, and doesnt reach the top of the button.
Steps to reproduce:

get the exploding star mission
notice that the button doesnt click at some Y levels

What i would expect to happen: for the button hitbox to correctly work.

[ ] Narrowed to event 2
[ ] Narrowed to 502-514
[ ] Changed event2 = math.random(1, 3) at line 89 to event2 = math.random(2, 2) to force reproduce bug
[ ] Top 2 pixels have no hitbox. Bottom pixel hitboxes are correct.  -bug reproduced
[ ] Changed at lines 502 and 507 respectively `

		if click and clickx > 10 and clickx < 59 and clicky > 70 and clicky < 77 and repairst == false then
to
		if click and clickx > 10 and clickx < 59 and clicky > 68 and clicky < 77 and repairst == false then
and
		elseif click and clickx > 10 and clickx < 59 and clicky > 70 and clicky < 77 and repairst == true then
to
		elseif click and clickx > 10 and clickx < 59 and clicky > 68 and clicky < 77 and repairst == true then

[ ] Bug confirmed fixed. Thanks for the report.


[PHP] Add @deprecated tag in PHP generated methods and services

When we deprecate services in proto-file:
syntax = "proto3";

option php_namespace = "Test";

service MyService {
  option deprecated = true;
  rpc Call(Test) returns (Test) {
    option deprecated = true;
  };
}

message Test {
}

It might be a good idea to propagate this flat in the generated classes so that the final users will be aware of the changes coming soon.
Example of the generated code:
<?php
// GENERATED CODE -- DO NOT EDIT!

namespace Test;

/**
 * @deprecated
 */
class MyServiceClient extends \Grpc\BaseStub {

    /**
     * @param string $hostname hostname
     * @param array $opts channel options
     * @param \Grpc\Channel $channel (optional) re-use channel object
     */
    public function __construct($hostname, $opts, $channel = null) {
        parent::__construct($hostname, $opts, $channel);
    }

    /**
     * @deprecated
     * @param \Test\Test $argument input argument
     * @param array $metadata metadata
     * @param array $options call options
     * @return \Grpc\UnaryCall
     */
    public function Call(\Test\Test $argument,
      $metadata = [], $options = []) {
        return $this->_simpleRequest('/MyService/Call',
        $argument,
        ['\Test\Test', 'decode'],
        $metadata, $options);
    }

}

These tags will cause IDE to highlight the class/method to bring the developer's attention.
This is consistent with how "protoc" handles the "deprecated" option for PHP

Could anyone take a look at this? Maybe I did something wrong?..
I will be happy to fix if there is an issue

@alto-php, @stanley-cheung, would it be possible to take a quick look at this, please?

Sorry for late response, LGTM, thanks for the PR.


Example code of how to force keyframe with GstForceKeyUnit?

The problem persists that the video stream from browser back to sendrecv demo is corrupted when network packet loss occurs.  The previously identified solution to this problem of forcing keyframes to be sent was given by Matthew Waters below.  However, I'm having trouble implementing this approach.  Could you provide a sample code snippet that's compatible with webrtc-sendrecv.c or webrtc-sendrecv.py that implements this suggested solution?  This would go a long way to making this demo workable.

One can force a keyframe by sending a GstForceKeyUnit request upstream on the src pad of the video stream output by webrtcbin.  This is converted internally by GStreamer's rtp implementation into a fir/pli rtcp message that is sent to the peer: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/blob/master/gst/rtpmanager/rtpsession.c#L4435 .
https://gstreamer.freedesktop.org/documentation/design/keyframe-force.html#upstream
Originally posted by @ystreet in https://github.com/centricular/gstwebrtc-demos/issues/61#issuecomment-438915464

Does gstreamer's SDP offer contain? rtcp-fb lines? e.g.:
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli

Congestion control is a really important part of WebRTC, and i would expect that webrtcbin provides it "out of the box". Otherwise, it is not really usable outside corporate networks.

Yes.   Below is my sending offer.  The received answer also has the "a=rtcp-fb:96 nack pli" line.  I did notice the performance is much better if I vpn into my corporate network and run the same demo.  I guess less packet loss that way.  I did set up and am using my own TURN server using Coturn as described here.  I'm not sure if that makes any difference.
It seems like something like the following lines should be triggered inside a conditional block in the "on_incoming_stream" function in webrtc-sendrecv.py but I'm not sure:
s = Gst.Structure("GstForceKeyUnit")
s.set_value('timestamp', timestamp, 'uint64')
s.set_value('stream-time', timestamp, 'uint64')
s.set_value('running-time', running_time, 'uint64')
s.set_value('all-headers', True)
pad.push_event(Gst.Event.new_custom(Gst.EventType.CUSTOM_DOWNSTREAM,s))

Sending offer:
v=0
o=-<PHONE_NUMBER>397648158 0 IN IP4 <IP_ADDRESS>
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS sendrecv
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 <IP_ADDRESS>
a=setup:actpass
a=ice-ufrag:IJzec+HLQlYoOP2GQFaG2MtdSWxKIjfj
a=ice-pwd:8LmS7WrSIAMjhNYlIVWZg/+ZxxT8l3nd
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=framerate:30
a=ssrc:2550752610 msid:user2509883025@host-55dfee6e webrtctransceiver0
a=ssrc:2550752610 cname:user2509883025@host-55dfee6e
a=mid:video0
a=fingerprint:sha-256 BE:56:80:74:42:02:A5:17:73:71:63:BF:13:5D:E7:31:04:91:74:D4:8C:90:09:3C:C3:F4:44:7F:3E:D8:09:DF

I do want to send a GstForceKeyUnit to upstream only if I receive a PLI from downstream webrtc viewers instead of time interval like every 10 secs
I have a setup as below:
Chrome-A(webrtc)(Sender) ->=Network=> Gstreamer(webrtcbin-A)->rtpvp8depay->tee->rtpvp8pay->webrtcbin-B->=Network=> Chrome-B(webrtc)(Reciever)
Now the Chrome-B sends a PLI request as below, I verified it from Wireshark:

...0 0001 = RTCP Feedback message type (FMT): Picture Loss Indication (1)
Packet type: Payload-specific Feedback (206)

Currently I am triggering below KeyFrame request(send_key_frame_request_to_sender) every 10 secs or at some defined interval:
void send_key_frame_request_to_sender() {
GstPad *videosrcpad;
        videosrcpad = gst_element_get_static_pad(webrtcbin-A, "videosrcpad");
        gst_pad_send_event(videosrcpad, gst_event_new_custom(GST_EVENT_CUSTOM_UPSTREAM,
                                                                 gst_structure_new("GstForceKeyUnit", "all-headers",
                                                                                G_TYPE_BOOLEAN, TRUE, NULL)));
}

But I really want trigger the above(send_key_frame_request_to_sender) only when Chrome-B(webrtc)  RTCP Feedback message type (FMT): Picture Loss Indication (1) is received
I tried to fetch the RTCP-PSFB-PLI as below from webrtcbin-B:
rtpbin = gst_bin_get_by_name(GST_BIN(webrtcbin-B), "rtpbin");
        if (rtpbin) {
            g_signal_connect (rtpbin, "on-new-ssrc", G_CALLBACK(on_new_ssrc_callback_receiver), webRtcPeerPtr.get());
            g_object_unref(rtpbin);
        }

void on_new_ssrc_callback_receiver(GstElement *rtpbin, guint session, guint ssrc, gpointer udata) {
   GstPad *rtcp_sink_pad;
    gchar *padName;
    padName = g_strdup_printf("recv_rtcp_sink_%u", session);
    rtcp_sink_pad = gst_element_get_static_pad(rtpbin, padName);
    if (rtcp_sink_pad) {
        GST_INFO("Pad Already Exists with name %s ", padName);
    } else {
        rtcp_sink_pad = gst_element_get_request_pad(rtpbin, padName);
    }
    g_free(padName);
    if (rtcp_sink_pad) {
        gst_pad_add_probe(rtcp_sink_pad, GST_PAD_PROBE_TYPE_BUFFER, (GstPadProbeCallback)
          rtcp_sink_buffer_pad_probe_cb,
                          NULL, NULL);
        gst_object_unref(rtcp_sink_pad);
    }
}

GstPadProbeReturn rtcp_sink_buffer_pad_probe_cb(GstPad *pad, GstPadProbeInfo *info, gpointer user_data) {
    GstBuffer *gstBuffer;
    gstBuffer = gst_pad_probe_info_get_buffer(info);
    gboolean result = gst_rtcp_buffer_validate(gstBuffer);
    if (result) {
        //GST_INFO("Pad Probe: RTCP Buffer valid: %d ", result);
        GstRTCPBuffer rtcpBuffer = {NULL,};
        result = gst_rtcp_buffer_map(gstBuffer, GST_MAP_READ, &rtcpBuffer);
        if (result) {
            //GST_INFO("RTCP Buffer valid: %d ", result);
            GstRTCPPacket packet;
            gboolean morePackets;
            morePackets = gst_rtcp_buffer_get_first_packet(&rtcpBuffer, &packet);
            while (morePackets) {
                GstRTCPType type;
				GstRTCPFBType gstRtcpfbType;
                type = gst_rtcp_packet_get_type(&packet);
                switch (type) {
                    case GST_RTCP_TYPE_PSFB:
					    gstRtcpfbType = gst_rtcp_packet_fb_get_type(&packet);
                        switch (gstRtcpfbType) {
                            case GST_RTCP_PSFB_TYPE_FIR:
							    GST_INFO("################# FIR Type found");
                                break;
                            case GST_RTCP_PSFB_TYPE_PLI:
                                GST_INFO("################# PLI Type found");
                                // **Actually here I want to trigger**
                               // **send_key_frame_request_to_sender**
                                break;
                            default:
                                GST_INFO ("Default /Unhandled PSFB FB Type: %d", gstRtcpfbType);
                        }
                        break;
                    case GST_RTCP_TYPE_RTPFB:
                        gstRtcpfbType = gst_rtcp_packet_fb_get_type(&packet);
                        switch (gstRtcpfbType) {
                            default:
                                GST_INFO ("Default /Unhandled RTP FB Type: %d", gstRtcpfbType);
                        }
                        break;
                    default:
                        GST_DEBUG ("Default /Unhandled RTCP Type: %d", type);
                }
                morePackets = gst_rtcp_packet_move_to_next(&packet);
            }
            gst_rtcp_buffer_unmap(&rtcpBuffer);
        }
    }
    return GST_PAD_PROBE_OK;
}

But in above switch case:GST_RTCP_PSFB_TYPE_PLI is never triggered even though Wireshark shows Chrome-B sends a PLI request.
Please let me know if this is right way to read a PLI from downstream and forward upstream for webrtc video streaming. Does GStreamer-WebRTC has any custom API to do this... or as above standard RTP/RTCP Api's are to be used?
Any suggestions would really help

@sampleref Thank you very much for your post.  I don't have an answer for your problem.  However, using/building on the code you posted, I was able to remove the video degradation problems sufficiently well for my applications (using periodic 10 second key frame pushes with posix threads).
In case it's helpful to others, just adding the following code to webrtc-sendrecv.c worked great for me for video error resiliency during network packet loss events:
#include <pthread.h>
static void
send_key_frame_request_to_sender() {
GstPad *videosrcpad;
videosrcpad = gst_element_get_static_pad(webrtc1, "src_0");
gst_pad_send_event(videosrcpad, gst_event_new_custom(GST_EVENT_CUSTOM_UPSTREAM,
gst_structure_new("GstForceKeyUnit", "all-headers",G_TYPE_BOOLEAN, TRUE, NULL)));
}
void *threadproc(void *arg)
{
while(1)
{
sleep(10);
send_key_frame_request_to_sender();
}
return 0;
}
...
(inside main)
pthread_t tid;
pthread_create(&tid, NULL, &threadproc, NULL);
...

@sampleref Thank you very much for your post. I don't have an answer for your problem. However, using/building on the code you posted, I was able to remove the video degradation problems sufficiently well for my applications (using periodic 10 second key frame pushes with posix threads).
In case it's helpful to others, just adding the following code to webrtc-sendrecv.c worked great for me for video error resiliency during network packet loss events:
#include <pthread.h>
static void
send_key_frame_request_to_sender() {
GstPad *videosrcpad;
videosrcpad = gst_element_get_static_pad(webrtc1, "src_0");
gst_pad_send_event(videosrcpad, gst_event_new_custom(GST_EVENT_CUSTOM_UPSTREAM,
gst_structure_new("GstForceKeyUnit", "all-headers",G_TYPE_BOOLEAN, TRUE, NULL)));
}
void *threadproc(void *arg)
{
while(1)
{
sleep(10);
send_key_frame_request_to_sender();
}
return 0;
}
...
(inside main)
pthread_t tid;
pthread_create(&tid, NULL, &threadproc, NULL);
...

I am glad it helped, was just trying to trigger this send_key_frame_request_to_sender on request of PLI/FIR RTCP from downstream webrtc video viewers rather than every defined interval
Hopefully there is way of doing this, will update if something found

The RTP session inside webrtcbin should already be translating PLI/FIR into GstForceKeyUnit events in the pipeline. That should show up in the GST_DEBUG=rtpsession:6 logs.
Is the peer sending FIR/PLI?

The RTP session inside webrtcbin should already be translating PLI/FIR into GstForceKeyUnit events in the pipeline. That should show up in the GST_DEBUG=rtpsession:6 logs.
Is the peer sending FIR/PLI?

Yes the Chrome-B(webrtc)(Reciever) is sending FIR/PLI and I can see that in Wireshark logs too...
But Chrome-A(webrtc)(Sender)  is not receiving it, unless I send using GstForceKeyUnit at regular intervals...
And also is the above approach correct to read/get notified of RTCP FIR/PLI requests via rtpbin?

Ah, I see. In the case where GStreamer is the receiver, the problem is related to https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/455

Ah, I see. In the case where GStreamer is the receiver, the problem is related to https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/455

Ok Thanks for pointing to it,
But until its merged or stable fix is available, can you please suggest what is the appropriate way to get hold of RTPSession from rtpbin and connect to signals on-sending-rtcp and on-receiving-rtcp  signals, so I can trigger PLI/FIR to Sender peer accordingly
Are the signals/callbacks on-new-ssrc/get-storage on rtpbin right to get hold of RTPSession with g_signal_emit_by_name(rtpbin, "get-internal-session", 0, &session);
Most of the examples I see pass 0 as argument for session id but is it same session Id received on callback signals of on-new-ssrc/get-storage?

I think what I would be doing is putting an event pad probe on the source pad from webrtcbin and looking for the packet lost messages from the rtp jitterbuffer for the stream. That will let you generate a force keyframe event in the way that the depayloader will hopefully do in the future.

Various video payloaders can produce automatic keyframe requests when they receive packet loss with GStreamer 1.20+.  Others can be added as needed.  This is a GStreamer issue though and not an issue with these examples/demos.
These examples have moved to https://gitlab.freedesktop.org/gstreamer/gst-examples/-/tree/master/webrtc and this repository will not be updated further.


Chart rendering issue with siemens/ix-echarts ^2.3.0 (Renderer 'undefined' is not imported. Please import it first.)

Prerequisites

[X] I have read the Contributing Guidelines.
[X] I have not leaked any internal/restricted information like screenshots, videos, code snippets, links etc.

What happened?
Description:
Expected Behavior
When using siemens/ix-echarts version ^2.3.0, charts should render correctly as it does with version ^1.6.0.
Actual Behavior
After upgrading to siemens/ix-echarts version ^2.3.0, charts fail to render and the following error appears in the console:
ERROR Error: Uncaught (in promise): Error: Renderer 'undefined' is not imported. Please import it first.
Error: Renderer 'undefined' is not imported. Please import it first.
    at new ZRender (vendor.js:116727:15)
    at Module.init (vendor.js:116955:12)
    at new ECharts (vendor.js:78008:78)
    at init (vendor.js:79761:15)
    at vendor.js:117222:13
    at _ZoneDelegate.invoke (polyfills.js:7543:158)
    at Zone.run (polyfills.js:7326:35)
    at polyfills.js:8418:28
    at _ZoneDelegate.invokeTask (polyfills.js:7570:171)
    at Zone.runTask (polyfills.js:7372:37)
    at new ZRender (vendor.js:116727:15)
    at Module.init (vendor.js:116955:12)
    at new ECharts (vendor.js:78008:78)
    at init (vendor.js:79761:15)
    at vendor.js:117222:13
    at _ZoneDelegate.invoke (polyfills.js:7543:158)
    at Zone.run (polyfills.js:7326:35)
    at polyfills.js:8418:28
    at _ZoneDelegate.invokeTask (polyfills.js:7570:171)
    at Zone.runTask (polyfills.js:7372:37)
    at resolvePromise (polyfills.js:8358:19)
    at polyfills.js:8268:9
    at polyfills.js:8283:25
    at asyncGeneratorStep (vendor.js:117538:5)
    at _throw (vendor.js:117557:9)
    at _ZoneDelegate.invoke (polyfills.js:7543:158)
    at Object.onInvoke (vendor.js:53313:25)
    at _ZoneDelegate.invoke (polyfills.js:7543:46)
    at Zone.run (polyfills.js:7326:35)
    at polyfills.js:8418:28

Reproduction Steps

Clone the repository: issue-echarts-renderer-siemens-ix-v2_3_0
Install dependencies: npm install
Run the application: npm start
Observe the console for the error and the line chart not rendering.

Additional Context

The issue does not occur with siemens/ix-echarts version ^1.6.0.
The error suggests that a renderer is not imported, but it is unclear which renderer is required or how to import it.

Please let me know if any additional information is needed.
What type of frontend framework are you seeing the problem on?
Angular
Which version of iX do you use?
2.5.0
Code to produce this issue.
https://github.com/sakethsusarla/issue-echarts-renderer-siemens-ix-v2_3_0

Hello @sakethsusarla! The problem is in your app.module.ts, according to the ngx-echarts documentation (https://github.com/xieziyu/ngx-echarts?tab=readme-ov-file#ngmodule), the NgxEchartsModule should be imported like:
 NgxEchartsModule.forRoot({ echarts: () => import('echarts'), }),

Many thanks 😅, the issue got fixed with this change.


Fix docker-compose run on linux machine

The script within is failed on linux with the following command:
~/badoo# docker-compose up infinite-scheduler
Creating network "badoo_default" with the default driver
Creating selenium-hub ... done
Creating badoo_chrome-node_1 ... done
Creating badoo-infinite-scheduler ... error

ERROR: for badoo-infinite-scheduler  Cannot start service infinite-scheduler: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/code/docker-entry.sh\": stat /code/docker-entry.sh: no such file or directory": unknown

ERROR: for infinite-scheduler  Cannot start service infinite-scheduler: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/code/docker-entry.sh\": stat /code/docker-entry.sh: no such file or directory": unknown
ERROR: Encountered errors while bringing up the project.

Fixed by upgrading docker instance via:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get install docker-ce docker-ce-cli containerd.io


Signature misalignment of Directory.with_new_file and Container.with_new_file

What is the issue?
In the Python SDK, the signatures of Directory.with_new_file and Container.with_new_file are inconsistent. For Directory, contents is a required positional argument, while for Container, contents is a keyword-only argument. This inconsistency is confusing and often requires referencing the API documentation to determine the correct usage. I vote for setting up a PR, if you could give me a hint on the preferred behaviour.
Dagger version
dagger v0.10.2 (registry.dagger.io/engine) darwin/arm64
Steps to reproduce
No response
Log output
No response

Yeah, that's a good point. I think the Directory version, with contents being a required argument is preferred, but it's not a Python change, it's a core API change.

Just to explain the reason why, we've changed Directory but not Container, because of:

https://github.com/dagger/dagger/issues/3884


[PR-Agent] New fundraising banner and in-app webview donation page

Phabricator: https://phabricator.wikimedia.org/T333108 & https://phabricator.wikimedia.org/T346823
Notes

This PR updates the UI and actions on our fundraising modal - now the user can choose to be reminded at a later date
It also keeps the donation flow within the app, opening the pages from an in-app webview
This PR depends on Apple Pay - finalizing authorization sheet and thank you toast presentation #4641

Test Steps

Run the experimental app, open it once, background it, and then open it again
Set your region to the Netherlands - I was hard-coding "NL" on all methods that verify the country code
Add Dutch to your languages
Test closing using the "X" button or the "already donated" button (the last one) - it should not show the modal again
Delete the app, reinstall, repeat step 1
Select maybe later. Check if the reminder toast shows.
Visit the article again. it should not show the modal.
Change the date on your phone to tomorrow. Visit the article, it should show the modal.
Reinstall and uninstall, repeat step 1. This time, donate.
Verify that the back to article button shows on the bottom of the Thank you page.
Close the web view and verify that the thank you toast shows

Preparing review...

PR Review

 ⏱ Estimated effort to review [1-5]
4, because the PR introduces significant changes across multiple files, including new UI components, logic for handling donation flows, and updates to localization. The complexity of the changes requires thorough testing and validation.

 🧪 Relevant tests
No

 🔍 Possible issues
Possible Bug: The method showShowMaybeLaterOption has a typo in its name. It should likely be shouldShowMaybeLaterOption.

 🔒 Security concerns
No

 Code feedback:
relevant fileWKData/Sources/WKData/Data Controllers/Donor Experience/WKFundraisingCampaignDataController.swift
suggestion

The method name showShowMaybeLaterOption seems to have a typo. It should be renamed to shouldShowMaybeLaterOption to better reflect its purpose and follow naming conventions. [important]

relevant linepublic func showShowMaybeLaterOption(asset: WKFundraisingCampaignConfig.WKAsset, currentDate: Date) -> Bool {
relevant fileWikipedia/Code/SinglePageWebViewController.swift
suggestion

The setupBottomView method should be called only if didReachThankyouPage is true. Consider moving the call to setupBottomView inside the if block to avoid unnecessary method calls. [medium]

relevant linesetupBottomView()
relevant fileWikipedia/Code/ArticleViewController+Announcements.swift
suggestion

In the showNewDonateExperienceCampaignModal method, consider using Date() instead of .now for consistency and clarity. [medium]

relevant linelet shouldShowMaybeLater = dataController.showShowMaybeLaterOption(asset: asset, currentDate: Date())
relevant fileWikipedia/Code/SinglePageWebViewController.swift
suggestion

The didTapReturnButton method should dismiss the view controller instead of popping it to ensure a consistent user experience. Use self.dismiss(animated: true) instead of navigationController?.popViewController(animated: true). [important]

relevant linenavigationController?.popViewController(animated: true)

 ✨ Review tool usage guide:
Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:
/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:
[pr_reviewer]
some_config1=...
some_config2=...

  Utilizing extra instructions
The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.
Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.
Examples for extra instructions:
[pr_reviewer] # /review #
extra_instructions="""
In the 'possible issues' section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

  How to enable\disable automation

When you first install PR-Agent app, the default mode for the review tool is:

pr_commands = ["/review", ...]

meaning the review tool will run automatically on every PR, with the default configuration.
Edit this field to enable/disable the tool, or to change the used configurations

  Auto-labels
The review tool can auto-generate two specific types of labels for a PR:

a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

  Extra sub-tools
The review tool provides a collection of possible feedbacks about a PR.
It is recommended to review the possible options, and choose the ones relevant for your use case.
Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example:
require_score_review, require_soc2_ticket, require_can_be_split_review, and more.

  Auto-approve PRs
By invoking:
/review auto_approve

The tool will automatically approve the PR, and add a comment with the approval.
To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:
[pr_reviewer]
enable_auto_approval = true

(this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)
You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag:
[pr_reviewer]
maximal_review_effort = 5

  More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs 💎: Generate docstring for new components introduced in the PR.
/generate_labels 💎: Generate labels for the PR based on the PR's contents.
/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

Preparing review...

PR Review

 ⏱ Estimated effort to review [1-5]
4, このPRは多くのファイルと複雑なロジックを含んでいるため、レビューにはかなりの時間と労力が必要です。特に、新しい機能の追加と既存コードの変更が組み合わさっているため、注意深くチェックする必要があります。

 🧪 Relevant tests
No

 🔍 Possible issues

Possible Bug: showShowMaybeLaterOption メソッドの名前が誤解を招く可能性があります。showMaybeLaterOption という名前が適切かもしれません。

Performance Concern: ArticleViewController+Announcements.swift で、showNewDonateExperienceCampaignModal メソッド内で毎回 WKFundraisingCampaignDataController の新しいインスタンスを作成しています。これはパフォーマンスに影響を与える可能性があります。
 🔒 Security concerns
No

 Code feedback:
relevant fileWKData/Sources/WKData/Data Controllers/Donor Experience/WKFundraisingCampaignDataController.swift
suggestion

showShowMaybeLaterOption メソッドの名前を showMaybeLaterOption に変更することを提案します。これにより、メソッドの目的がより明確になり、誤解を避けることができます。 [important]

relevant linepublic func showShowMaybeLaterOption(asset: WKFundraisingCampaignConfig.WKAsset, currentDate: Date) -> Bool {
relevant fileWikipedia/Code/ArticleViewController+Announcements.swift
suggestion

WKFundraisingCampaignDataController のインスタンスを ArticleViewController のプロパティとして保持し、再利用するように変更してください。これにより、メソッドが呼び出されるたびに新しいインスタンスを作成する必要がなくなり、パフォーマンスが向上します。 [important]

relevant linelet activeCampaignAsset = WKFundraisingCampaignDataController().loadActiveCampaignAsset(countryCode: countryCode, wkProject: wkProject, currentDate: .now) {

 ✨ Review tool usage guide:
Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:
/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:
[pr_reviewer]
some_config1=...
some_config2=...

  Utilizing extra instructions
The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.
Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.
Examples for extra instructions:
[pr_reviewer] # /review #
extra_instructions="""
In the 'possible issues' section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

  How to enable\disable automation

When you first install PR-Agent app, the default mode for the review tool is:

pr_commands = ["/review", ...]

meaning the review tool will run automatically on every PR, with the default configuration.
Edit this field to enable/disable the tool, or to change the used configurations

  Auto-labels
The review tool can auto-generate two specific types of labels for a PR:

a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

  Extra sub-tools
The review tool provides a collection of possible feedbacks about a PR.
It is recommended to review the possible options, and choose the ones relevant for your use case.
Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example:
require_score_review, require_soc2_ticket, require_can_be_split_review, and more.

  Auto-approve PRs
By invoking:
/review auto_approve

The tool will automatically approve the PR, and add a comment with the approval.
To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:
[pr_reviewer]
enable_auto_approval = true

(this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)
You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag:
[pr_reviewer]
maximal_review_effort = 5

  More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs 💎: Generate docstring for new components introduced in the PR.
/generate_labels 💎: Generate labels for the PR based on the PR's contents.
/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.


How to forward only some critical alerts from Nagios to Pagerduty

We're looking at Alerta (big fan) but have run into a problem. Because we're forwarding events from Nagios to Pagerduty, they show up as critical but they don't all page/notify. I'm wondering if there is a way to pass a critical alert to Alerta that doesn't trigger a notification flow with the existing Pagerduty integration. Thanks for your time.

Hi, can I clarify a few things?
Is the situation at present that Nagios sends notifications directly to PagerDuty? And do only some of the critical alerts you get in Nagios trigger PagerDuty notifications? That is...
Current?
Nagios => PagerDuty
Would you like Alerta to receive Nagios alerts and for Alerta to trigger notifications to PagerDuty? And if so, do you want control over which alerts from Nagios that are sent to Alerta actually trigger notifications with PagerDuty? That is...
Future?
Nagios => Alerta => PD integration => PagerDuty

With the limited information I have so far I can only suggest you use a "routing rule" to dynamically determine which alerts should be notified based on severity or any other alert attribute. See https://github.com/guardian/alerta/tree/master/contrib/routing


Overlapping instances?

Hmm.  Trying to write slightly more general interpreters for running things that might be sitting "on top of" other monads where all we have are constraints for them.  Should that work?  I am getting
 • Overlapping instances for Polysemy.Internal.Union.Find
                                  '[P.Lift IO, P.Lift m] (P.Lift m)
        arising from a use of ‘KPM.runPandoc’
      Matching instances:
        instance forall k (z :: [k]) (t :: k) (_1 :: k).
                 (Polysemy.Internal.Union.Find z t,
                  Polysemy.Internal.Union.Found (_1 : z) t
                  ~ 'Polysemy.Internal.Union.S
                      (Polysemy.Internal.Union.Found z t)) =>
                 Polysemy.Internal.Union.Find (_1 : z) t
          -- Defined in ‘Polysemy.Internal.Union’
        instance [overlapping] forall k (t :: k) (z :: [k]).
                               Polysemy.Internal.Union.Find (t : z) t
          -- Defined in ‘Polysemy.Internal.Union’
      (The choice depends on the instantiation of ‘m’
       To pick the first instance above, use IncoherentInstances
       when compiling the other instance declarations)
    • In the expression: KPM.runPandoc @m
      In an equation for ‘runPandoc’: runPandoc = KPM.runPandoc @m
      In the expression:
        do let runPandoc ::
                 forall a.
                 Semantic '[Pandoc,
                            KLog.Logger KLog.LogEntry,
                            KLog.PrefixLog,
                            PE.Error KPM.PandocError,
                            P.Lift IO,
                            P.Lift m] a
                 -> Semantic '[KLog.Logger KLog.LogEntry,
                               KLog.PrefixLog,
                               PE.Error KPM.PandocError,
                               P.Lift IO,
                               P.Lift m] a
               runPandoc = KPM.runPandoc @m
...

Can you share all the code?

Yes, but it's..a lot.
For now, I retreated to specifying PandocIO as m.  Once that works, I'll go back and re-create, check in a branch with the error and point you to that, okay?   But it might not be for a bit...
Thanks for trying to help!

Okay.  Here's the code which won't compile (the master branch has the same thing with PandocIO specified and one more run function to run PandocIO, which maybe makes the difference?):
https://github.com/adamConnerSax/knit-haskell/blob/OverlappingInstances/src/Knit/Report.hs#L145
src/Knit/Report.hs:145:7-22: error:
    • Overlapping instances for Polysemy.Internal.Union.Find
                                  '[P.Lift IO, P.Lift m] (P.Lift m)
        arising from a use of ‘KPM.runPandoc’
      Matching instances:
        instance forall k (z :: [k]) (t :: k) (_1 :: k).
                 (Polysemy.Internal.Union.Find z t,
                  Polysemy.Internal.Union.Found (_1 : z) t
                  ~ 'Polysemy.Internal.Union.S
                      (Polysemy.Internal.Union.Found z t)) =>
                 Polysemy.Internal.Union.Find (_1 : z) t
          -- Defined in ‘Polysemy.Internal.Union’
        instance [overlapping] forall k (t :: k) (z :: [k]).
                               Polysemy.Internal.Union.Find (t : z) t
          -- Defined in ‘Polysemy.Internal.Union’
      (The choice depends on the instantiation of ‘m’
       To pick the first instance above, use IncoherentInstances
       when compiling the other instance declarations)
    • In the first argument of ‘(.)’, namely ‘KPM.runPandoc @m’
      In the second argument of ‘(.)’, namely
        ‘KPM.runPandoc @m
           . maybe id wrapPrefix loggingPrefixM
               . KD.toNamedDocListWithM
                   (fmap BH.renderHtml . KO.toBlazeDocument writeConfig)’
      In the second argument of ‘(.)’, namely
        ‘filteredLogEntriesToIO ls
           . KPM.runPandoc @m
               . maybe id wrapPrefix loggingPrefixM
                   . KD.toNamedDocListWithM
                       (fmap BH.renderHtml . KO.toBlazeDocument writeConfig)’
    |
145 |     . KPM.runPandoc @m -- PA.PandocIO
    |       ^^^^^^^^^^^^^^^^

Thanks for looking!  I've got no idea how to think about that error.

This looks odd to me:
``haskell
type KnitEffectsMany m =
'[ KD.Docs KP.PandocWithRequirements
, KPM.Pandoc
, KLog.Logger KLog.LogEntry
, KLog.PrefixLog
, PE.Error PA.PandocError
, P.Lift IO
, P.Lift m
]
type KnitEffectsManyC m = (P.Member (P.Lift m) (KnitEffectsMany m))
`KnitEffectsManyC m` says that `P.Lift m` is a `Member` of `KnitEffectsMany m`, but `P.Lift m` is already explicitly in `KnitEffectsMany m`. Was that necessary to get things to type check?

Yeah, something definitely weird is going on here. For example, you have a function which takes a forall a . P.Semantic r a -> P.Semantic (KnitEffectsMany m) a... and then just runs it. Why not just ask callers of your function to call that first?
Furthermore, as @jhenahan points out,
type KnitEffectsOne m =
  '[ KP.ToPandoc
   , KPM.Pandoc
   , KLog.Logger KLog.LogEntry
   , KLog.PrefixLog
   , PE.Error PA.PandocError
   , P.Lift IO
   , P.Lift m
   ]

type KnitEffectsOneC m = (P.Member (P.Lift m) (KnitEffectsMany m))

this thing is wild.   It's weird.  But without it I get:
src/Knit/Report.hs:145:7: error:
    • Ambiguous use of effect 'P.Lift'
      Possible fix:
        add (Member (P.Lift m) '[KLog.Logger KLog.LogEntry,
                                 KLog.PrefixLog, PE.Error KPM.PandocError, P.Lift IO,
                                 P.Lift m]) to the context of
          the type signature
      If you already have the constraint you want, instead
        add a type application to specify
          'm' directly
    • In the first argument of ‘(.)’, namely ‘KPM.runPandoc @m’
      In the second argument of ‘(.)’, namely
        ‘KPM.runPandoc @m
           . maybe id wrapPrefix loggingPrefixM
               . KD.toNamedDocListWithM
                   (fmap BH.renderHtml . KO.toBlazeDocument writeConfig)’
      In the second argument of ‘(.)’, namely
        ‘filteredLogEntriesToIO ls
           . KPM.runPandoc @m
               . maybe id wrapPrefix loggingPrefixM
                   . KD.toNamedDocListWithM
                       (fmap BH.renderHtml . KO.toBlazeDocument writeConfig)’
    |
145 |     . KPM.runPandoc @m -- PA.PandocIO
    |       ^^^^^^^^^^^^^^^^

I actually get that (Ambiguous use of P.Lift...) a lot, even when I have the constraint and the type application, which makes me think I am doing something wrong only I can't spot what it is.
@isovector You're right about the runOthers function.  I'm trying to make a function that is kind of "one-stop-shopping" for running this set of effects.  I'm imagining a user who uses this set of features and doesn't want to know more than they have to about how polysemy works.  So they might want to run it on top of another monad. But also have other effects, like randomness, that this function doesn't handle.  But you're right, I don't add much value by just running a function they have to provide.  So I dropped it.
I tried your constraint--it would be cool if that worked!  I thought you might have to specify an order to actually run things--but then I get Ambiguous use of effect 'P.Lift' at both places where I type-apply m and I don't know why.
I've checked that version in now, in case it's of interest.
Thanks!

Okay, so looking at what your code is actually doing (at least at a high level), here's what I think is going on.
knitHtmls interprets the effects listed in KnitEffectsMany (a whole bunch of them!). The way I intuitively think of the "visible" part of interpretation is that you're stripping an effect off the top of r. So the issue, I think, is that your type says "I take a Semantic with some arbitrary effect stack (that I can't look at, because forall r), and I also handle interpretation of all of these other effects". But then the constraint on that type says that you have an unhandled KnitEffectsMany (that is, a stack of unhandled effects) in that r you can't look at. But that's not really what you want to say, I don't think.
Really, for any interpreter on which you can runM and get a result out, all you can have left is a Semantic '[Lift m] a, which I think is what you have. The interpreter obligations for KnitEffectsMany are entirely discharged by your chain of interpreters.
To that end, I think your actual type looks more like
knitHtmls
  :: ( PA.PandocMonad m
     , MonadIO m
     , MonadError PA.PandocError m
     )
  => Maybe T.Text -- ^ outer logging prefix
  -> [KLog.LogSeverity] -- ^ what to output in log
  -> PandocWriterConfig -- ^ configuration for the Pandoc Html Writer
  -> P.Semantic r () -- ^ Knit effects "over" m
  -> m [KP.NamedDoc TL.Text] -- ^  named documents, converted to Html as Text.

(I didn't check if that typechecks, so it's more of a feely guess)
I think that'd also let you strip out the @ms.
You still don't know anything about r, but the interpreters that your user chains in here would admit additional effects (though I don't really know what happens if they use their own interpretation of one of the effects you handle). I think this feels weird because this interpreter doesn't compose with so much as it consumes another interpreter, which seems odd.
If the interpreter is built with reinterpret (and so handles its own effects), your type doesn't change, so that'd be fine. But if you want to allow some arbitrary interpreter, then you're going to run into the issue of people interpreting your effects some other way, and I have no idea how to write a constraint for "might have any effect at all". On the other hand, this does let you say explicitly what the chained interpreter can do, by expressing Member SomeEffect r for the r in that Semantic r ().

Hmm. I don’t get how that can work without constraints on r that allow all the interpreters to be run. They each have constraints and/or types that indicate specific effects being in r. But maybe that’s what you’re saying at the end?
But I think I’m being dumb. If I know (MonadIO m), I can just run my (working!) version using PandocIO giving something in IO and then use liftIO to get to m. So what I though was a requirement for running this set of effects in a larger environment isn’t one, right?
I think I got lost trying to be general instead of trying to have the function do what it says, use a specific set of effects to create a set of html docs (in IO, since Pandoc requires that bit).
I’ll close this soon, I’m away from a laptop for a bit...
Anyway, thanks for all the help! I definitely still don’t quite get what’s happening but I’m pretty sure this doesn’t need to work in this complicated way.
Thanks!
Adam
On Apr 19, 2019, 10:09 PM -0400, Jack Henahan<EMAIL_ADDRESS>wrote:

Okay, so looking at what your code is actually doing (at least at a high level), here's what I think is going on.
knitHtmls interprets the effects listed in KnitEffectsMany (a whole bunch of them!). The way I intuitively think of the "visible" part of interpretation is that you're stripping an effect off the top of r. So the issue, I think, is that your type says "I take a Semantic with some arbitrary effect stack (that I can't look at, because forall r), and I also handle interpretation of all of these other effects". But then the constraint on that type says that you have an unhandled KnitEffectsMany (that is, a stack of unhandled effects) in that r you can't look at. But that's not really what you want to say, I don't think.
Really, for any interpreter on which you can runM and get a result out, all you can have left is a Semantic '[Lift m] a, which I think is what you have. The interpreter obligations for KnitEffectsMany are entirely discharged by your chain of interpreters.
To that end, I think your actual type looks more like
knitHtmls
:: ( PA.PandocMonad m
, MonadIO m
, MonadError PA.PandocError m
)
=> Maybe T.Text -- ^ outer logging prefix
-> [KLog.LogSeverity] -- ^ what to output in log
-> PandocWriterConfig -- ^ configuration for the Pandoc Html Writer
-> P.Semantic r () -- ^ Knit effects "over" m
-> m [KP.NamedDoc TL.Text] -- ^  named documents, converted to Html as Text.
(I didn't check if that typechecks, so it's more of a feely guess)
I think that'd also let you strip out the @ms.
You still don't know anything about r, but the interpreters that your user chains in here would admit additional effects (though I don't really know what happens if they use their own interpretation of one of the effects you handle). I think this feels weird because this interpreter doesn't compose with so much as it consumes another interpreter, which seems odd.
If the interpreter is built with reinterpret (and so handles its own effects), your type doesn't change, so that'd be fine. But if you want to allow some arbitrary interpreter, then you're going to run into the issue of people interpreting your effects some other way, and I have no idea how to write a constraint for "might have any effect at all". On the other hand, this does let you say explicitly what the chained interpreter can do, by expressing Member SomeEffect r for the r in that Semantic r ().
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Okay, maybe I was 0.5 dumb. liftIO solves part of my problem but I’d still prefer a version of the original idea to work so that the thing knitHtmls is called on can have access to m. I’ll keep fiddling...

Having had more time to think, and you all to point out what’s weird, I think I can ask this more clearly.  Sem r x -> Sem ‘[Lift m] x) -> SemT r m x -> m x
runSemT consume = runM . consume

where that run interprets/consumes all the effects in r and can use m to do it.
I’m happy to abandon the mtl-style stuff but I’m trying to figure out how to interoperate with it most smoothly.
That is, I’m trying to figure a way that a function using the polysemy effects can also access effects in another monad, at least some of them...
Does that make sense?

SemT r m is indistinguishable from Sem r -- it's just a Member (Lift m) r constraint.
You can also double down on constraints for those things:
-- MonadResource is from the `resourcet` package
foo :: forall m r. (Member (Lift m) r, MonadResource m) => Semantic r ReleaseKey
foo = do
  (key, ()) <- sendM $ allocate @m (putStrLn "hello") (const $ putStrLn "goodbye")
  pure key

Hmm.  Maybe I need a simpler question answered:
Given that I have a Member (Lift m) r constraint and that I have type-applied @m on the line in question, what is the "Ambiguous use of effect 'P.Lift'..." message telling me?
I'd like to chase it down and fix it or understand why it's impossible.
I know that maybe I'm doing something silly or impossible but I don't yet get why. I'm just trying to run all the effects in a stack with Lift m at the bottom and be left with m a which seems like it ought to be easy.  The only wrinkles here are the use of m in the interpreter.  But as you point out, that shouldn't be a problem.  So what is going wrong?

Sorry, I'm definitely bogged down in the details here. That being said, I'm 65% sure the issue comes from explicitly having KnitEffectStack rather than allowing the polymorphism to do what it's supposed to.
Try disabling custom errors in polysemy via adding this to your stack.yaml:
flags:
  polysemy:
    custom-errors: false

and see if that gives a more helpful error message for your ambiguous lift.

Don't be sorry!  I so appreciate all the help.
I got the explicit HList version to work.  I had defined LastMember wrong.  But I know it feels like a code-smell that I need it, or the explicit HList version at all.
Code is here if you are interested.  Example use
Things can be inferred at use-sites so it's not so bad that the interpreter needs the list to be explicit, maybe?
When I try the polymorphic version of consumeKnitEffectStack I get (without the "error-messages" flag) a "Could not deduce r ~ XXX" where XXX is the stack I was using explicitly in the monomorphic version. So that seems to push back towards the explicit HLIst version.
I am planning to remove PandocIO from this all-together pretty soon since it's just a combination of MonadError, IO and some MonadRandom.  So I will interpret it all via those.  And then maybe this will be simpler and I can figure out how to get the polymorphic version to work...Until then, should I close this?
In general, should type inference be powerful enough to figure out the HList required for a set of effects to be interpreted/consumed? If so, something in one of my interpreters must break this, requiring more ordering information or something.  It would be good to know where that breaks down here...

I guess the thing I'm failing to grasp is why you need to have the explicit KnitEffectsStack. As I understand it, your r is polymorphic (i.e., the order of effects is not fixed) until you start consuming the effects. Your KnitEffectsStack seems to impose an ordering on r, which feels like it's working against the library rather than with it.

Right!  I would also rather not have it be explicit.  So your question confuses me.  My best guess is that you could be asking one of two things:

Why, in these "knit" functions, am I using an explicit HList instead of a polymorphic r?  The answer there is that I would if I could!  But when I try I get, depending on the use of custom-errors, a failure to deduce the exact stack or an ambiguous use of Lift m.  My rough understanding of this is that my series of interpreters require Member (Lift m) r in a few different stacks, as each effect is run/interpreted.
But maybe you are asking why I want to run/interpret these effects over an arbitrary Monad m?  That's because I am imagining a use-case where someone has an existing monad stack and wants to use my library to create Html reports while working in that stack.  And they might want to use their monad to, e.g., make DB calls, while building the document.  That is, I am trying to make this flexible and usable by someone who isn't interested in replacing their entire stack with polysemy but wants to use this package to create Html reports this way.
@jhenahan Does that make sense?

It might be clearer with an example.  If you have a chance, take a look at:
https://github.com/adamConnerSax/knit-haskell/blob/master/examples/MtlExample.hs
That shows use of the polysemy stack being run (via knitHtml) "over" an mtl stack in a case where liftIO wouldn't be sufficient because the function that uses the polysemy effects also uses effects from that base monad.
I want this to work (which it does now! But using an explicit stack and that LastMember thing in the intepreter...) for any user who wants to be able to use this functionality without converting an existing app to use polysemy.

I'll dig around this week/end and see if I can figure something out, assuming the author doesn't beat me to it. :smile:

I also want this to work! Is there any chance I can persuade you into putting together a single-file simplification of your code, which only has polysemy/base/containers/transformers/mtl as dependencies? I'm sure this would be easy to track down if I could see the exact errors in context.
Alternatively we could set up a call and pair on it if you'd like!

Thank you both so much for being so helpful! This particular thing has gotten simpler as I've worked on it and I am down to just the polymorphic vs. explicit type-level-list of effects.
( I re-implemented the PandocIO monad just using IO--because it seemed very annoying to ask users to put PandocIO at the bottom of their stacks!--and the the necessity of the LastMember thing went away. Not sure why...)
I'm happy to try and replicate it in a simpler context.  But let me summarize where I am and what questions remain just to be sure it's worth solving and I know what I'm trying to replicate!

I have successfully written an (interpreter?  run-function? effect-consumer?) which:
a. Consumes a fixed set of effects, some of which require IO.
b. Returns a base monad (non-polysemy), via runM, said monad having a MonadIO instance.
c. Allows the effectful thing being "run" to access the base monad via the standard mechanism of using sendM (and the presence of Lift m in the TLL).
Said interpreter requires, at least in my version, a fixed type-level-list (TLL) of effects (rather than a polymorphic TLL and constraints) in it's signature.
If I try to use a polymorphic TLL and constraints I get a "could not deduce r ~ XXX..." error, where XXX is exactly the stack I specify in the working version.
I can run this interpreter on effectful things that are specified polymorphically, as demonstrated in the example linked in my last post.

So, on the one hand, I'm not sure there's still a problem.  I have bent it, more or less, to my grasping and ineffectual will.  But if it still seems to you both that I ought to be able to write this interpreter with a polymorphic type in the signature--though it can only possibly work if that type is inferred to be the precise stack I now have to specify! Which I think is why the polymorphic version fails: the function isn't really polymorphic.--then I can try to replicate the issue.  Which is just a question of what circumstances require explicit TLL's vs. polymorphic + constraints.
Also, the LastMember thing.  That went away but I'm not really sure why.  So I'm not sure I can reproduce the circumstances that require it!  It likely has something to do with running multiple single-effect interpreters some of which use sendM to use the base monad.  But why that's become simpler when it's MonadIO m rather than (MonadIO m, PandocMonad m), I'm not sure.  It also could be a difference between the Pandoc effects interpreter I used when I was relying on the base monad for Pandoc effects and the one I wrote as a re-implementation of that in IO.

Ahh, thanks for spelling that out. That makes a lot of sense to me! The reason you have a monomorphic type is because really and truly you have a set of interpreters you want to run in that order. In that case, I agree that you need the concrete type.
I'm also happy that whatever changes you've made have alleviated the need for LastMember! That thing always struck me as a wart in freer-simple.
If you're happy with the state of things, I'm also happy! It seems like there could be some work done on the library side to improve the ergonomics of typing explicit interpreter compositions, but besides that I'm not sure! Any suggestions for what else this bug has to offer, or is it OK to close?

I'm content to close this and I'l keep an eye out to see if I can understand why that LastMember thing happened.
Thanks again for all the help!